More balance is better, without limit. The only qualification is that as
you get better balance you lose observations. Losing some observations may
reduce heterogeneity and thus variance, but losing more will increase
variance. So there's a trade off between imbalance and sample size
remaining.
There are lots of imbalance measures. Some are discussed in our
original matchit
paper <http://j.mp/jPupwz>. A newer more comprehensive measure is L1, which
is a measure of the difference in the multivariate histograms between the
control and treated groups. See, e.g. the current Political
Analysis<http://j.mp/iUUwyH> or
this paper <http://j.mp/jCpWmk> which enables you to address the trade off
between imbalance and sample size left.
Gary
--
*Gary King* - Albert J. Weatherhead III University Professor - Director,
IQSS - Harvard University
GKing.Harvard.edu <http://gking.harvard.edu/> - King(a)Harvard.edu -
@kinggary<http://twitter.com/kinggary>- 617-500-7570 - Asst 495-9271 -
Fax 812-8581
On Tue, Aug 23, 2011 at 11:37 AM, Paul Diterwich <paul(a)diterwich.com> wrote:
First my complements to all contributors of MatchIt,
you really made my
life easier!
Now to my question. I estimated a variety of propensity scores by playing
with several different models: GBM, Logit and then trying out several
matching techniques: Nearest Neighbour (with a variety of calipers), Full
Matching, Subclassification and Genetic Matching.
In the literature I found
<http://www.sciencedirect.com/science/article/pii/S1010794009005727>researchers
to assess the balance by comparing the standardized difference in means and
calculating the variance ratio.
MatchIt summary output shows the standardized difference in means being
most effectively reduced with 1:1 Matching with a Caliper of 0.25 (all
covariates below a 10% level).
Since the t-test was understandably removed in later versions I wonder if
the variance ratio suffered the same fate? Or in other words why is it not
included in the output? And would it be possible to generate it from the
output?
And do you perhaps have any other pointers on how to assess balance or
generally accepted values that indicate a good balance?
Greatly appreciated if someone could point me in the right direction.
Regards,
Paul