Some of his models are also included in Zelig.
Kosuke
--
Department of Politics
Princeton University
http://imai.princeton.edu
On Fri, 11 Sep 2009, Olivia Lau wrote:
> Doesn't Thomas Lumley's survey package implement frequency weights in
> svyglm? You would have to figure out how to hack svydesign to produce
> something appropriate for full matching, though.
>
> On Fri, Sep 11, 2009 at 1:21 PM, Kosuke Imai <kimai(a)princeton.edu> wrote:
>> One thing to note is that lm() in R is probably not using frequency weights,
>> which may explain why you get similar results. I would explore alternative
>> approaches either based on random or fixed effects.
>>
>> Kosuke
>>
>> --
>> Department of Politics
>> Princeton University
>> http://imai.princeton.edu
>>
>> On Fri, 11 Sep 2009, Peter Steiner wrote:
>>
>>> Hi Liz,
>>>
>>> thanks for your references.
>>> I tried to use the matchit() weights, but the weights either didn't change
>>> results very much or sim() produced an error message (as described below).
>>>
>>> Thanks,
>>> Peter
>>>
>>> Stuart, Elizabeth A. wrote:
>>>>
>>>> Hi Peter,
>>>> Kosuke is right; full matching essentially uses all observations and so
>>>> your outcome models have to take into account the structure of the matching
>>>> in some way, either by using the subclasses that are formed (e.g., through
>>>> random or fixed effects) or by using the weights that matchit produces
>>>> (which are essentially frequency weights). I am not sure your zelig code
>>>> below does either of those things; it looks like it is just taking a simple
>>>> difference in means of the outcome in the matched samples (which will be the
>>>> same as the original samples).
>>>>
>>>> Ben Hansen discusses the subclass-specific fixed effects approach in his
>>>> 2004 paper (http://www.stat.lsa.umich.edu/%7Ebbh/hansen2004.pdf),
>>>> <http://www.stat.lsa.umich.edu/%7Ebbh/hansen2004.pdf%29,> or you can also
>>>> see description of both approaches in my paper on full matching (Stuart,
>>>> E.A., and Green, K.M. (2008). Using Full Matching to Estimate Causal Effects
>>>> in Non-Experimental Studies: Examining the Relationship between Adolescent
>>>> Marijuana Use and Adult Outcomes. /Developmental Psychology/ 44(2):
>>>> 395-406.)
>>>>
>>>> All of that said, I do also agree with Kosuke that more work is needed to
>>>> better understand the best approaches for doing analyses following full
>>>> matching. But I hope this helps.
>>>>
>>>> Thanks,
>>>> Liz
>>>>
>>>> On 9/11/09 9:55 AM, "Kosuke Imai" <kimai(a)Princeton.EDU> wrote:
>>>>
>>>> Full matching tries to use as many observations as possible, and
>>>> so if you
>>>> just do diff-in-means you will get a poor estimate (this is close
>>>> to not
>>>> doing any adjustment). I think that it is an open research
>>>> question as to
>>>> how to do model-based adjustment after full matching, and so I
>>>> don't want
>>>> to make any strong suggestion here. One way to do it however is
>>>> to use
>>>> either fixed or random effect type models so that the strata
>>>> created by
>>>> full matching can be incorporated into the model. But, others may
>>>> have
>>>> better ideas.
>>>>
>>>> Kosuke
>>>>
>>>> --
>>>> Department of Politics
>>>> Princeton University
>>>> http://imai.princeton.edu
>>>>
>>>> On Wed, 9 Sep 2009, Peter Steiner wrote:
>>>>
>>>> > Hi,
>>>> >
>>>> > after briefly outlining my problem with the full-matching option
>>>> in Zelig and
>>>> > MatchIt to Imai, I post it in more detail and with the LaLonde data.
>>>> > Unfortunately, I was not able to figure out how to correctly
>>>> implement
>>>> > full-matching with Zelig using all your documentation.
>>>> > Moreover, the help in R seems to be sometimes inconsistent (the
>>>> help file for
>>>> > match.data() differs for the MatchIt and Zelig Package; one
>>>> suggests that
>>>> > weights are allowed (MatchIt) the other does not show an option
>>>> for weights
>>>> > (Zelig))
>>>> >
>>>> > However, I'm interested in estimating ATE using full matching. A
>>>> > straightforward way seems to be the following:
>>>> >
>>>> > m.out <- matchit(treat ~ age + educ + black + hispan +
>>>> > nodegree + married + re74 + re75, data = lalonde,
>>>> > method = 'full', discard = 'both')
>>>> > z.out1 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
>>>> > married + re74 + re75, data = match.data(m.out, "control"),
>>>> > model = 'ls')
>>>> > z.out0 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
>>>> > married + re74 + re75, data = match.data(m.out, "treat"),
>>>> > model = 'ls')
>>>> > x.out1 <- setx(z.out1, data = match.data(m.out, "treat"), cond = T)
>>>> > x.out0 <- setx(z.out0, data = match.data(m.out, "control"), cond =
>>>> T)
>>>> > s.out1 <- sim(z.out1, x = x.out1)
>>>> > s.out0 <- sim(z.out0, x = x.out0)
>>>> > ate.all <- c(s.out1$qi$att.ev, -s.out0$qi$att.ev)
>>>> > c(mean(ate.all), sd(ate.all))
>>>> >
>>>> > This results in an ATE of 1335.5, which seems at least plausible.
>>>> But here is
>>>> > my problem. Using only full-matching without additional
>>>> covariance adjustment
>>>> > by setting
>>>> >
>>>> > z.out1 <- zelig(re78 ~ 1, data = match.data(m.out, "control"),
>>>> > model = 'ls')
>>>> > z.out0 <- zelig(re78 ~ 1, data = match.data(m.out, "treat"),
>>>> > model = 'ls')
>>>> >
>>>> > results in an ATE of 86.88, indicating a very poor performance of
>>>> full
>>>> > matching. (Using Ben Hansen's R package alone, full matching
>>>> would reduce
>>>> > almost all bias.) So I tried to use the weights generated by the
>>>> matchit()
>>>> > command. Using weights in match.data(..., weights = 'weights')
>>>> doesn't change
>>>> > the estimates very much. And whenever I include weights in
>>>> zelig() the sim()
>>>> > command produces an Error.
>>>> >
>>>> > So my question is, what is the correct way to perform
>>>> full-matching in Zelig?
>>>> >
>>>> > I have another question, concerning nearest neighbor matching
>>>> (1:1 matching,
>>>> > for instance). What does ATE exactly estimate in Zelig? ATE of
>>>> the original
>>>> > (unmatched) sample or ATE of the matched sample only (note that
>>>> the covariate
>>>> > distribution of the original and matched sample usually differ
>>>> considerably).
>>>> > I assume the latter.
>>>> >
>>>> > Thanks a lot,
>>>> > Peter
>>>> >
>>>> >
>>>>
>>>> --
>>>> MatchIt mailing list served by Harvard-MIT Data Center
>>>> List Address: matchit(a)lists.gking.harvard.edu
>>>> Subscribe/Unsubscribe: http://lists.gking.harvard.edu/?info=matchit
>>>> MatchIt Software and Documentation: http://gking.harvard.edu/matchit/
>>>> Browse/Search List Archive:
>>>> http://lists.hmdc.harvard.edu/lists/matchit/
>>>>
>>>>
>>>
>>>
>>>
>>
>> -
>> Zelig Mailing List, served by Harvard-MIT Data Center
>> Send messages: zelig(a)lists.gking.harvard.edu
>> [un]subscribe Options: http://lists.gking.harvard.edu/?info=zelig
>> Zelig program information: http://gking.harvard.edu/zelig/
>>
>
> -
> Zelig Mailing List, served by Harvard-MIT Data Center
> Send messages: zelig(a)lists.gking.harvard.edu
> [un]subscribe Options: http://lists.gking.harvard.edu/?info=zelig
> Zelig program information: http://gking.harvard.edu/zelig/
>
One thing to note is that lm() in R is probably not using frequency
weights, which may explain why you get similar results. I would explore
alternative approaches either based on random or fixed effects.
Kosuke
--
Department of Politics
Princeton University
http://imai.princeton.edu
On Fri, 11 Sep 2009, Peter Steiner wrote:
> Hi Liz,
>
> thanks for your references.
> I tried to use the matchit() weights, but the weights either didn't change
> results very much or sim() produced an error message (as described below).
>
> Thanks,
> Peter
>
> Stuart, Elizabeth A. wrote:
>> Hi Peter,
>> Kosuke is right; full matching essentially uses all observations and so
>> your outcome models have to take into account the structure of the matching
>> in some way, either by using the subclasses that are formed (e.g., through
>> random or fixed effects) or by using the weights that matchit produces
>> (which are essentially frequency weights). I am not sure your zelig code
>> below does either of those things; it looks like it is just taking a simple
>> difference in means of the outcome in the matched samples (which will be
>> the same as the original samples).
>>
>> Ben Hansen discusses the subclass-specific fixed effects approach in his
>> 2004 paper (http://www.stat.lsa.umich.edu/%7Ebbh/hansen2004.pdf),
>> <http://www.stat.lsa.umich.edu/%7Ebbh/hansen2004.pdf%29,> or you can also
>> see description of both approaches in my paper on full matching (Stuart,
>> E.A., and Green, K.M. (2008). Using Full Matching to Estimate Causal
>> Effects in Non-Experimental Studies: Examining the Relationship between
>> Adolescent Marijuana Use and Adult Outcomes. /Developmental Psychology/
>> 44(2): 395-406.)
>>
>> All of that said, I do also agree with Kosuke that more work is needed to
>> better understand the best approaches for doing analyses following full
>> matching. But I hope this helps.
>>
>> Thanks,
>> Liz
>>
>> On 9/11/09 9:55 AM, "Kosuke Imai" <kimai(a)Princeton.EDU> wrote:
>>
>> Full matching tries to use as many observations as possible, and
>> so if you
>> just do diff-in-means you will get a poor estimate (this is close
>> to not
>> doing any adjustment). I think that it is an open research
>> question as to
>> how to do model-based adjustment after full matching, and so I
>> don't want
>> to make any strong suggestion here. One way to do it however is
>> to use
>> either fixed or random effect type models so that the strata
>> created by
>> full matching can be incorporated into the model. But, others may
>> have
>> better ideas.
>>
>> Kosuke
>>
>> --
>> Department of Politics
>> Princeton University
>> http://imai.princeton.edu
>>
>> On Wed, 9 Sep 2009, Peter Steiner wrote:
>>
>> > Hi,
>> >
>> > after briefly outlining my problem with the full-matching option
>> in Zelig and
>> > MatchIt to Imai, I post it in more detail and with the LaLonde data.
>> > Unfortunately, I was not able to figure out how to correctly
>> implement
>> > full-matching with Zelig using all your documentation.
>> > Moreover, the help in R seems to be sometimes inconsistent (the
>> help file for
>> > match.data() differs for the MatchIt and Zelig Package; one
>> suggests that
>> > weights are allowed (MatchIt) the other does not show an option
>> for weights
>> > (Zelig))
>> >
>> > However, I'm interested in estimating ATE using full matching. A
>> > straightforward way seems to be the following:
>> >
>> > m.out <- matchit(treat ~ age + educ + black + hispan +
>> > nodegree + married + re74 + re75, data = lalonde,
>> > method = 'full', discard = 'both')
>> > z.out1 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
>> > married + re74 + re75, data = match.data(m.out, "control"),
>> > model = 'ls')
>> > z.out0 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
>> > married + re74 + re75, data = match.data(m.out, "treat"),
>> > model = 'ls')
>> > x.out1 <- setx(z.out1, data = match.data(m.out, "treat"), cond = T)
>> > x.out0 <- setx(z.out0, data = match.data(m.out, "control"), cond = T)
>> > s.out1 <- sim(z.out1, x = x.out1)
>> > s.out0 <- sim(z.out0, x = x.out0)
>> > ate.all <- c(s.out1$qi$att.ev, -s.out0$qi$att.ev)
>> > c(mean(ate.all), sd(ate.all))
>> >
>> > This results in an ATE of 1335.5, which seems at least plausible.
>> But here is
>> > my problem. Using only full-matching without additional
>> covariance adjustment
>> > by setting
>> >
>> > z.out1 <- zelig(re78 ~ 1, data = match.data(m.out, "control"),
>> > model = 'ls')
>> > z.out0 <- zelig(re78 ~ 1, data = match.data(m.out, "treat"),
>> > model = 'ls')
>> >
>> > results in an ATE of 86.88, indicating a very poor performance of
>> full
>> > matching. (Using Ben Hansen's R package alone, full matching
>> would reduce
>> > almost all bias.) So I tried to use the weights generated by the
>> matchit()
>> > command. Using weights in match.data(..., weights = 'weights')
>> doesn't change
>> > the estimates very much. And whenever I include weights in
>> zelig() the sim()
>> > command produces an Error.
>> >
>> > So my question is, what is the correct way to perform
>> full-matching in Zelig?
>> >
>> > I have another question, concerning nearest neighbor matching
>> (1:1 matching,
>> > for instance). What does ATE exactly estimate in Zelig? ATE of
>> the original
>> > (unmatched) sample or ATE of the matched sample only (note that
>> the covariate
>> > distribution of the original and matched sample usually differ
>> considerably).
>> > I assume the latter.
>> >
>> > Thanks a lot,
>> > Peter
>> >
>> >
>>
>> --
>> MatchIt mailing list served by Harvard-MIT Data Center
>> List Address: matchit(a)lists.gking.harvard.edu
>> Subscribe/Unsubscribe: http://lists.gking.harvard.edu/?info=matchit
>> MatchIt Software and Documentation: http://gking.harvard.edu/matchit/
>> Browse/Search List Archive:
>> http://lists.hmdc.harvard.edu/lists/matchit/
>>
>>
>
>
>
--
MatchIt mailing list served by Harvard-MIT Data Center
List Address: matchit(a)lists.gking.harvard.edu
Subscribe/Unsubscribe: http://lists.gking.harvard.edu/?info=matchit
MatchIt Software and Documentation: http://gking.harvard.edu/matchit/
Browse/Search List Archive: http://lists.hmdc.harvard.edu/lists/matchit/
Full matching tries to use as many observations as possible, and so if you
just do diff-in-means you will get a poor estimate (this is close to not
doing any adjustment). I think that it is an open research question as to
how to do model-based adjustment after full matching, and so I don't want
to make any strong suggestion here. One way to do it however is to use
either fixed or random effect type models so that the strata created by
full matching can be incorporated into the model. But, others may have
better ideas.
Kosuke
--
Department of Politics
Princeton University
http://imai.princeton.edu
On Wed, 9 Sep 2009, Peter Steiner wrote:
> Hi,
>
> after briefly outlining my problem with the full-matching option in Zelig and
> MatchIt to Imai, I post it in more detail and with the LaLonde data.
> Unfortunately, I was not able to figure out how to correctly implement
> full-matching with Zelig using all your documentation.
> Moreover, the help in R seems to be sometimes inconsistent (the help file for
> match.data() differs for the MatchIt and Zelig Package; one suggests that
> weights are allowed (MatchIt) the other does not show an option for weights
> (Zelig))
>
> However, I'm interested in estimating ATE using full matching. A
> straightforward way seems to be the following:
>
> m.out <- matchit(treat ~ age + educ + black + hispan +
> nodegree + married + re74 + re75, data = lalonde,
> method = 'full', discard = 'both')
> z.out1 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
> married + re74 + re75, data = match.data(m.out, "control"),
> model = 'ls')
> z.out0 <- zelig(re78 ~ age + educ + black + hispan + nodegree +
> married + re74 + re75, data = match.data(m.out, "treat"),
> model = 'ls')
> x.out1 <- setx(z.out1, data = match.data(m.out, "treat"), cond = T)
> x.out0 <- setx(z.out0, data = match.data(m.out, "control"), cond = T)
> s.out1 <- sim(z.out1, x = x.out1)
> s.out0 <- sim(z.out0, x = x.out0)
> ate.all <- c(s.out1$qi$att.ev, -s.out0$qi$att.ev)
> c(mean(ate.all), sd(ate.all))
>
> This results in an ATE of 1335.5, which seems at least plausible. But here is
> my problem. Using only full-matching without additional covariance adjustment
> by setting
>
> z.out1 <- zelig(re78 ~ 1, data = match.data(m.out, "control"),
> model = 'ls')
> z.out0 <- zelig(re78 ~ 1, data = match.data(m.out, "treat"),
> model = 'ls')
>
> results in an ATE of 86.88, indicating a very poor performance of full
> matching. (Using Ben Hansen's R package alone, full matching would reduce
> almost all bias.) So I tried to use the weights generated by the matchit()
> command. Using weights in match.data(..., weights = 'weights') doesn't change
> the estimates very much. And whenever I include weights in zelig() the sim()
> command produces an Error.
>
> So my question is, what is the correct way to perform full-matching in Zelig?
>
> I have another question, concerning nearest neighbor matching (1:1 matching,
> for instance). What does ATE exactly estimate in Zelig? ATE of the original
> (unmatched) sample or ATE of the matched sample only (note that the covariate
> distribution of the original and matched sample usually differ considerably).
> I assume the latter.
>
> Thanks a lot,
> Peter
>
>
--
MatchIt mailing list served by Harvard-MIT Data Center
List Address: matchit(a)lists.gking.harvard.edu
Subscribe/Unsubscribe: http://lists.gking.harvard.edu/?info=matchit
MatchIt Software and Documentation: http://gking.harvard.edu/matchit/
Browse/Search List Archive: http://lists.hmdc.harvard.edu/lists/matchit/
On 09/02/2009 12:20 AM, Donald Braman wrote:
> I can't seem to subscribe to the MatchIt list, so I'm posting here in
> hopes that the same people read both.
>
> I'm working with multiply imputed data from Amelia & wondering if any
> progress has been made at integrating Amelia, MatchIt, and Zelig -- in
> particular, pre-processing multiply imputed data and preparing them
> for use in Zelig.
>
> Not at all pressing, and (again) apologies for posting here rather
> than the MatchIt list.
>
> Cheers, Don
At the moment, MatchIt doesn't allow missing data on input (unless you
temporarily code it as observed and exact match on the missing values).
In part, this is because most matching algorithms don't have procedures
to deal with missingness. An exception is CEM which has a procedure for
multiply imputed data, but to access that you'll need to use the (R or
Stata) version of CEM directly (see http://gking.harvard.edu/cem)
instead of the one in MatchIt.
(separate message coming to fix your matchit subscription...)
Gary
---
Gary King
Albert J. Weatherhead III University Professor
Director, Institute for Quantitative Social Science
Harvard University, 1737 Cambridge St, Cambridge, MA 02138
http://GKing.Harvard.Edu, King(a)Harvard.Edu
Direct 617-495-2027, Assistant 495-9271, eFax 812-8581
--
MatchIt mailing list served by Harvard-MIT Data Center
List Address: matchit(a)lists.gking.harvard.edu
Subscribe/Unsubscribe: http://lists.gking.harvard.edu/?info=matchit
MatchIt Software and Documentation: http://gking.harvard.edu/matchit/
Browse/Search List Archive: http://lists.hmdc.harvard.edu/lists/matchit/
