Thanks for sending the new proposal. It looks great.
A few minor points to add to previous comments.
1. On ordering of vignettes: the model does not actually require that the
vignettes be ordered consistently across respondents, as each person
perceives the levels with some random error that could change ranks across
persons. To the extent that ambiguous ordering means the vignettes are
close to each other on the latent scale, though, we would hope that two
vignettes describe sufficiently different levels to give us as much
information as possible. Interestingly, in the pilots, it was actually
mobility vignettes C and D that were fairly close to each other, while C
and B were spaced reasonably far apart. The other possibility, if there is
still time to entertain one, would be Philip, who falls between A and B and
was estimated to be quite different from both in the pre-tests. For affect,
I think the change to "often" from "all the time" should put sufficient
distance between B and C. If you wanted to keep an additional vignette as
it was pilot tested, an alternative would be Margaret, who is similar to
the modified Eva.
2. On rotations: For at least one example (MO-B), the vignette is always
attached to the same name (Robert/Mary). Would it be worthwhile to rotate
the names a little further to leave room for possible analysis of name
effects for the same vignette? I'm not sure if the sample will be big
enough to reveal anything along these lines, but just in case. You could
switch the names for MO-B and AF-A on Form A and MO-B and MO-C on Form B,
which will add name variety to all 3 of these vignettes I think.
3. On mobility vignette A: Would an example help make 1/8 mile more
concrete? (2 football fields or city blocks?) I know this comes straight
from the 200 meters in the WHS version, which does not include an example,
but I agree with Gary's point that opportunities to improve on questions
should be taken if they preserve the meaning of the vignette. When we
designed vignettes, there were always tradeoffs between providing concrete
numbers, which presumably reduces variance in understanding of the vignette
among the numerate, but may not carry much meaning to some respondents, vs.
giving non-quantitative examples (e.g., carrying groceries), which may be
more relevant to some people but can introduce variance in the described
level (do the groceries weigh 10 lbs. or 50 lbs?). In the end, we included
examples of both types to try to cover all fronts, and I think your
selection does this as well.
Josh
At 11:12 PM 10/21/2002 -0400, Gary King wrote:
>On Mon, 21 Oct 2002, Jeremy Freese wrote:
>
>[material deleted, all of which I agree with]
>
> > Some of your other points were good suggestions regarding vignettes or
> > questions that are currently being implemented by the World Health
> > Survey. We stuck to these very closely out of a desire to make what
> > came out of our study as suitably comparable to their work as possible.
> > If faced with the choice of tweaking these in ways that we might see as
> > improvements or maintain verbatim equivalence, which direction would you
> > recommend that we go?
>
>My personal view is that all research on the cutting edge is almost by
>definition highly error prone and uncertain (in part because when its not,
>we usually ask harder questions), and we always think of plenty of things
>after the fact we would like to have done differently. And so when we
>actually know of real improvements that could feasibly be made ahead of
>time, I'd rather make them (if indeed they are improvements). This would
>seem to contradict the advantage of having questions comparable across
>surveys, and that may indeed be the case (and if so I might still do it
>since comparability even across all your respondents is an important goal
>in its own right). However, the vast majority of the WHS instruments are
>translated into many other languages. Since it is supposedly the meaning
>of the questions rather than the exact words used that are translated, you
>might view many of the improvements as better translations of this meaning
>into English. And so, assuming the translators understand the questions
>in the same way as you do, maybe there is somewhat less of a
>contradiction,
>
>Gary
>
> : Gary King, King(a)Harvard.Edu http://GKing.Harvard.Edu :
> : Center for Basic Research Direct (617) 495-2027 :
> : in the Social Sciences Assistant (617) 495-9271 :
> : 34 Kirkland Street, Rm. 2 HU-MIT DC (617) 495-4734 :
> : Harvard U, Cambridge, MA 02138 eFax (928) 832-7022 :
>
> >
> > Thanks once again for all your patience and help.
> >
> > --Jeremy
> >
> >
> > -
> > vign mailing list served by Harvard-MIT Data Center
> > List Address: vign(a)latte.harvard.edu
> > Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign
> >
>
>
>
>
>-
>vign mailing list served by Harvard-MIT Data Center
>List Address: vign(a)latte.harvard.edu
>Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign
--
Anchoring Vignettes mailing list served by Harvard-MIT Data Center
[Un]Subscribe/View Archive: http://lists.hmdc.harvard.edu/?info=vign
Hi, all.
I have previously used gllamm to estimate several chopit models, but I
recently decided to switch to R to see if there are any gains in
computational efficiency (and easier stochastic simulation). I am wondering
if anyone else has estimated the model with the example data (China and
Mexico's political efficacy).
When I estimate the model using the unaltered data, I get a few curious
results:
First, the cutpoints for the ordered probit model (i.e. "gamma1.cut1.ones")
are intransitive (.0955, -.6177, -.3790, -.2718)
Second, the reported lnse.self if 0.0000, with NaN standard error (the
lnse.re is also 0, but this makes sense since there is only one measure of
efficacy in the example).
Finally, all of the coefficients and standard errors that are not in the
model cause warnings (about 50 in all) and are reported NaN in the output.
I should note also that the nonparametric example seems to work fine.
If anyone else has tried the examples, did they get the same results?
I assume that the data are not the original survey data, but rather
simulated (this was the case for the gllamm implementation as well). Thus,
all of the points I mention may be either artifacts of the simulated data
(especially the first point) or normal operation of the program, but I just
want to be sure before I estimate a more complex, multiple measures model.
I am using R 1.6.0 (the October 2002 release) with the precompliled binaries
for Windows, on a PIII450 box.
Thanks,
Jack
___________________
Jack Buckley
Department of Political Science
State University of New York at Stony Brook
sbuckley(a)ic.sunysb.edu
Voice:(631) 632-4353
Fax: (631) 632-4116
Web: www.sinc.sunysb.edu/Stu/sbuckley
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.394 / Virus Database: 224 - Release Date: 10/3/2002
-
vign mailing list served by Harvard-MIT Data Center
List Address: vign(a)latte.harvard.edu
Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign
Jonathan Wand and I have just posted some software to analyze anchoring
vignettes data (we call it Anchors) at http://gking.harvard.edu/vign/.
Everything proposed in the King/Murray/Tandon/Salomon paper is programmed
in there, and a fair number of other things have been included too. Some
are those suggested by the conference participants and those on this email
list, and others are from our current research. We will continue to add
features as they are developed.
This software runs under the R statistics package. If you don't know R,
you'll find links to it at our site. R is free and available for every
operating system. It is open source and has numerous contributors all
over the world. The "business" model is the same as that of science with
individuals working in cooperation and competition to achieve closely
related goals (if you find a bug and post it at 2am, you'll likely get an
answer for how to fix it by 4am) rather than that of most commercial
software (if you find a bug, you have a bug, and no, you may not talk to
the developer!). R seems to be where the statistical world is converging.
On the same site, there is software to run chopit within Stata, although
it does not have the other features of Anchors. We'll also have a version
that runs under Gauss in the next couple of weeks.
Gary
: Gary King, King(a)Harvard.Edu http://GKing.Harvard.Edu :
: Center for Basic Research Direct (617) 495-2027 :
: in the Social Sciences Assistant (617) 495-9271 :
: 34 Kirkland Street, Rm. 2 HU-MIT DC (617) 495-4734 :
: Harvard U, Cambridge, MA 02138 eFax (928) 832-7022 :
-
vign mailing list served by Harvard-MIT Data Center
List Address: vign(a)latte.harvard.edu
Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign
