New subject: wls revised design

25 Oct 2002

Thanks for sending the new proposal. It looks great.

A few minor points to add to previous comments.

1. On ordering of vignettes: the model does not actually require that the 
vignettes be ordered consistently across respondents, as each person 
perceives the levels with some random error that could change ranks across 
persons. To the extent that ambiguous ordering means the vignettes are 
close to each other on the latent scale, though, we would hope that two 
vignettes describe sufficiently different levels to give us as much 
information as possible. Interestingly, in the pilots, it was actually 
mobility vignettes C and D that were fairly close to each other, while C 
and B were spaced reasonably far apart. The other possibility, if there is 
still time to entertain one, would be Philip, who falls between A and B and 
was estimated to be quite different from both in the pre-tests. For affect, 
I think the change to "often" from "all the time" should put
sufficient 
distance between B and C. If you wanted to keep an additional vignette as 
it was pilot tested, an alternative would be Margaret, who is similar to 
the modified Eva.

2. On rotations: For at least one example (MO-B), the vignette is always 
attached to the same name (Robert/Mary).  Would it be worthwhile to rotate 
the names a little further to leave room for possible analysis of name 
effects for the same vignette? I'm not sure if the sample will be big 
enough to reveal anything along these lines, but just in case. You could 
switch the names for MO-B and AF-A on Form A and MO-B and MO-C on Form B, 
which will add name variety to all 3 of these vignettes I think.

3. On mobility vignette A: Would an example help make 1/8 mile more 
concrete? (2 football fields or city blocks?)  I know this comes straight 
from the 200 meters in the WHS version, which does not include an example, 
but I agree with Gary's point that opportunities to improve on questions 
should be taken if they preserve the meaning of the vignette. When we 
designed vignettes, there were always tradeoffs between providing concrete 
numbers, which presumably reduces variance in understanding of the vignette 
among the numerate, but may not carry much meaning to some respondents, vs. 
giving non-quantitative examples (e.g., carrying groceries), which may be 
more relevant to some people but can introduce variance in the described 
level (do the groceries weigh 10 lbs. or 50 lbs?). In the end, we included 
examples of both types to try to cover all fronts, and I think your 
selection does this as well.

Josh

At 11:12 PM 10/21/2002 -0400, Gary King wrote:
...
 On Mon, 21 Oct 2002, Jeremy Freese wrote:

[material deleted, all of which I agree with]

  Some of your other points were good suggestions
regarding vignettes or
 questions that are currently being implemented by the World Health
 Survey.  We stuck to these very closely out of a desire to make what
 came out of our study as suitably comparable to their work as possible.
 If faced with the choice of tweaking these in ways that we might see as
 improvements or maintain verbatim equivalence, which direction would you
 recommend that we go? 
My personal view is that all research on the cutting edge is almost by
definition highly error prone and uncertain (in part because when its not,
we usually ask harder questions), and we always think of plenty of things
after the fact we would like to have done differently.  And so when we
actually know of real improvements that could feasibly be made ahead of
time, I'd rather make them (if indeed they are improvements).  This would
seem to contradict the advantage of having questions comparable across
surveys, and that may indeed be the case (and if so I might still do it
since comparability even across all your respondents is an important goal
in its own right).  However, the vast majority of the WHS instruments are
translated into many other languages.  Since it is supposedly the meaning
of the questions rather than the exact words used that are translated, you
might view many of the improvements as better translations of this meaning
into English.  And so, assuming the translators understand the questions
in the same way as you do, maybe there is somewhat less of a
contradiction,

Gary

      : Gary King, King(a)Harvard.Edu    http://GKing.Harvard.Edu :
      : Center for Basic Research      Direct    (617) 495-2027 :
      :   in the Social Sciences       Assistant (617) 495-9271 :
      : 34 Kirkland Street, Rm. 2      HU-MIT DC (617) 495-4734 :
      : Harvard U, Cambridge, MA 02138    eFax   (928) 832-7022 :

 Thanks once again for all your patience and help.

 --Jeremy

 -
 vign mailing list served by Harvard-MIT Data Center
 List Address: vign(a)latte.harvard.edu
 Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign

-
vign mailing list served by Harvard-MIT Data Center
List Address: vign(a)latte.harvard.edu
Subscribe/Unsubscribe: http://lists.hmdc.harvard.edu/?info=vign 

--
Anchoring Vignettes mailing list served by Harvard-MIT Data Center
[Un]Subscribe/View Archive: http://lists.hmdc.harvard.edu/?info=vign

RE: [vign] wls revised design