Vignette Study on Web Paradata

Download data and study materials from OSF

Principal investigators:

Mick P. Couper

University of Michigan

Email: mcouper@umich.edu

Homepage: http://www.psc.isr.umich.edu/people/profile/472

Eleanor Singer

University of Michigan

Email: elsinger@umich.edu

Homepage: http://www.psc.isr.umich.edu/people/profile/578/Eleanor_Singer


Sample size: 5000

Field period: 9/22/2009-12/28/2009

Abstract

Using vignettes, we explored the effect of alternative wording of statements informing Web survey respondents about the capture of paradata (e.g., time stamps and mouse clicks) on willingness to participate in such a survey.

Research Questions:

1. We expected willingness to participate (WTP) in the hypothetical survey to be higher for surveys about leisure time and sports than on surveys about risk behaviors.
2. We expected WTP to be highest in the control condition (no mention of paradata), and higher in the condition that gives a simple description of paradata only than in the conditions that also describes how paradata might be used.
3. We expected WTP to be higher in the condition that justifies the paradata use for technical reasons than the condition where it is used to understand respondent behavior.
4. We expected WTP to be higher in the condition that included a hyperlink for additional information than in the corresponding condition containing the same information without a hyperlink.

Experimental Manipulations

1. (Control condition) No mention of paradata
2. (Basic description) In addition to your responses to the survey, we collect technical data (such as keystrokes and time stamps). As soon as you finish answering the survey, we stop collecting the technical data. We never track your browsing behavior.
3. (Basic + Respondent Behavior) In addition to your responses to the survey, we collect technical data (such as keystrokes and time stamps) to help us better understand your answers. As soon as you finish answering the survey, we stop collecting the technical data. We never track your browsing behavior.
4. (Basic + Technical Quality) In addition to your responses to the survey, we collect technical data (such as keystrokes and time stamps) to help us improve the questionnaire. As soon as you finish answering the survey, we stop collecting the technical data. We never track your browsing behavior.
5. (Basic + Respondent Behavior and Technical Quality) In addition to your answers to the survey, we collect technical data (such as keystrokes and time stamps) to help us better understand your answers and to improve the questionnaire. As soon as you finish answering the survey, we stop collecting the technical data. We never track your browsing behavior.
6. (Basic + Hyperlink to Behavior and Quality) In addition to your responses to the survey, we collect technical data (such as keystrokes and time stamps). (Click here for more information about how these data are used.) As soon as you finish answering the survey, we stop collecting the technical data. We never track your browsing behavior.

Outcomes

Willingness to participate (WTP) in the hypothetical survey on a 0-10 scale.

Summary of Results

Mentioning paradata significantly (p<.0001) reduced stated willingness to participate in the hypothetical survey, whether examining mean WTP or percent willing. Contrary to expectation, justifying paradata capture for technical reasons produces lower agreement to permit use than justifying it in terms of understanding respondent behavior. We also find no evidence that giving more explanation increases willingness to permit use of paradata.

References

Couper, M.P. and Singer, E. (2011), "Ethical Dilemmas in Dealing with Web Paradata." Paper to be presented at the European Survey Research Association conference, Lausanne, July.