Maximizing Your Number of Respondents Effectively and Ethically
Student participation is critical to survey success. Simply put, more respondents generally yield better quality data (Laguilles, Williams, & Saunders, 2011; Schaefer & Dillman, 1998). Sampling error, total completions, representativeness, and response rates are important measures of data quality (Cook, Heath, & Thompson, 2000), and are important for increasing confidence in the results among a wider audience. (See NSSE’s Response Rate FAQ for further discussion about data quality indicators.) Despite the challenges of obtaining high response rates (Laguilles, Williams, & Saunders, 2011; Umbach, 2004), NSSE works with institutions to personalize materials used to contact each student, emphasizing the survey's value for institutional improvement. Institutional efforts can effectively supplement the invitation messages sent to students by NSSE. Survey participation is more likely when students are not suffering from survey fatigue. Therefore, institutions are advised to plan survey cycles to avoid contributing to survey fatigue. However, always keep in mind that students' decisions not to participate must be respected.
First Steps - Campus Culture/Acceptable Practices
Ideas for Encouraging Student Participation in NSSE
General Campus Promotion
Ethical Considerations in Survey Research
- Perceived importance of the survey (value to the student, perceived legitimacy).
- Level of interest students have in the topic
- Creation of respondent trust
- Increasing perception of rewards for participation
- Decreasing perceptions of respondent burden and survey fatigue
The abovementioned factors are worth thinking about and addressing in survey promotion. Of course, survey publicity such as flyers and media articles as well as incentives provided for participation can help send a message to the whole campus that the data are valuable for institutional improvement. We outline suggestions for increasing response rates below, as well as practices that should not be used because they can result in undue influence on participation.
Acceptable survey promotion practices
- Customizing survey invitation messages (Cook, Heath, & Thompson, 2000).
- Including small guaranteed incentives and lotteries* in contact materials (Millar & Dillman, 2011; Laguilles, Williams, & Saunders, 2011)
- Flyers, press releases, and videos
- Informing students that survey responses will be kept confidential (Dillman, Singer, Clark, and Treat, 1996)
- Multiple reminders with the sample (Dillman, 2007; Schaefer & Dillman, 1998)
Unacceptable Survey Promotional Practices (per NSSE's IRB documentation)
- More than five direct requests to participate in the survey
- Direct and personal contact by campus officials with individual students
- Revoking rights or privileges for non-response (e.g., blocking course or housing registration)
- Improperly describing NSSE as an anonymous survey. Identified student responses are provided to institutional personnel, so survey responses are not anonymous
- Offer small incentives (e.g., bookstore gift certificates or tokens for free goods or services) for each respondent. These work best if the incentive is received in advance. For example you could send a voucher by mail or by postcard that can be "cashed in" after completing the survey. These can be difficult to administer, so often institutions choose lotteries or drawing.
- Enter respondents in a drawing, noting the odds of winning (e.g., the number of students invited to complete the survey and number of prizes offered) and the value of the prizes in promotional materials. Lottery prize ideas include parking permits, institution memorabilia, or tickets to athletic events.
- Post flyers on campus.
- Send a press release to student and local newspaper.
- Submit an article, or place an ad or insert in the student newspaper.
- Submit an editorial to the student newspaper written by a student or administrator, explaining why the survey is important, the purpose, how the survey will be used and that the survey is sponsored by the institution; emphasize that students' participation is voluntary.
- Post notices on the institutional Web site or course management systems.
- Create a Facebook group or Twitter account and share results and update students
- Signs on campus buses or along local transportation routes.
- Produce locations for campus TV and radio stations.
- Set up tables in the student center or union with survey information.
- For additional ideas of how to encourage survey participation, click here
NSSE is both an institutional improvement effort and a research project, and as such, student participation in NSSE must be fully voluntary. Participants in research studies should always be informed of their rights relative to their participation and should know any potential risks of involvement in the study. NSSE provides this information in its contacts with students (see the informed consent statement). Additional efforts you make to increase response rates should never cause students to feel they will be penalized for not participating.
The Belmont Report (1979) established guidelines that led to the creation of Institutional Review Boards (IRB) or Internal Review Boards (IRB) to regulate research involving human subjects. This report outlined three considerations for beneficence in research: (a) maximize possible benefits, (b) minimize possible harms, and (c) equitably distribute the risks and rewards of research.
Inappropriate efforts to increase participation in research fall into two general categories: coercion and undue influence. Coercive interactions are those that imply directly or indirectly that a potential participant might lose rights or privileges for not participating in the study. Explicit examples of coercive measures include requiring students to complete the survey in order to register for classes or graduate. Implicit examples include the use of language such as “you must complete this survey” or “students with real NSSEville State University pride would complete this survey.”
Undue influence occurs when the incentives used to increase participation become the primary reason why students participate, or when the number or nature of attempts to encourage participation are excessive. Students should not perceive that the compensation is so great that they participate because they need the incentive, regardless of a sense that completing the survey would be a heavy burden to them.
Small incentives provided to each student who completes the survey are generally considered ideal. Larger incentives such as lotteries that draw prize winners from all participants may also be acceptable, but publicity for these efforts should include an estimate of the odds of winning. Survey promotion for these incentives should not emphasize the prizes to a degree that minimizes the requirement of survey participation.