NSSE: National Survey of Student Engagement

There is often a gap between how much college faculty think students are studying and what they are actually doing. NSSE combined with FSSE points to steps institutions can take to ensure that student performance and faculty expectations align.

Carol A. Twigg, President and CEO, National Center for Academic Transformation

Maximizing Your Number of Respondents
Effectively and Ethically

Introduction
Suggestions for survey incentives
Suggestions for general campus promotion
Ethical considerations in survey research

Introduction

Student participation is critical to survey success. Simply put, more respondents generally yield better quality data. Sampling error, total completions, and response rates are important measures of data quality, and are important for increasing confidence in the results among a wider audience. NSSE works with institutions to personalize materials used to contact each student, emphasizing the survey’s value for institutional improvement. Institutional efforts can effectively supplement the messages sent to students by NSSE. However, always keep in mind that students’ decisions not to participate must be respected.

Each institution needs to assess their campus culture and determine appropriate methods to reach their students; there is no one “right way” to increase student participation. Survey research expert Dillman (2007) indicates several important factors that contribute to higher response rates:

Perceived importance of the survey (value to the student, perceived legitimacy).
Level of interest students have in the topic.
Creation of respondent trust.
Increasing perception of rewards for participation.
Decreasing perceptions of respondent burden.

Survey publicity such as flyers and media articles as well as incentives provided for participation can help send a message to the whole campus that the data are valuable for institutional improvement. We outline suggestions for increasing response rates below, as well as practices that should not be used because they can result in undue influence on participation.

Acceptable survey promotion practices

Customizing survey invitation messages with specifics for your campus.
Including small incentives and lotteries* in contact materials.
Flyers and press releases.

*Empirical research indicates that the effectiveness of lotteries and incentives is limited, although many NSSE schools do report that such efforts appear to have helped encourage students to respond. The estimated chance of winning a lottery should be included on any promotional information. Our recommended approach to this is to state the sample size and the number and value of prizes offered. (e.g., There will be drawings for two iPod Nanos ($149 value) and two $100 gift cards to the bookstore (about 2400 students have been invited to participate).

Unacceptable methods

More than five direct requests to participate in the survey.
Direct and personal contact by campus officials with individual students.
Revoking rights or privileges for non-response (e.g., blocking course or housing registration).

Suggestions for survey incentives

Offer small incentives (e.g., bookstore gift certificates or tokens for free goods or services) for each respondent. These work best if the incentive is received in advance. For example you could send a voucher by mail or by postcard that can be "cashed in" after completing the survey. These can be difficult to administer, so often institutions choose lotteries or drawing.
Enter respondents in a drawing, noting the odds of winning (e.g., the number of students invited to complete the survey and number of prizes offered) and the value of the prizes in promotional materials. Lottery prize ideas include parking permits, institution memorabilia, or tickets to athletic events.

Suggestions for general campus promotion

Post flyers on campus.
Send a press release to student and local newspaper
Submit an article, or place an ad or insert in the student newspaper.
Write an editorial for the student newspaper, by a student or administrator, explaining why the survey is important, the purpose of the survey, how the survey can be used as a tool for change on campus, and that the survey is sponsored by the institution. The editorial should emphasize that students’ participation is voluntary.
Post notices on the institutional Web site or course management systems.
Create a Facebook profile that describes the survey and ask prominent faculty or administrators to add it to their network.
Signs on campus buses or along local transportation routes.
Produce spots for campus TV and radio stations.
Set up tables in the student center or union with survey information.
For ideas of how to encourage survey participation, click here

Ethical Considerations in Survey Research

NSSE is both an institutional improvement effort and a research project, and as such, student participation in NSSE must be fully voluntary.. Participants in research studies should always be informed of their rights relative to their participation and should know any potential risks of involvement in the study. NSSE provides this information in its contacts with students (see the informed consent statement). Additional efforts you make to increase response rates should never cause students to feel they will be penalized for not participating.

The Belmont Report established guidelines that led to the creation of Institutional Review Boards (IRB) or Internal Review Boards (IRB) to regulate research involving human subjects. This report outlined three considerations for beneficence in research: (a) maximize possible benefits, (b) minimize possible harms, and (c) equitably distribute the risks and rewards of research.

Inappropriate efforts to increase participation in research fall into two general categories: coercion and undue influence. Coercive interactions are those that imply directly or indirectly that a potential participant might lose rights or privileges for not participating in the study. Explicit examples of coercive measures include requiring students to complete the survey in order to register for classes or graduate. Implicit examples include the use of language such as “you must complete this survey” or “students with real NSSEville State University pride would complete this survey.”

Undue influence occurs when the incentives used to increase participation become the primary reason why students participate, or when the number or nature of attempts to encourage participation are excessive. Students should not perceive that the compensation is so great that they participate because they need the incentive, regardless of a sense that completing the survey would be a heavy burden to them.

Small incentives provided to each student who completes the survey are generally considered ideal. Larger incentives such as lotteries that draw prize winners from all participants may also be acceptable, but publicity for these efforts should include an estimate of the odds of winning. Survey promotion for these incentives should not emphasize the prizes to a degree that minimizes the requirement of survey participation.

References

Dillman, D. A. (2007). Mail and Internet Surveys: The Tailored Design Method (2nd ed.). Hoboken, NJ: John Wiley & Sons.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1979, April 18). The Belmont Report: Ethical principles and guidelines for the protection of human subject of research. Washington, DC: Department of Health, Education, and Welfare. http://www.nihtraining.com/ohsrsite/guidelines/belmont.html

Maximizing Your Number of Respondents Effectively and Ethically