The NSSE 2002 sample1 was comprised of 206,844 first-year and senior students who were randomly selected from electronic data files provided by 366 participating four-year colleges and universities. A list of these institutions is available in Appendix A of the institutional report. NSSE sampling procedures call for an equal number of first-year and senior students to be sent the survey with the standard sample size determined by the number of undergraduate students enrolled at the institution. Students at the majority of colleges and universities (67% or 245 schools) had the option of responding either via a traditional paper questionnaire or via the World Wide Web. One hundred and twenty-one (33%) schools opted to be Web-only institutions where students received an introduction letter through the mail and all further contact electronically.

Tables 1 and 2 on the next two pages show that NSSE 2002 participating institutions and respondents approximate the characteristics of students enrolled at participating schools as well as the national profile of all fouryear colleges and universities. The source of the comparative data is the 1999-2000 Integrated Postsecondary Education Data System (IPEDS) database, the most recent complete data file available. However, the IPEDS data are two years old so the comparisons may not accurately reflect certain institutional and student characteristics for the 2001-2002 academic year.

Profile of NSSE 2002 Institutions

NSSE 2002 schools closely resembled the national profile of four-year colleges and universities in terms of region of the country and location, as demonstrated in Table 1. However, NSSE 2002 institutions included more Doctoral/ Research Universities and Baccalaureate Colleges- Liberal Arts and fewer Baccalaureate Colleges-General as defined by the 2000 Carnegie Classification of Institutions of Higher Education.

Doctoral/Research Universities and Master?s Colleges and Universities enroll more than threequarters of all undergraduates. At the same time, ample numbers of smaller, independent colleges also took part in NSSE 2002, insuring that the results reflect the experiences of a broad crosssection of students attending four-year colleges and universities from both the public and private sector, from all regions of the country, and from different types of settings.

Profile of NSSE 2002 Respondents

Table 2, on the following page, shows selected characteristics of the 80,597 students who completed The College Student Report in 2002. The first column represents students who responded to the NSSE survey in 2002, the second column shows the characteristics of students at the four-year schools that participated in NSSE 2002 as reflected by 1999-2000 IPEDS data, and the third column represents the national profile of students at all four-year colleges and universities from IPEDS data.

Year in School

The respondents were about equally divided between first-year (49%) and senior (51%) students.

Gender

Women made up two-thirds (66%) of the respondents compared with 57% of the students enrolled at NSSE 2002 schools and 58% nationally (Table 2). The larger proportion of women respondents is consistent with the widely reported survey research findings that women are more likely than men to return questionnaires.

Race and Ethnicity

White, Asian/Pacific Islander, Hispanic, and American Indian/Alaska Native students are slightly over-represented and African American students are slightly under-represented (Table 2).

Age

Students 19 years of age or younger compose the largest group (44%), reflecting the fact that half the students selected to receive the survey were in their first year of college. About 37% of respondents were 20-23, 9% were between the ages of 24 and 29, and 10% were 30 years of age or older.

Living Arrangements

Forty-four percent of all students lived in campus housing (68% of first-year students, 21% of seniors). The remainder lived within driving distance (42%), within walking distance (12%), or in a fraternity or sorority house (2%).

Fraternity or Sorority

Thirteen percent of men and 11% of women were members of a social fraternity or sorority.

Grades

Just over 21% of all students reported that they have earned mostly A grades. Only 3% of students reported earning mostly C?s or lower.

Parents' Education

Forty-two percent of all respondents were first-generation college students. Approximately one third (32%) had both parents who graduated from college.

Enrollment Status

About 88% of all students were enrolled full-time (Table 2). Approximately 24% of all students attended one or more other institutions in addition to the one at which they were currently enrolled. Of this group of multiple-institution attendees, 50% went to another four-year college, 42% to a community college, 2% to a vocationaltechnical education, health-related fields, and the social sciences.

Response Rates

The average institutional response rate for NSSE 2002 was 41%2. The average institutional response rate for standard schools (institutions where students had the option of completing either the paper or the Web version of The Report) was 41%, with a range of 16% to 89% across schools. The average institutional response rate for NSSE 2002 Web-only schools (institutions where students only had the option of completing the survey online) was 39%, with a range of 10% to 73% across schools. About 55% of the NSSE 2002 respondents completed the paper version of The College Student Report and approximately 45% completed it using the Web. Additional information about response rates, including the response rate for your institution, can be found in Table 7 at the end of the ?Overview? on page 16.

Selected Results

From a preliminary analysis of the 2002 results at the national level, we offer the following observations about certain areas of student engagement at four-year colleges and universities. For all analyses, selected student and institutional characteristics are controlled.

Grades and Engagement

As expected, student-reported grade point average (GPA) is positively linked to nearly all engagement items on the survey. In particular, a higher GPA is associated with more time spent preparing for class, asking more questions in class, more frequently tutoring other students, more favorable relationships with faculty, receiving feedback more frequently from faculty, and a more positive evaluation of overall educational experiences in college. These patterns generally hold for both first-year and senior students, though they don?t explain the direction of the relationship between grades and engagement (e.g., does engagement result in higher grades, or do higher grades promote more engagement?). What is clear is that engagement and grades go hand-in-hand.

Academic Challenge and Major Field

The nature of course work presented to students in different majors varies. For example, more than three quarters (77%) of the seniors in ethnic studies majors and more than two thirds (68%) in multi/interdisciplinary studies wrote a paper of 20 pages or more. Only one third of mathematics (33%) and visual and performing arts (37%) majors had written a paper of 20 pages or more.

Nearly all of the seniors majoring in the humanities (93%), ethnic studies (89%) and foreign languages (89%) had been assigned five or more books. Less than two thirds of the seniors in engineering (65%), visual and performing arts (65%), computer sciences (65%) and agriculture (60%) were assigned five or more books.

Students in health-related majors (63%) and education majors (61%) were the most likely to indicate that they had frequently (often or very often) worked harder than they thought they could to meet an instructor?s expectations. Students in agriculture (44%) and parks and recreation (50%) majors were the least likely to state that they had frequently worked harder than they thought they could to meet an instructor?s expectations.

Diversity Matters

Diversity-related experiences are positively related to other areas of effective educational practice. Specifically, (a) talking with others of different races/ ethnicities, (b) talking with others who are very different from oneself in terms of their religious beliefs or personal values, (c) incorporating diverse perspectives into class discussions or writing, and (d) attending an institution that encourages contact among students of different backgrounds are all associated with greater self-reported collegiate gains in personal-social and general education realms. In addition, students who experience more diversity reported more involvement in active and collaborative learning activities and also reported that their courses more frequently required critical thinking.

Seniors in the applied sciences (engineering, agriculture, computer science), physical sciences, and math report the fewest experiences with diversity. Conversely, students in ethnic studies, liberal/general studies, humanities, and social sciences report the greatest number of experiences with diversity.

International Students

International students appear generally more engaged in various college experiences than students who are citizens of the United States. Specifically, international students reported greater gains in personal and social development, practical competence, and general education compared with their American classmates. However, international students were slightly less satisfied with their college experience and less engaged in community or volunteer work compared with American students.

In addition, first-year international students reported higher levels of academic challenge and perceived the campus environment to be more supportive than their American counterparts. However, international first-year students spent significantly less time relaxing and socializing than their American peers.

Older Students

Students age 26 or older spent substantially more time providing care for dependents living with them, working for pay off-campus, and commuting to class than did their younger, traditional-age peers. However, older students spent less time relaxing and socializing than their traditional-aged counterparts. Older first-year and senior students were considerably less engaged in enriching educational activities compared to traditional-age students. For example, older students are less likely to:

• Use technology (e-mail, list-serve, chat group, and Internet) to communicate with an instructor or to discuss or complete an assignment
• Interact with faculty members
• Report gains in personal social development and practical competence.

Despite these differences both older first-year and senior students appear to be more satisfied with their school than their traditional-age peers. For example, older students are more likely to:

• Spend time preparing for class
• Report positive relationships with administrative personnel and offices
• Engage in active and collaborative learning.

Transfer Students

Forty percent of all seniors who completed the NSSE survey attended college at one or more different institutions before enrolling in the school from which they were about to graduate. What is the quality of the educational experience for those who enter an institution at some ?midstream? point in their undergraduate years? What do they look like? Does their college experience differ in systematic ways from their counterparts who started college at the same institution from which they will graduate?

Senior transfer students differ in some marked ways from non-transfers in terms of their background characteristics and in patterns of educational engagement. These students share many characteristics with both older students and commuters. For example, senior transfer students are more likely to:

• Be older, enrolled part-time, and drive to campus
• Spend time caring for dependents
• Be a first-generation student (neither parent graduated from college)
• Be a student of color
• Work at a job off campus
• Attend a Master?s institution.

In general, these characteristics often translate into less frequent and meaningful contact with peers and faculty members, involvement in campus activities and programs, and overall engagement in learning. Transfer students also perceive their campus environment to be less supportive of their needs. Specifically, transfer students are less likely to:

• Work with classmates outside of class to complete class assignments
• Tutor other students
• Use email to communicate with an instructor
• Talk about career plans with a faculty member or advisor
• Work with faculty members on activities other than coursework
• Be in a social fraternity or sorority
• Participate in co-curricular activities
• Spend time relaxing and socializing
• Perceive the campus environment as supportive of their social needs
• Believe the campus emphasizes attendance at campus events and activities
• Report positive relationships with other students.

However, transfer students are at least as academically focused as non-transfer students. For example, transfer students reported grades on par with non-transfer students and were just as academically challenged. In addition, they are more likely to rewrite a paper two or more times before turning it in and more likely to have done readings and assignments in preparation for class.

Before sharing your NSSE results institution-wide, become familiar with the nature of the data and ?story line? of your school?s performance. Here are some things to consider.

Check the Representativeness of Your Respondents

An essential early step is comparing your student respondents? demographic characteristics, summarized in the Frequency Distribution section, with your institutional data files for first-year and senior students. Women and some historically underrepresented groups are somewhat over-represented among NSSE 2002 respondents. Check to see if this is also true in your case and whether your respondents differ in any other ways from the profiles of your firstyear and senior students. The determination of student year in school (?firstyear? or ?senior?) is based on the information from the electronic file that your school provided to us last fall. The Frequency Distribution section contains students? responses to this question on The Report, which in a few cases may differ from the institution?s classification.

Another way to gauge representativeness is through sampling error, an estimate of the margin by which the ?true? score for your institution on a given item could differ from the reported score for one or more reasons, such as differences in one or more important characteristics between the sample and the populations. For example, if 60% reply "very often" to a particular item and the sampling error is +/- 5% there is a 95% chance that the population value is between 55% and 65%. Keep in mind that sampling error is based on the population of interest. If you want to estimate the sampling error for first-year male students, it must be calculated using the numbers of all first-year male students and the first-year male respondents (as contrasted with all undergraduates or all male and female firstyear students). Increasing the number of respondents relative to the total population reduces sampling error. For this reason some schools are increasing their sample size using NSSE?s oversampling.

Look for Patterns in Item Differences

In addition to focusing on items with medium to large effect sizes, look for patterns in your students? responses. For example, are your students consistently above or below the mean of your comparison group in certain areas of engagement? Are the differences explainable, perhaps a function of your school?s mission, the nature of the undergraduate program, or certain students? characteristics?

Also, don?t rely exclusively on statistical significance tests to identify areas that warrant attention. A consistent pattern of scoring above the mean, even though all items may not reach statistical significance, may indicate your institution is doing the right things in terms of good educational practice. At the same time, some institutions have very high expectations for student engagement and may fall short of their own aspirations even though comparisons with other institutions are favorable.

The Results are Unweighted

The data in the Means Summary Report comparisons are not weighted. That is, no adjustments were made to correct for potential bias in students? responses to approximate the populations of first-year and senior students at your school and other colleges and universities in your comparison groups. Later this fall, when we prepare the five national benchmarks of student engagement, we?ll use appropriate weighting techniques, similar to those employed in previous years, to make the appropriate adjustments. That said, the unweighted and weighted results for most NSSE items tend to be very similar at the institution, comparison group, and national levels. Some possible exceptions may be the reading, writing, and time on task questions (e.g., study hours, caring for dependents) at schools that have substantial proportions of part-time students, as they take fewer classes per term and cannot be expected to read and write as much as full-time students. Keep this in mind when interpreting the results.

Look Carefully at Items with Large Effect Sizes

In the Means Summary Report an asterisk (*) marks those items where your students? responses differ at a statistically significant level from students at schools in your respective comparison group(s) or at all NSSE 2002 institutions. The more asterisks reported for a particular item indicate a smaller probability that the difference noted is due to chance. ability that the differences noted are due to chance (p < .01 for consortia comparisons, p < .001 for Carnegie and national comparisons). Even so, the actual magnitude of some item score differences may seem trivial, even though they are highly reliable and statistically significant. For this reason, we also report the effect size associated with those item comparisons that are statistically significant. The effect size represents the magnitude of the discrepancy in the student or institutional behavior represented by the item. When the effect size is large, or a pattern of moderate effect sizes exists, it?s likely that the quality of the student experience is appreciably different and, therefore, may be of practical as well as statistical significance in the respective area of student engagement.

Finding large effect sizes is not that common in most areas of non-experimental educational and social science research including the NSSE project. If your results include some medium or large effects, something may be going on that warrants immediate attention, especially if other empirical or anecdotal information corroborate the NSSE data. Here are some general guidelines for determining the relative importance of a Cohen?s d effect size:

• .20 is a small effect
• .50 is a medium effect
• .80 is a large effect

If Your School is in a Consortium

If your school belongs to a consortium that used additional questions, the responses to these additional questions are included in the Means Summary Report and Frequency Distribution sections. These data are also in the institutional data file. Answers to such questions as ?What is your reason for working off campus?? and ?Who is your academic advisor?? have categorical response options that are meaningless when displayed in the Means Summary Report format. For this reason the response cells for such questions are empty. When presenting the results to categorical questions to colleagues and others, please use the information in the Frequency Distributions.

Take into Account Possible Mode-of-Administration Effects

Our analyses show that a mode-of-administration effect slightly favors schools where a high percentage of students completed The College Student Report 2000 via the Web. However, the differences that favor the Web mode have very small effect sizes. This phenomenon has also been noted by others using the Web for survey research and is discussed in more detail in the Appendix A. We still don?t know for sure whether this pattern of responses is a function ol to communicate with an instructor,? ?used an electronic medium to discuss or complete an assignment,? and self-reported gain in ?using computing and information technology?). We?ll continue to monitor this issue and alert you if our analyses lead us to modify our tentative conclusion that the Web mode has little practical impact on student responses to The College Student Report.

Review Responses to Experimental Questions (If Applicable)

In an effort to test potential survey items for future administration, beginning April 1st, 2002, a small set of experimental questions were added to the NSSE online survey. These questions were attached to the end of the survey and only students responding to the online version after this date received these extra questions. The experimental questions explored themes such as technology and distance education, spirituality development, student academic and social selfesteem, academic motivation, awareness of current news and events, and the impact of September 11 on student learning and experiences. Institutions were categorized into four groups and four sets of different experimental questions were administered to the respective groups.

If your institution participated in this item testing trial, the responses to the experimental questions are included in your institutional data file. Students? answers to the open-ended questions are also provided in an Excel file called ?Open-Ended Experimental Items.? However, due to their experimental nature and the small sample sizes for most of the participating institutions, these questions are not included in the Frequency Distribution Report and Means Summary Report. Rather, the frequency distributions and grand means by Carnegie types and at the national level are provided in a separate file named ?Experimental Item Summary by Carnegie and National? to inform institutional comparisons.

When reviewing your institution?s experimental item results, please pay attention to the number of respondents. If the number is small compared with your overall respondent group, interpret your results with extreme caution. The responses to the open-ended questions may contain rich information that might be important to understanding your students and informing institutional improvement efforts.

Consortium, Carnegie, and National Comparisons Do Not Include Oversampled Students

NSSE?s minimum sample sizes are determined by undergraduate enrollment (e.g., less than 4,000 students = 450; 4,000 to 15,000 students = 700; greater than 15,000 students = 1,000). It is possible to add students to the minimum sample size by oversampling in one of two ways: (1) all Web-only schools are oversampled using an algorithm based on undergraduate enrollment; and (2) some institutions request oversampling, which requires an additional fee. An increasing number of schools are using the oversampling option to add students to their sample to reduce sampling error and to insure an adequate number of respondents to analyze the information by major field, race and ethnicity, or other variables.

NSSE?s policy is to use only respondents from the institution?s standard random sample when developing the national benchmarks of effective educational practice and sector and national norms. This protects against the possibility that colleges and universities with oversamples might unduly influence the results. However, if your school requested an oversample, the responses of all your students (standard sample and oversample) are included in your institution's reports and data file.

Notes

1The NSSE 2002 number of respondents reported in the ?Overview? does not include the additional students who were oversampled. Oversampling was done at Web-only institutions and at schools that requested more of their students be surveyed than dictated by the NSSE sampling strategy, which is a function of institutional size. All in all, more than 118,350 students responded to the NSSE 2002 survey.

2 The NSSE 2002 average institutional response rates most likely underestimate the actual adjusted rate. Student postal service and e-mail addresses were based on fall 2001 enrollment information provided by the institutions. An unknown number of students in the sample were no longer eligible to complete the survey because they had dropped out or transferred to another institution. Even though first-class postage was used to guarantee the return of survey packets that could not be delivered, experience suggests that packets were not returned for some students who were no longer in school or residing at their fall 2001 address. In addition, institution provided email addresses were used to send students, at Web-only schools, their invitation to participate in NSSE 2002. We have found that many students have multiple e-mail accounts (e.g., Yahoo, AOL, Hotmail). Some institutions have more difficulty tracking these multiple email accounts and some students may not forward their institution assigned e-mail. Therefore, the actual response rate for Web-only institutions, when corrected for the unknown number of students who were no longer in school or did not receive the invitation to participate, is probably several percentage points higher than 39%. We are in the midst of checking undeliverable e-mails to students in an effort to more accurately estimate the Web-only response rate.

3 The regression of each cluster of items on a group characteristic is net of the following student and institutional controls: class, residence, gender, enrollment status, race/ethnicity, age, major, parental education, 2001 Barron?s admissions selectivity, sector, and 2000 Carnegie Classification.