The psychometric analyses show
that the vast majority of items on The College Student
Report are valid and reliable and have acceptable kurtosis
and skewness indicators. What cannot be demonstrated from
such psychometric analyses is whether respondents are
interpreting the items as intended by the NSSE Design
Team and whether students' responses accurately represent
their behaviors and perceptions. That is, even when psychometric
indicators are acceptable, students may be interpreting
some items to mean different things.
It is relatively rare that survey researchers go into
the field and ask participants to explain the meaning
of items and their responses. However, because of the
importance of the NSSE project, we conducted focus groups
of first-year and senior students during March and April
2000 at eight colleges and universities that participated
in NSSE 2000. The schools included four private liberal
arts colleges (including one woman's college) and four
public doctoral-granting universities. Between three and
six student focus groups were conducted on each campus.
The number of students participating in the groups ranged
from 1 to 17 students, for a total of 218 student participants.
More women (74%) and freshmen (52%) participated than
men (26%) and seniors (48%). Approximately 37% were students
of color. Although there was not enough time to discuss
every item during each focus group, every section of the
instrument was addressed in at least one group on each
campus.
In general, students found The Report to be clearly worded
and easy to complete. A few items were identified where
additional clarity would produce more accurate and
|
consistent interpretations.
For example, the "number of books read on your own"
item confused some students who were not sure if this
meant reading books for pleasure or readings to supplement
those assigned for classes. This item is an illustration
of a handful of items where students suggested that we
provide additional prompts to assist them in understanding
questions. However, students generally interpreted the
item response categories in a similar manner. The meanings
associated with the response sets varied somewhat from
item to item, but students' interpretations of the meaning
of the items were fairly consistent. For example, when
students marked "very often" to the item "asked
questions in class or contributed to class discussions"
they agreed that this indicated a daily or during every
class meeting. When answering the "made a class presentation"
item, students told us that "very often" meant
about once a week.
The information from student focus groups allows us to
interpret the results with more precision and confidence.
This is because the focus group data indicated that students
consistently interpreted items in a similar way and that
the patterns of their responses accurately represent what
they confirm to be the frequency of their behavior in
various areas. We also have a better understanding of
what students mean when they answer various items in certain
ways. In summary, we are confident that student self-reports
about the nature and frequency of their behavior are reasonably
accurate indicators of these activities. For additional
detail about the focus group project review at the Ouimet,
Carini, Kuh, and Bunnage (2001) paper on the NSSE website.
|
