National Survey of Student Engagement NSSE May Office of by ColeBer

VIEWS: 0 PAGES: 15

									2005 National Survey of
 Student Engagement
       (NSSE)




              May, 2006

    Office of Institutional Analysis
        University of Windsor
        www.uwindsor.ca/info
                          An Overview of Results from the 2005 NSSE

Background

The National Survey of Student Engagement (NSSE) is an annual survey developed by researchers at the
Indiana University Center for Postsecondary Research. It was first piloted in 1999 and has been used widely
in the U.S since 2000. In 2004, 11 Canadian universities participated for the first time. The University of
Windsor was one of 10 Canadian universities that took part in 2005 and in 2006, all Ontario universities are
participating as a group.

NSSE was designed to assess the level of engagement of undergraduate students in activities associated with
good educational practices and what they gain from their experiences at university. Research has shown that
students who are highly engaged in their studies and active on campus fare better academically and enjoy a
better learning experience than those who are not. There is some evidence from U.S. NSSE data suggesting
that students with lower SAT scores benefit more from engagement than their counterparts with higher SAT
scores (Carini, Kuh & Klein, 2004).

In February through April 2005, a link to the NSSE instrument was emailed to 2,800 randomly selected first-
year and fourth-year students at the University of Windsor. The total number of students who completed the
survey was 1,124; 530 first-year and 594 fourth-year students. The breakdown of respondents’ year by
faculty is shown in Table 1.

Table 1: Year by Faculty
Faculty         #     1st year (%)   4th year (%)
FASS           580            51%            49%
Business       189            32%            68%
Engineering     79            49%            51%
HK              62            34%            66%
InterFac        20          100%              0%
Nursing         48            54%            46%
Science        146            46%            54%
Overall       1124            47%            53%



In the 2005 NSSE, the University of Windsor along with seven other Canadian universities formed the
Canadian Consortium. In this report, selected results for Windsor respondents are compared to the aggregate
results of the seven other institutions in the consortium. The following universities are included in the
Canadian Consortium:

    •   Acadia University
    •   Carleton University
    •   Ryerson University
    •   University of New Brunswick – Fredericton Campus
    •   University of Ottawa
    •   University of Regina
    •   University of Windsor
    •   York University

The Windsor results are also compared to the results of a group of 40 U.S. Doctoral Research-Intensive
institutions (NSSE Peer Group), based on the Carnegie Classification of Universities. Examples of some of
the Doctoral Research-Intensive universities are Central Michigan University, Illinois State University,
Indiana State University and Oakland University.


                                                                                                            1
For the purposes of this report, a significant difference occurs when the mean difference is significant at the
.001 level. Since large sample sizes like those produced by NSSE tend to produce more statistically
significant differences than are meaningful, the effect size (ES) which indicates the “practical significance”
of the mean difference is also provided. The effect size is calculated by dividing the mean difference by the
standard deviation of the group Windsor is being compared to (either the Canadian Consortium or the NSSE
Peer Group). In practice an effect size of .2 is considered to be small, .5 moderate and .8 large. A practical
significant difference is one where both the mean difference is significant and the effect size is moderate or
large.

Respondent Characteristics

Respondent characteristics are shown in Table 2. The percentage of part-time Windsor respondents differs
somewhat from the Canadian Consortium and the NSSE Peer Group. Windsor has a larger percentage of
part-time first-year students (12%) and a smaller percentage of part-time fourth-year students (10%).

Generally, there is little difference between Windsor and the Canadian Consortium in terms of the
percentage living on-campus. However it is worthwhile to note that 51% of first-year Windsor respondents
either live on-campus or in accommodations within walking distance of the campus compared to 45% of the
Canadian Consortium. Almost half (48%) of fourth-year Windsor respondents live on-campus or within
walking distance compared to only 35% of the Canadian Consortium. In the NSSE Peer Group, about two-
thirds of first-year students and only 43% of fourth-year respondents live either on-campus or within walking
distance.


Table 2: Respondent Characteristics
                          Windsor                Cdn. Cons.      NSSE Peer Gp.

                           1st yr    4th yr    1st yr   4th yr   1st yr    4th yr
Gender
   Female                  62%       62%       60%      59%      64%       63%
   Male                    38%       38%       40%      41%      36%       37%
Class Level                47%       53%       49%      51%      48%       52%
Enrolment Status
   FT                      88%       90%       94%      83%      95%       83%
   PT                      12%       10%       6%       17%      5%        17%
Place of Residence
   On-campus               29%       3%        33%      6%       59%       11%
   Off-campus              71%       97%       67%      94%      41%       89%


Selected Results

Student-Faculty Contact

Students were asked how often they asked questions in class or contributed to class discussions. The
average Windsor response of both first-year and fourth-year students does not differ significantly from the
Canadian Consortium figures. The difference between Windsor and the NSSE Peer Group findings is
significant (p<.001, ES1st yr=.55, ES4th yr= .42), with Windsor respondents being much less likely to ask
questions or contribute in class. The results are illustrated in Figure 1.



                                                                                                              2
Figure 2 shows the response when students were asked how often they received prompt feedback from
faculty on their academic performance. Here again the difference between the Windsor response and that of
the NSSE Peer Group is significant (p<.001, ES1st yr=.51, ES4th yr=.48). The results for Windsor and the
Canadian Consortium are quite similar.


                     Figure 1: How often have you asked questions in class or
                                 contributed to class discussions?

                                             Never      Sometimes     Often      Very Often


                NSSE Peer Gp 2%            26%                 33%                     39%
   Final Year




                   Cdn. Cons 6%                   40%                  28%                   26%

                     W indsor    8%               37%                  30%                   25%




                NSSE Peer Gp 3%                  41%                   33%                   22%
   First-Year




                   Cdn. Cons         14%                     50%                   24%         12%

                     W indsor        14%                      53%                      24%         9%

                                0%           20%              40%      60%             80%           100%




                   Figure 2: How often have you received prompt feedback from
                       faculty on your academic performace (w ritten or oral)

                                             Never      Sometimes     Often      Very Often

                NSSE Peer Gp 3%            27%                       48%                     21%
   Final Year




                   Cdn. Cons     11%                   38%                    40%                  11%

                     W indsor 7%                   45%                           39%               9%
   First-Year




                NSSE Peer Gp 6%                  34%                       45%                 15%

                   Cdn. Cons         16%                     44%                  31%              9%

                     W indsor        17%                     46%                    28%            9%

                                0%           20%             40%       60%             80%          100%



                                                                                                            3
The response to the question: ‘how often have you discussed ideas from your readings or classes with faculty
members outside of class?’ indicates that this is not something that today’s university student commonly
does. See Figure 3 for details. Although statistically, senior Windsor respondents have these discussions
less often than their counterparts in the NSSE Peer Group, the effect of the mean difference is considered to
be small (p<.001, ES4th yr=.2)



                     Figure 3: How often have you discussed ideas from your
                    readings or classes w ith faculty members outside of class?

                                       Never      Sometimes     Often     Very Often


                NSSE Peer Gp         28%                  46%                  16%        9%

                                      37%                       44%                 13% 6%
   Final Year




                   Cdn. Cons

                     W indsor         36%                     43%                   17%    4%
   First-Year




                NSSE Peer Gp               44%                    38%               13% 5%

                   Cdn. Cons                50%                       34%            12% 4%

                     W indsor               52%                         31%          12% 5%

                                0%     20%          40%          60%          80%          100%




Quality of Student Relationships

Students were asked to rate the quality of their relationships with other students, faculty members and
administrative personnel and offices. Possible answers ranged from 1 to 7 with the higher numbers
indicating a more positive relationship.

The quality of relationships with other students is shown in Figure 4. A score of 1 represented ‘unfriendly,
unsupportive, sense of alienation’ and 7 was ‘friendly, supportive, sense of belonging’. About three-quarters
of Windsor first-year and fourth-year respondents reported positive relationships with other students. This
result does not differ in any meaningful way from the Canadian Consortium or NSSE Peer Group results.

Figure 5 illustrates how respondents rated their relationships with faculty members. For this question, a
score of 1 represented ‘unavailable, unhelpful, unsympathetic’ and 7 was ‘available, helpful, sympathetic’.
Not surprisingly, fourth-year students reported more positive relationships with faculty than did first-year
students. Both first-year and fourth-year Windsor respondents reported significantly lower scores than the
NSSE Peer Group with small effect sizes (p<.001, ES1st yr=.27, ES4th yr=.3).




                                                                                                               4
            Figure 4: Quality of relationships w ith other students

                             5 or higher on a 7-point scale

100%
                                                                           81%
                    76%                       76%               77%
80%     74%                         74%

60%


40%


20%


 0%
       1st year     Final         1st year    Final            1st year    Final
                    year                      year                         year
         W indsor                  Cdn. Cons.                   NSSE Peer Gp.




           Figure 5: Quality of relationships w ith faculty members

                              5 or higher on a 7-point scale

100%

                                                                            78%
80%                                                              72%
                    69%                       70%
        61%                         62%
60%


40%


20%


 0%
       1st year     Final          1st year   Final             1st year    Final
                    year                      year                          year
         W indsor                     Cdn. Cons.                   NSSE Peer GP.




                                                                                    5
Only about half in each group of respondents reported having high quality relationships with administrative
personnel and offices. The ratings ranged from 1 for ‘unhelpful, inconsiderate, rigid’ to 7 for ‘helpful,
considerate, flexible’. See Figure 6.



             Figure 6: Quality of relationships w ith admin personnel and
                                         offices

                                    5 or higher on a 7-point scale

   60%
                       52%                                               53%       53%
            48%                           49%        49%


   40%



   20%



    0%
          1st year     Final             1st year   Final              1st year    Final
                       year                         year                           year
            W indsor                     Cdn. Cons.                    NSSE Peer Gp.




Time Usage

The way in which students spend their time is affected to a large extent by their enrolment status at
university. One would expect full-time students to spend more hours preparing for class than their part-time
counterparts. NSSE provides each institution with its own response data, not the data from the comparison
groups so there is no way to examine time usage for only full-time students in the other institutions. For this
reason, in Figures 7 and 8 only the response for University of Windsor full-time students is shown.

Figure 7 suggests that at this stage of their university career, there is very little difference in the weekly
studying patterns of full-time Windsor first-year and fourth-year students.

Figure 8 shows how many hours in a typical week full-time Windsor respondents said they worked for pay,
either on or off campus. Close to three-quarters (72%) of full-time senior students were employed at the
time of the survey, compared to less than half (47%) of full-time first-year students.




                                                                                                                 6
                Figure 7: Time spent preparing for class (hours per w eek
                        studying, reading, homew ork, labw ork...)

                                1-10 hrs      11-20 hrs     21-30 hrs     over 30 hrs




          FT Final Year              43%                        36%               15%    6%
Windsor




          FT First-Year              42%                        34%             16%      8%




                          0%           25%                50%             75%             100%




                 Figure 8: Time spent w orking for pay (hours per w eek)

                                     0 hrs    1-10hrs     11-20 hrs     over 20 hrs




          FT Final Year        28%              22%              27%              23%
Windsor




          FT First-Year                 53%                      18%        16%         13%




                          0%           25%                50%             75%             100%




                                                                                                 7
Overall Satisfaction

NSSE asks two questions to get a sense of students’ overall satisfaction with their educational experiences.
The questions are “how would you evaluate your entire educational experience at this university so far?”
(Figure 9) and “if you could start over, would you attend the same institution you are now attending?”
(Figure10).

On average, first-year Windsor respondents in their second semester at university, were significantly less
favourable in their responses to both questions than their peers in the Canadian Consortium group although
the practical significance of both mean differences is less than moderate (p<.001, ES 1st yr=.3).

It is encouraging that for both questions, there was no significant difference between the scores of Windsor
fourth-year respondents and their Canadian Consortium counterparts as they are approaching graduation.

First-year and final year Windsor respondents scored significantly lower than those in the NSSE Peer group
on both questions (p<.001, ES<.4).



                    Figure 9: How w ould you evaluate your entire educational
                                  experience at this university?

                                             Poor   Fair    Good       Excellent


                NSSE Peer Gp 2% 14%                 52%                            32%
   Final Year




                   Cdn. Cons 4%    19%                      56%                      21%

                     W indsor 4%   20%                      57%                      19%




                NSSE Peer Gp 3% 14%                   55%                          28%
   First-Year




                   Cdn. Cons 3% 17%                        56%                      24%

                     W indsor 5%      23%                        57%                     15%

                            0%              25%            50%              75%            100%




                                                                                                               8
                    Figure 10: If you could start over again, w ould you go to the
                              same institution you are now attending?

                       Definitely no   Probably no    Probably yes    Definitely yes


                NSSE Peer Gp 6%        16%            42%                       36%
   Final Year




                    Cdn. Cons 6%       18%                 46%                   30%

                      W indsor 7%        23%                 46%                  24%




                NSSE Peer Gp 6%        14%            43%                       37%
   First-Year




                    Cdn. Cons 4% 12%                 47%                        37%

                      W indsor    8%    18%                 48%                   26%

                                 0%           25%           50%          75%            100%




NSSE Benchmarks

In order to quantify how effectively universities are contributing to learning, NSSE developed five indicators
or benchmarks of effective educational practice:

    •            The level of academic challenge faced by students
    •            The amount of active and collaborative learning taking place
    •            The quality of student-faculty interaction
    •            The availability of enriching educational experiences
    •            The supportiveness of the campus environment


For ease of comparison, each benchmark is expressed as a 100-point scale. The items included in each of the
benchmarks are listed in Appendix A. Details on how the benchmarks were constructed can be found on the
NSSE website.

Statistical comparisons indicate that there were no practical significant differences between the mean
benchmark scores of Windsor students and those of the Canadian Consortium. Note that while the average
first-year active and collaborative learning and fourth-year level of academic challenge scores were
significantly lower for Windsor respondents, the effect size of the differences was not meaningful (ES<.2).

Not unexpectedly, significant differences of medium effect size were found between Windsor and its U.S.
NSSE peer group. In particular, both first-year and senior Windsor respondents gave the University lower
scores for the quality of student-faculty interaction (p<.001, ES=.5). First-year Windsor respondents scored
lower on active and collaborative learning (p<.001, ES 1st yr=.5) and fourth-year students on average scored
lower on the enriching educational experiences measure (p<.001, ES 4th yr=.4). Windsor first and fourth-year
respondents also scored the University lower on the supportive campus environment benchmark although the
difference may not be important due to the less than moderate effect (p<.001, ES=.35).
                                                                                                              9
The mean benchmark scores for Windsor and its comparison groups are displayed in Figures 11 – 15.




                       Figure 11: Level of Academic Challenge

                         Windsor       Canadian Consortium     NSSE Peer Group


                 100


                  75
    Mean Score




                                           51.2         51.9       54.1        55.3
                       47.5     49.4
                  50


                  25


                   0
                              First-Year                         Final Y ear




                                                                                                    10
                   Figure 12: Active and Collaborative Learning

                       Windsor       Canadian Consortium      NSSE Peer Group

             100



              75
Mean Score




                                                                              51.2
                                                       45.9       45.6
              50                          41.2
                    33.2      36.0

              25



               0
                           First-Y ear                          Final Year




                     Figure 13: Student-Faculty Interaction

                      Windsor        Canadian Consortium     NSSE Peer Group

             100


              75
Mean Score




              50                                                             42.1
                                         32.9         32.6       31.9
                    23.5     23.6
              25


               0
                           First-Year                          Final Year




                                                                                     11
                       Figure 14: Enriching Educational Experiences

                           Windsor       Canadian Consortium     NSSE Peer Group

                 100


                  75
    Mean Score




                  50
                                                                                 39.9
                                                          32.3         33.2
                                  24.1         26.7
                         22.8
                  25


                   0
                                First-Year                        Final Year




                        Figure 15: Supportive Campus Environment

                            Windsor        Canadian Consortium        NSSE Peer Group

                 100


                  75
                                                57.4
    Mean Score




                                                                                        55.6
                         50.9       53.4                       49.6       49.8
                  50


                  25


                   0
                                 First-Y ear                           Final Y ear




More information

In addition to the survey instrument and how the benchmarks were constructed, detailed information on the
National Survey of Student Engagement can be found on their website: http://www.nsse.iub.edu.

All Ontario universities are participating in the 2006 NSSE with preliminary results expected in July 2006.


                                                                                                          12
13
                                                     Appendix A

                             NSSE Benchmarks of Effective Educational Practice

Level of Academic Challenge (11 items)
    •    Preparing for class (studying, reading, writing, rehearsing, and other activities related to your academic
         program)
    •    Worked harder than you thought you could to meet an instructor’s standards or expectations
    •    Number of assigned textbooks, books, or book-length packs of course readings
    •    Number of written papers or reports of 20 pages or more
    •    Number of written papers or reports between 5 and 19 pages
    •    Number of written papers or reports fewer than 5 pages
    •    Coursework emphasizes analyzing the basic elements of an idea, experience or theory
    •    Coursework emphasizes synthesizing and organizing ideas, information or experiences
    •    Coursework emphasizes making judgments about the value of information, arguments or methods
    •    Coursework emphasizes applying theories or concepts to practical problems or in new situations
    •    Campus environment emphasizes spending significant amounts of time studying and on academic work

Active and Collaborative Learning (7 items)
     •   Asked questions in class or contributed to class discussions
     •   Made a class presentation
     •   Worked with other students on projects during class
     •   Worked with classmates outside of class to prepare class assignments
     •   Tutored or taught other students
     •   Participated in a community-based project as part of a regular course
     •   Discussed ideas from your reading or classes with others outside of class (students, family members, co-
         workers, etc.)

Student Interactions with Faculty Members (5 items)
    •    Discussed grades or assignments with an instructor
    •    Talked about career plans with a faculty member or advisor
    •    Discussed ideas from your reading or classes with faculty members outside of class
    •    Worked with faculty members on activities other than coursework (committees, orientation, student-life
         activities, etc)
    •    Received prompt feedback from faculty on your academic performance
    •    Worked with a faculty member on a research project

Enriching Educational Experiences (5 items)
    •    Talking with students with different religious beliefs, political opinions or values
    •    Talking with students of a different race or ethnicity
    •    An institutional climate that encourages contact among students from different economic, social, and racial or
         ethnic backgrounds
    •    Using electronic technology to discuss or complete assignments
    •    Participating in:
              o internships or field experiences
              o community service or volunteer work
              o foreign language coursework
              o study abroad
              o independent study or self-designed major
              o culminating senior experience
              o co-curricular activities
              o learning communities

Supportive Campus Environment (6 items)
   •     Campus environment provides support you need to help you succeed academically
   •     Campus environment helps you cope with your non-academic responsibilities
         (work, family, etc.)
   •     Campus environment provides the support you need to thrive socially
   •     Quality of relationships with other students
   •     Quality of relationships with faculty members
   •     Quality of relationships with administrative personnel and offices



                                                                                                                          14

								
To top