Slide 1 - Wichita State University

Document Sample
Slide 1 - Wichita State University Powered By Docstoc
					    Student Engagement at WSU: What
    can we Learn From Five years of NSSE
    Information


        Office of Institutional Research




1
NSSE: National Study of Student Engagement

    • A national survey of first year students and
      seniors in 4-year colleges and universities.
    • It was designed to measure “engagement” or
      how students perceive their involvement in the
      total academic experience
    • Engagement is defined by George Kuh as:
      – What students do -- time and energy devoted to
        educationally purposeful activities both in and out of the
        classroom
      – What institutions do -- using effective educational
        practices to induce students to do the right things
2
    NSSE: National Study of Student Engagement

    • First full data collection in
      2000
    • 1300 different 4-year
      schools have participated
    • Averages 600+ schools
      for the last five years
    • Survey of a random
      sample of freshmen and
      seniors conducted in
      spring each year

3
    Questions

    • Students are asked to respond to questions
      about their experiences at the university
      including academics, university services, and
      various out of classroom activities.
    • 85 items with varying response options such as
      – How often have you… 1 Never to 4 very often
      – How would you rate… 1 very unfavorable to 7 highly
        favorable
      – How would you evaluate…1 poor to 4 excellent
      – How much emphasis… 1 very little to 4 very much


4
 Seven Principles of Good Practice in
 Undergraduate Education*

   Student-faculty contact
   Active learning
   Prompt feedback
   Time on task
   High expectations
   Experiences with
    diversity
   Cooperation among
    students
* Chickering & Gamson, 1987; Pascarella & Terenzini, 2005
5
              NSSE Indicators of
        Effective Educational Practice


                                        Active &
    Level of Academic                 Collaborative
       Challenge                        Learning


                         Enriching
                        Educational
                        Experiences



    Student – Faculty                  Supportive
       Interaction                      Campus
                                      Environment

6
The WSU Experience
Several administrations
Pooling data on several administrations
Providing college level results with sufficient
sample size
    Background Information

    NSSE has now been administered at WSU six
     times (2002, 2004, 2005, 2007, 2008 and 2009)
    Although part of two recent KBOR Performance
     Agreements, use by the academic colleges or
     other areas has been minimal.
    The Foundation of Excellence Project last year
     did use the results for the first year students.
    Academic colleges have expressed concern that
     the sample size in a single year is insufficient for
     their use.

8
    Thus …

The University
Assessment asked us to
see if we could
legitimately combine
several years’ data and
then consider differences
among the academic
colleges.




9
     Purposes of Current Analyses
     Reviewed data over several years to identify any
      trends or changes.
     Using the data from several survey administrations,
      disaggregate the WSU NSSE data to individual
      colleges by major with sample sizes large enough to
      encourage confidence in results.
     Provide information about groups of items that are
      conceptually and empirically confirmed as related
      (i.e. benchmarks and scalelets).
     Allow colleges and other campus groups the
      opportunity to use the NSSE as another component
      of their total outcome assessments.
     Limited to seniors since most logical at the college
      level.

10
Step 1: Merging data over multiple
survey administrations
1. Validated item consistency
2. Determined differences, if any, for WSU
   data over administrations
           Item consistency over time:
          Review using NSSE materials

     • Reviewed the detailed report produced by the
       NSSE staff for item changes over each
       administration since 2002.
       – Sufficient number of changes before 2004 that deletion
         of 2002 data seemed essential.
       – One item changed in 2004 but this item not used in our
         analysis
     • The wording of the survey items and the
       response options on the surveys are consistent
       over the years since 2004.

12
             Item consistency over time:
               Review using WSU Data
      Compared each item by class and year using Chi-
       Square or Analysis of Variance.
      Used complete responses (e.g. 1-4) and pooled
       responses (e.g. often/very often) as another means to
       validate any differences in an item over time.
      Removed 2004 and repeated the analyses. No
       additional meaningful differences noted.
      With this large number of cases, occurrence of
       significant differences by chance alone are highly
       probable.
      Reviewed items for meaningful differences if statistically
       significant.




13
Item consistency over time: Summary

     Differences in items over administrations were
      minimal.
     Fluctuations in scores rather than progressive
      pattern of changed was noted for some items.
     Responses from seniors were remarkably stable
     NSSE developers indicate that unless deliberate
      and systemic university-wide interventions were
      made, differences should be minimal.



14
     Two interesting items

        E-mail as a method for communication with
         faculty
            Increased from 49% in 2004 to 72% In 2009 as reported by
             freshmen.
            No differences for seniors
     University encouraging students to vote quite
     a bit or very much.
         Increases were noted in the two presidential election years i.e.
          2005 and 2009.
         In 2005 and 2009, freshmen reported encouragement at 30 %
          and 37% respectively but only 18-19% in other years.
         In 2005 and 2009, seniors reported encouragement at 24% and
          27% respectively but only 18-19% in other years.

15
Demographics
1. Sample versus total seniors
2. Characteristics of Students for sample
   versus total seniors
     Comparison of Sample and Total Enrollment
               by College (N=2173)

          College/Division   Sample   University*

     Business                 20%        20%
     Education                14%        13%
     Engineering              13%        11%
     Fine Arts                7%          7%
     Health Professions       11%        12%
     LAS - Humanities         5%          4%
     LAS - Math/Science       8%          7%
     LAS - Social Science     21%        20%
     LAS - Other              2%          2%


*Fall 2008
17
     Demographics of Seniors
                Sample   WSU   Comment
 >=24 yrs       53%      56%   No significant difference among the colleges.
Women           64%      56%   Clear gender differences. Women predominate in
                               HP(90%), ED(80%) & LAS SS(79%). Men
                               predominate in EN(77%) & LAS Math/SC(56%).
Minority        17%      18%   Range from 10% and 12% in Education and Fine
                               Arts to 21% and 22% in LAS Social Sciences and
                               LAS Math/Science respectively. Differences are not
                               statistically different (p>.001)
International   6%       6%    No significant difference among colleges
Full time       73%      63%   All colleges are above 70% full time.
Working(Pay)    83%      NA    Range form 71% n Education to 89% in Business.
                               All colleges above 70%.
Transfer        88%      87%   No difference among colleges
Hours
18
A few Individual Items
     Rating for Academic Advising and
     Satisfaction Entire Educational Experience




20
Interrelated items
        Benchmarks represent Areas of
             Effective Practice


                                         Active &
     Level of Academic                 Collaborative
        Challenge                        Learning


                          Enriching
                         Educational
                         Experiences



     Student – Faculty                  Supportive
        Interaction                      Campus
                                       Environment

22
     Scalelets versus Benchmarks
     • Scalelets include a small number of items related to
       specific concept that have empirically validated.
       (Pike,2002,2006)
     • 14 scalelets with 12 as subsections of the benchmarks
     • Advantages over benchmarks
        – More specific and meaningful concepts
        – More congruence between items and concept
     • Benchmark and scalelet scores are converted to 0-100
       scale.
     • The mean for the items in each scalelet and benchmark
       for each student were calculated if he/she answered 3/5
       or 60% of items
23
Academic Challenge
Benchmark: Academic Challenge
Scalelets: Course Challenge and Writing
     Level of Academic Challenge: Items
     1.    Preparing for class (studying, reading, writing, doing homework or lab work, etc.
           related to academic program)
     2.    Number of assigned textbooks, books, or book-length packs of course readings
     3.    Number of written papers or reports of 20 pages or more
     4.    Number of written papers or reports of between 5 and 19 pages
     5.    Number of written papers or reports of fewer than 5 pages
     6.    Coursework emphasizes: Analysis of the basic elements of an idea, experience
           or theory
     7.    Coursework emphasizes: Synthesis and organizing of ideas, information, or
           experiences into new, more complex interpretations and relationships
     8.    Coursework emphasizes: Making of judgments about the value of information,
           arguments, or methods
     9.    Coursework emphasizes: Applying theories or concepts to practical problems or
           in new situations
     10.   Working harder than you thought you could to meet an instructor's standards or
           expectations
     11.   Campus environment emphasizes: Spending significant amount of time studying
           and on academic work.


25
     Level of Academic Challenge: Mean Scores




26
     Course challenge

 How often…Worked harder than you thought you could
                                                                WORKHARD
 to meet an instructor's standards or expectations

 How often…Come to class without completing readings or
                                                              CLUNPREP(reverse)
 assignments


 To what extent have your examinations during the current
                                                                   EXAMS
 school year challenged you to do your best work.


 How many hours per week spent…Preparing for class
 (studying, reading, writing, doing homework or lab work,
                                                                 ACADPR01
 analyzing data, rehearsing, and other academic activities)

 To what extent does your institution emphasize spending
 significant amounts of time studying and on academic            ENVSCHOL
 work
27
       Writing

     How often have you...Prepared two or
     more drafts of a paper or assignment         REWROPAP
     before turning it in

     How often have you...Worked on a paper or
     project that required integrating ideas or   INTEGRAT
     information from various sources

     In current school year…Number of written
                                                  WRITEMOR
     papers or reports of 20 pages or more

     In current school year…Number of written
                                                  WRITEMID
     papers or reports between 5 and 19 pages

     In current school year…Number of written
                                                  WRITESML
     papers or reports of fewer than 5 pages
28
     Higher-Order thinking Skills
     Memorizing facts, ideas, or methods from
     your courses and readings so you can repeat    MEMORIZE(reverse)
     them in pretty much the same form
     Analyzing the basic elements of an idea,
     experience, or theory, such as examining a
                                                        ANALYZE
     particular case or situation in depth and
     considering its components
     Synthesizing and organizing ideas,
     information, or experiences into new, more        SYNTHESZ
     complex interpretations and relationships
     Making judgments about the value of
     information, arguments, or methods, such
     as examining how others gathered and              EVALUATE
     interpreted data and assessing the soundness
     of their conclusions
     Applying theories or concepts to practical
                                                       APPLYING
     problems or in new situations
29
     Academic Challenge: Related Scalelets




30
Active and Collaborative Learning
Collaborative Learning
Active Learning
     Active & Collaborative Learning
     1. Asked questions in class or contributed to class
        discussions
     2. Made a class presentation
     3. Worked with other students on projects during class
     4. Worked with classmates outside of class to prepare
        class assignments
     5. Tutored or taught other students (paid or voluntary)
     6. Participated in a community-based project (e.g.,
        service learning) as part of a regular course
     7. Discussed ideas from your readings or classes with
        others outside of class (students, family members, co-
        workers, etc.)


32
     Active Learning

     How often have you...Asked questions
     in class or contributed to class        CLQUEST
     discussions

     How often have you..Made a class
                                             CLPRESEN
     presentation

     How often have you..Participated in a
     community-based project (e.g. service   COMMPROJ
     learning) as part of a regular course




33
      Collaborative Learning


     How often have you...Worked with
                                                 CLASSGRP
     other students on projects during class


     How often have you...Worked with
                                                 OCCGRP
     classmates outside of class
     How often have you...Tutored or
     taught other students                        TUTOR
     (paid or voluntary)
     How often have you...Discussed ideas
     from your readings or classes with
                                                 OOCIDEAS
     others outside of class (students, family
     members, co-workers, etc.)
34
     Active and Collaborative Learning




35
Faculty/Staff – Student Interaction
Course Interaction
Out of Classroom Interaction
     Course Interaction

     How often have you...Discussed
     grades or assignments with an           FACGRADE
     instructor

     How often have you..Discussed ideas
     from your readings or classes with      FACIDEAS
     faculty members outside of class

     How often have you..Received prompt
     written or oral feedback from faculty   FACFEED
     on your academic performance



37
      Out-of-Class Interaction

     How often have you...Talked about
     career plans with a faculty member            FACPLANS
     or advisor
     How often have you...Worked with
     faculty members on activities other
                                                   FACOTHER
     than coursework (committees,
     orientation, student life activities, etc.)
     How often have you...Work on a
     research project with a faculty member
                                                     RES
     outside of course or program
     requirements




38
     Course Interaction and Outside Class
     Interaction




39
Enriching Educational Experiences
Varied Experiences
Information Technology
Diversity
     Varied Experiences(Part I)

     Have you or plan to do ... A practicum,
     internship, field experience, co-op       INTERN04
     experience, or clinical assignment
     Have you, or plan to.. Community
                                               VOLNTR04
     service or volunteer work
     Have you, or plan to.. Participate in a
     learning community or some other
     formal program where groups of            LRNCOM04
     students take two or more classes
     together
     Have you, or plan to.. Foreign language
                                               FORLNG04
     coursework
     Have you or do you plan to.. Study
                                               STDABR04
     abroad
41
     Varied Experiences(Part II)

     Have you, or plan to.. Independent study or self-designed
                                                                 INDSTD04
     major

     Have you, or plan to.. Culminating senior experience
     (capstone course, senior project or thesis, comprehensive    SNRX04
     exam, etc.)
     How many hours per week...Participating in co-curricular    COCURR01
     activities (organizations, campus publications, student
     government, fraternity or sorority, intercollegiate or
     intramural sports, etc.)

     To what extent does your institution emphasize… Attending
     campus events and activities (special speakers, cultural
                                                                 ENVEVENT
     performances, athletic
     events, etc.)


42
     Technology
     How often have...Used an electronic
     medium (listserv, chat group, Internet,
                                               ITACADEM
     instant messaging, etc.) to discuss or
     complete an assignment
     How often have.. Used e-mail to
                                                EMAIL
     communicate with an instructor
     to what extent does your institution
     emphasize...Using computers in            ENVCOMPT
     academic work




43
     Diversity

     How often have you...Had serious
     conversations with students of a
                                             DIVRSTUD
     different race or ethnicity than your
     own
     How often have you...Had serious
     conversations with students who are
     very different from you in terms of     DIFFSTU2
     their religious beliefs, political
     opinions, or personal values
     To what extent has your institution
     encouraged contact among students
                                             ENVDIVRS
     from different economic, social, and
     racial or ethnic backgrounds

44
     Varied Experiences, Technology, Diversity




45
Supportive Campus Environment
Support for Student Success
Interpersonal Environment
     Support for Student Success

     To what extent does your institution
     emphasize… Providing the support
                                             ENVSUPRT
     you need to help you succeed
     academically

     To what extent does your institution
     emphasize...Helping you cope with
                                             ENVNACAD
     your non-academic responsibilities
     (work, family, etc.)

     To what extent does your institution
     emphasize...Providing the support you   ENVSOCAL
     need to thrive socially



47
     Interpersonal Environment

     Relationships with other students    ENVSTU




     Relationships with faculty members   ENVFAC




     Relationships with administrative
                                          ENVADM
     personnel and offices




48
     Support for Student Success and Supportive
     Interpersonal Environment




49
Outcome Measures
Practical Skills
General Education
     Gains in Practical Skills

     To what extent does your institution
     emphasize...Using computing and        GNCMPTS
     information technology

     To what extent does your institution
     emphasize...Analyzing quantitative     GNQUANT
     problems

     To what extent does your institution
     emphasize...Acquiring job or work-
                                            GNWORK
     related knowledge
     and skills



51
     Gains in General Education

     To what extent does your institution
     emphasize...Writing clearly and         GNWRITE
     effectively
     To what extent does your institution
     emphasize...Speaking clearly and        GNSPEAK
     effectively
     To what extent does your institution
     emphasize...Thinking critically and     GNANALY
     analytically
     To what extent does your institution
     emphasize...Acquiring a broad general   GNGENLED
     education


52
     Practical Skills and General Education




53
     Thank You




54

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:4/22/2013
language:English
pages:54