Slide 1 - University of South Carolina by lanyuehua

VIEWS: 3 PAGES: 81

									                                   Maximizing Your
                          Assessment by Combining
                            Student Survey Results:
                               EBI, NSSE, and CIRP
Jillian Kinzie, Ph.D.
Associate Director, Center for Postsecondary Research and National Survey of Student
Engagement (NSSE) Institute

John Pryor, M.A.
Director, Cooperative Institutional Research Program, Higher Education
Research Institute

Darlena Jones, Ph.D.
Director of Research and Development, Educational Benchmarking
Session Objectives
   Introduction
   Overview of CIRP, NSSE, and EBI
   Data Triangulation
   Using Studies to Measure…
       Retention
       Integration to Campus
       First-Year Program Improvement
       Learning Outcomes
   Using Multiple Data Sources
   Other Data Considerations
   Small Group Discussion
Introduction
CIRP: Cooperative Institutional Research Program




                    What is CIRP?
Introduction
   CIRP surveys have been
    used for decades to
    understand and illustrate
    the impact of college.

   Can use the surveys as
    cross-sectional surveys,
    but the key advantage of
    CIRP surveys is that they
    are longitudinal.
        Astin’s I – E – O Model
                                     Environments
                                      YFCY/CSS
                                (e.g., place of residence
                               during college, interactions
                            with peers and faculty, curricular
                             and co-curricular experiences)




        Inputs                                                   Outcomes
CIRP Freshman Survey                                             YFCY/CSS
   (e.g., academic performance
                                                           (e.g., satisfaction with college,
in high school, financial concerns
                                                          retention, gains in college, post-
        prior to college entry,
                                                                    college plans)
     expectations for college,
         degree aspirations,
    self-concept in high school)
   Higher Education Research Institute at UCLA




           Cooperative Institutional          Funded
           Research Program                   Research



Freshman
 Survey          YFCY              CSS   • Ford Foundation
                                         • Templeton Foundation
                                         • National Institutes
                                               of Health

     Faculty Survey
      CIRP Freshman Survey (TFS)

   Largest and longest-running national study of
    American college students…2008 is 43nd
    administration


   Initiated in 1966 at the American Council on
    Education; Housed at HERI (UCLA) since 1973
CIRP Freshman Survey (TFS)

   Three types of input items:
     Pretests on possible outcomes measures
     Self-predictions about possible future
      outcomes
     Personal characteristics that might effect
      the propensity to attain certain outcomes
1966                    2007


 206,865            374,261
 respondents    respondents

 251                    536
 Colleges         Colleges
 and                    and
 Universities   Universities



    Total over 42 years:
     13,408,942 students
1,708 colleges and universities
   Your First College Year
    (YFCY)

       Launched in 2000
          208,157 students at 378
           schools

       Designed to assess academic
        and personal development over
        the first year of college

       Developed in collaboration with
        the Policy Center on the First
        Year of College
Your First College Year Survey (YFCY)
   Mission: Longitudinal assessment of student development
    and institutional/programmatic impact during the first year of
    college

   Study Launch: Pilot Studies in Spring 2000 & 2001; National
    Assessment launched in Fall 2002

   Data Collected: Since its inception, surveyed nearly 150,000
    first-year students at nearly 250 institutions nationwide

   Administration: Disseminated at end of the first year in an
    on-campus administration format; paper and/or web versions;
    overall response rates 35-82%.
YFCY…
   Comprehensive in content to assess academic & social
    experiences as well as cognitive & affective development
   Designed as a follow-up instrument:
       40% of items are direct post-tests to questions on the
        CIRP Freshman Survey
   Survey Content
      Academic achievement & engagement
      Learning strategies and pedagogical practices
      Residential & employment experiences
      Interactions with family, peers, faculty & staff
      Patterns of behavior
      Student values and life goals
      Satisfaction, self-concept, & feelings of personal
       success
      Plans for the next academic year
   Space for 20 questions of local relevance
YFCY Deliverables and Data…

   Standard Deliverables     Additional Data Services
    Institutional Profile     Institutional Data in
                                PowerPoint Executive
    Longitudinal Profile       Summary
    Electronic Data File      Peer Group Reports
    Reports on                Data Merges
     Spreadsheet
Using YFCY Data to Enhance
Campus Assessment Efforts
          Purposes                         Methodologies
   Self-study reports &             Descriptive analyses with
    strategic planning                campus data
   Retention studies                Comparative analyses
   Examining group differences         Institutional subgroups
    among students                      Institutional vs. national data
   Programmatic assessment             Aspirant group or consortium
   Measuring student                Measures of association
    development & institutional
    impact
                                     Factor analyses
   Feedback to campus               Trends analyses
    constituents                     Multivariate analyses
   Creation of a student
    information system
NSSE: National Survey of Student Engagement




                    Who is NSSE?
NSSE Stats…
    Launched in 2000, supported by institutional participation fees
    Parent survey CSEQ (1979)
    National design team created survey
    Administered by Indiana University Center for Postsecondary
     Research
    More than 1,300 different baccalaureate colleges/universities from
     50 states, Puerto Rico, and Canada (CCSSE is for 2-year
     institutions)
    Consortia can add up to 20 questions (e.g., women’s colleges,
     HBCUs, state systems)
    Spring administration to first-year & senior students
    Third party, standardized administration; average 40% response
     rate, random sample, and select comparison groups.
NSSE Purposes
      Provide reliable indicators of “good
       educational practices”

      Support institutional improvement and
       accountability efforts

      Foster comparative & consortium activity

      Refocus conversations about quality in
       undergraduate education

      Direct measure of frequency of participation
       in educational practices associated with
       learning and development.
Survey Instrument


     College student
          survey that
 assesses the extent
   to which students
           engage in
         educational
            practices
     associated with
       high levels of
        learning and
        development
Two Components of Student Engagement

   What students do – time and
   energy devoted to
   educationally purposeful
   activities


   What institutions do – using
   effective educational
   practices to induce students
   to do the right things
NSSE Reports
    Comparative data (National, Carnegie Class, Select
     Comparison Groups, and/or Consortium)
      Respondent     Characteristics
      Frequency    Distributions
      Mean   Comparisons
      Benchmark     Comparisons
      Multi-year   Benchmark Comparisons
      Pocket   Guide Report (for prospective students)
      Executive    Snapshot Report
    Data File [so you can link to other data!!]
BCSSE – Beginning College Survey of Student Engagement
    Based on requests for related pre-
     college measures
    Measures selected high school
     experiences and students interest in and
     expectations for participating in
     educationally purposeful activities during
     college
    Designed to be combined with spring
     NSSE data for “pre- and post”- views of
     the first-year experience; Use to examine
     gap between expectations and
     engagement and study effect of students’
     background on NSSE scores
    Can also be used as stand alone
     assessment
    Pilots in 2005, 2006; Officially launched
     in 2007
NSSE Possibilities
   Merging NSSE data with         Comparisons against
    school records                  aspirational, regional,
   Descriptive displays of         and mission-related
    engagement patterns by          institutions
    any number of student          Program assessment
    characteristics                Accreditation reporting
   Prediction models for          Consortium and
    retention, degree               system data sharing
    attainment, grades, other      Scholarly research
    outcomes
   Tracking student
    engagement year to year
EBI: Educational Benchmarking, Inc.




                    Who is EBI?
EBI Stats…
    Founded in 1994 by Joseph Pica, Ed.D and Glenn Detrick (retired
     in 2002)
    Over 1500 Colleges and Universities (U.S.A. and 15 other
     countries like Australia, Mexico, Spain, and Egypt) have
     participated in EBI’s studies
    Surveyed over 12 million people
    Nearly 100 Assessments in 10 areas of higher education
    Over 500 custom assessments for schools like MIT, Ohio State,
     UCLA, University of Georgia, and University of Florida
    Produced over 17,000 customized reports
    Creation of WESS
EBI’s Partners (alphabetical order)
       AACN (American Association of Collegiate Nursing)

       ACUHO-I (The Association of College and University
       Housing Officers International)

       ACUI (Association of College Unions International)

       AFA (Association of Fraternity Advisors)

       Ball State University –

       MHLI (Military Housing and Lodging Institute)

       OTC (Outside the Classroom)

       Policy Center on the First Year of College
Relevant Assessments…

EBI First-Year Initiative (FYI)
    Mission: To assess the effectiveness of first-year courses /
     seminars.
    Survey Development: Created in conjunction with the Policy
      f
     Center for the First Year of College. Initial funding of project by the
     Pew Charitable Trusts. Currently funded by institution participation.
    Study Stats: Pilot Study in Spring 2001; National Assessment
     launched in Fall 2001. 169 colleges and universities have submitted
     over 185,000 responses.
    Why Participate: Provides institution information on course
     effectiveness. Provides instructors’ their class results for individual
     improvement.
Relevant Assessments…

ACUHO-I/EBI Resident Assessment
    Mission: To assist college and universities in assessing resident
     satisfaction of residence hall functions and the effectiveness of
     residence hall life.
    Survey Development: Created in conjunction with ACUHO-I.
     Housing professionals collaborated with EBI to design survey
     questions.
    Study Stats: National Assessment began Spring 1998. Since
     then, 567 colleges and universities worldwide have submitted over
     2.6 million responses.
    Why Participate: Provides evidence of student satisfaction of their
     on-campus living experience and student learning outcomes linked
     to CAS Standards.
Relevant Assessments…


MAP-Works
    Mission: An early warning indicator of retention and student
     success
    Survey Development: Created in conjunction with Ball State
     University. Sherry Woosley from BSU collaborated with EBI to
     design survey questions.
    Study Stats: Study piloted in 2006 – full national study in 2007.
     Over 40 institutions are utilizing MAP-Works in 2008
    Why Participate: Provides reporting directly to each first-year
     student regarding their behaviors benchmarked against their first-
     year cohort. Provides information directly to advisors, hall
     directors, and FYE instructors regarding their students so early
     intervention can occur.
Relevant Assessments…

FoE/EBI Foundation of Excellence Student
and Faculty/Staff Assessments
    Mission: Assist campus professionals in the self-study of their
     first-year program in support of the Foundations of Excellence
    Survey Development: Created in conjunction with the Policy
     Center on the First Year of College. Randy Swing, John Gardner,
     and Betsy Barefoot collaborated with EBI to design survey
     questions.
    Study Stats: Studies piloted Summer 2005. National
     Assessment launched Fall 2005. Since 2005, 67 schools have
     participated in the 4 year study and 39 schools have participated
     in the 2 year study.
Assessment Features…
 General Features…
    Professional Survey Development: EBI’s survey development
     team working in partnership with professional organizations
    Data Collection: Paper surveys (where available) or online
    WESS: State-of-the art data collection, reporting, and distribution
     system. Programmed and maintained by EBI software engineers.
        View Reports Online: Provides visualizations of results and ability to
         download responses and/or calculated data for higher-level analysis
        Share Reports Online: Ability to allow access to online reporting to others
        Written Analysis: Analysis notebooks with full detailed descriptive analysis
         (factor and question means, frequency distributions, etc.) Analysis also
         available online in PDF format.
Assessment Features…
 Comparisons…
    Inter-Institutional Comparisons: Provide comparative
     information for high level decision-making and resource allocation
    Intra-Institutional Comparisons: The ability to code units
     (areas/hall/floors, chapters, or course sections) for internal
     comparison to support internal improvement
    Longitudinal Comparisons: Provide trend data to assess the
     impact of initiatives
 Customization…
    Statistical Analysis: Provides statistical testing and information
     for resource allocation
    Institutional Specific Questions: Institutions may add up to 10
     questions (5 can be open-ended if using our online surveying
     system, WESS)
Data Triangulation
What Is Data Triangulation?
   Application and combination of several data points or
    sources to overcome weaknesses of single-measure
    studies


Why Use Data Triangulation?
   Increase confidence in findings through convergence
    of different perspectives
   The point at which perspectives converge is seen to
    represent reality
   Builds a rich data resource
                Astin’s I-E-O Model
                             Environments
                          CSS, YFCY, EBI, NSSE
                              (e.g., place of residence,
                         interactions with peers and faculty,
                       engagement, programmatic features,
                       curricular & co-curricular experiences)


         Inputs
                                                          Outcomes
CIRP Freshman, BCSSE,
                                                         YFCY, CSS,
      MAP-Works
    (e.g., academic performance
                                                         EBI, NSSE
                                                  (e.g., post-college aspirations,
 in high school, financial concerns
                                                     satisfaction with college,
         prior to college entry,
      expectations for college,                  academic and social adjustment,
                                                     degree completion rates)
          degree aspirations,
     self-concept in high school)
Similarities and Differences
 Comparing these instruments, we’ll be answering
   the questions...
  What are the unique features of these instruments?
  What are the similarities between these instruments?
  What are the differences between these instruments?
  How do these instruments complement each other?


 Next, how these instruments address…
  Retention
  Integration
  Learning Outcomes
  First Year Program Improvement
Retention
NSSE and Retention…
 Links between NSSE & Retention Goals:
    NSSE is a stimulus for reflection on what the institution
     does well & areas for improvement
    Focuses attention on ultimate goal of persistence –
     improved educational experiences (not just retention)
    Identifies what distinguishes the institution, where
        Student experience matches espoused mission
        Student experience falls short
    NSSE results can be connected to institutional data to
     build persistence models
NSSE and Retention…
 NSSE items and Tinto’s Model of Student Persistence
  Students engagement in effective educational practices
   as indication of level of academic & social integration
   (1a-v; 6a-f; 9a-g etc.)
  Institutional environment items indicate the level of
   commitment to student success (10a-g)
  Student intentions to engage in educational activities
   before graduating (7a-h)
NSSE and Retention…
    Retention Indicators
        Monitor satisfaction
        Examine intent to engage in enriching educational
         experiences
        Identify problematic student behaviors (e.g., % preparing
         for class; % preparing 2 or more drafts; frequency of
         contact with faculty outside of class)
    Retention Studies
        Study relationship between engagement and first-year
         retention
        Compare persisters vs. non-persisters
        Study program impact (e.g., first-year learning
         communities; course-based service experience)
NSSE and Retention at one institution…
                 Retention to Second Year by Engagement
    100%                                  95%
                       93%                                                           93%                    93%
                    91% 92%            91%                   92%
                                                           91% 91%
                                                                                   89%                 89%
                                                                                                          92%
                                     88%                                        88%
                                                                              87%
        90%      84%              84%               85%86%                                           85%
                                            84%                                                   82%
                                                                           82%
              79%
        80%

        70%

        60%

        50%

        40%

        30%

        20%

        10%

        0%
                                   Academic and
              Level of Academic                     Student Interaction   Enriching Educational   Supportive Campus
                                    Collaborative
                  Challenge                            w ith Faculty           Experiences           Environment
                                      Learning

 Very Low           79%                84%                 85%                    82%                   82%
 Low                84%                88%                 86%                    87%                   85%
 Average            91%                91%                 91%                    88%                   89%
 High               93%                95%                 92%                    89%                   92%
 Very High          92%                84%                 91%                    93%                   93%
CIRP and Retention…
    CIRP Freshman Survey asks students to predict the
     likelihood that they will transfer, stop-out, or leave
    YFCY asks if they have transferred from a different
     institution; CIRP TFS pre-test data will be merged with
     the YFCY responses even if the pre-test was
     administered at a different institution
    YFCY asks students to share their plans for their
     sophomore year, i.e., intent to re-enroll
    Data can be merged with information from the registrar
     to compare predicted vs. actual retention rates
CIRP and Retention…
    One of the biggest problems with studying retention is
     that it is inextricably linked with who you admit (pre-
     college characteristics and performance)
    Longitudinal assessment (e.g. CIRP Freshman
     Survey and YFCY) allows institutions to control
     potentially-biasing background characteristics when
     assessing first-to-second year retention rates

     “Many of the so-called highly productive institutions turn out to
      be underproductive when the caliber of their entering students
      is taken into account.” Alexander Astin, 2003
CIRP and Retention…
 Consideration of “inputs” and “environments” in retention
 analyses identifies populations of at-risk students and
 programs/experiences that facilitate or inhibit persistence
    Ex. What entering characteristics positively predict first-to-
     second year enrollment? Emotional health is a positive
     predictor, but recent cohorts of students are exhibiting more
     stress & greater levels of depression
    Ex. How do service-learning, first-year seminars, and
     learning communities affect the decision to re-enroll?
     Service learning is a positive predictor on its own; First-year
     seminars and learning communities are effective when taken in
     combination
EBI and Retention…
                                               Enrollment
                                              Management/R
                                                etention
                                 Academic                            Student
                                  Affairs                            Affairs


   Who is responsible for student
    success on your campus?
       Enrollment Management/
        Retention?
       Student Affairs?                    Student ID: YD252952
                                            HS GPA: 3.93
       Academic Affairs?                   SAT Verbal: 29
                                            Location: In state
                                            Gender: Female
                                            Race: African American
   What information do you know            Age: 18
                                            Major: Undecided
    about this first-year student?
                                      Do you really know them?
EBI and Retention… I’m thinking                                          I’m struggling
                                                                           in my math
                                           about         Enrollment           class
                                        transferring    Management /
Paradigm Shift                          Academic
                                                         Retention
                                                                            Student
   What would happen                                                       Affairs
                                         Affairs
    if…                                                                  Residence
       ALL faculty/staff were          First-Year                       Hall Staff
        responsible for                   Seminar
        student success?                 Instructor                        Academic
                                                                            Advisor
      YOU knew            I’m really
        student was        homesick        Financial
                                               Aid                       Academic
        struggling?                                                     Department
   Could you do something                                Minority        Heads
    about it before it was too
                                      I don’t think I can Student
    late?                               afford college    Affairs
                                                                       My roommate
                                                                       and I argue all
                                                                         the time
          MAP-Works Process
• Expectations
                       • Student Profile
• Behaviors
                       • Institution Profile
                       • Campus Resources




  • Social Norming     • Student
  • Expectations       Summary
  • Campus             • Scan Students
    Resources
Integration to Campus
NSSE and Engagement…

 What is the level of student engagement on
  your campus?
    NSSE data demonstrates what students do and what
     students believe the institution emphasizes
    Results can be used to shape new student behavior
    Criterion reference benchmarking to compare against
     predetermined value (e.g., 20% FY students worked
     with peers on assignments outside of class – should this
     be higher??)
    CIRP and Student Involvement
Astin’s Involvement Theory

   The amount of student learning and development associated with
    any educational program is directly proportional to the quantity
    and quality of student involvement in that program

   The effectiveness of any educational policy or practice is directly
    related to the capacity of that policy or practice to increase
    student involvement
  CIRP and Academic Involvement
                                                     CIRP   CIRP
                                                     TFS    YFCY
Studied > 6 hours/week                               49%    68%
Look up scientific research articles and resources   23%    29%
Revise your papers to improve your writing           54%    53%
Seek feedback on your academic work                  49%    42%
Ask questions in class                               59%    34%
Been bored in class                                  38%    37%
Asked professor (teacher) for advice after class     28%    15%
“A” average                                          38%    21%
  CIRP and Co-Curricular Involvement

                                             CIRP   CIRP
                                             TFS    YFCY
Performed volunteer work                     89%    63%
Attended religious services                  79%    53%
Socialized with Friends >6 hrs/week          70%    79%
Student Clubs or Groups > 3 hrs/week         40%    31%
Online Social Network Sites > 3 hours/week   43%    56%
EBI and Interaction…
 First-Year Initiative Assessment
    Course Improved Connections with Faculty
    Course Improved Connections with Peers
    Sense of Belonging and Acceptance

 AFA/EBI Fraternity/Sorority Assessment
    “Fraternity/Sorority Enhanced Interpersonal Relationship Skills”
        Top predictor of Overall Program Effectiveness
EBI and Interaction…
FoE/EBI Foundations of Excellence Student
  Assessment
   Top two predictors of Overall Evaluation of Program:
        Overall Evaluation: Transition Support
        All Students: Campus Environment


ACUHO-I/EBI Resident Assessment
   “Interaction with Others in the Hall” is a top predictor
    of Overall Evaluation of Program for 90% of schools
EBI and Interaction…
MAP-Works – all about Interaction…
   Levels of Personal One-on-One Interaction
       Student to Faculty/Staff (like FYE Instructor / Advisor / Hall Director)
       Faculty/Staff interactions with other Faculty/Staff
       Upper Administration interaction with Faculty/Staff
       Upper Administration interaction with Student
   Aggregate Study of Student Interaction
       The factor, “Social Integration” is top predictor of a successful social
        transition to college
First-Year Program
   Improvement
NSSE and FYP Improvement…
    Identify strengths and areas for growth
       Ex. FY students frequently engage in active learning
        (+); but are involved in enriching educational
        experiences at low levels (-)
    Assess effectiveness of FY interventions
       Ex. Low student faculty interaction scores at small
        college prompts institution to establish mentoring
        program, and involve students in undergraduate
        research; 2 years later NSSE results rise on this
        benchmark
NSSE and FYP Improvement…
    Sewanee: University of the South: Concern about FY
     students’ low active and collaborative learning results led
     Sewanee faculty to rejuvenate first-year seminar course,
     increase active pedagogies & collaborate with student affairs to
     create rich co-curricular elements.
    Worcester Polytechnic Institute: NSSE results showed FY
     students less engaged than seniors. Created new FY
     interdisciplinary, inquiry-based seminars; better integration of
     disciplines; engaging introductory courses. Associate Dean
     appointed to Office for the First Year. Assessment plan for the
     FY in development with NSSE indicators as key component.
    UW Green Bay: Used NSSE items in Freshman Seminar Pilot
     Study to test impact of revised seminar.
CIRP and FYP Improvement…
    Several common first-year programs are included on
     the survey
        First-year seminars
        Service learning
        Student-centered pedagogies
        Special Interest Housing
        Tutoring
        Remedial education
        Diversity/multicultural experiences
        Student-faculty research partnerships
    Measures satisfaction with numerous campus
     services and facilities
CIRP and FYP Improvement…
 Using YFCY Data to Improve Programs
    Identify patterns of participation in specific programs
    Measure levels of satisfaction with services/programs
    Conduct sub-group analyses of involvement in programs to
     examine differential involvement
         Ex. First-year seminars and involvement with faculty
    Identify the impact of programs and services on key outcomes of
     the first year individually and in combination
EBI and FYP Improvement…
 First-Year Initiative Assessment - Mission of study is the
     improvement of the first-year course/seminar. Pertinent factors
     include “Course Improved…”
    Study Strategies
    Academic and Cognitive Skills
    Critical Thinking
    Connections with Faculty
    Connections with Peers
    Out-of-Class Engagement
    Knowledge of Campus Policies
    Knowledge of Academic Services
    Managing Time and Priorities
    Knowledge of Wellness
EBI and FYP Improvement…
 ACUHO-I/EBI Resident Assessment
 AFA/EBI Fraternity/Sorority Assessment
 ACUI/EBI College Union Assessment and
  Student Activities Assessment
    All studies contain class standing as a categorical question.
     Institutions can filter results to first-year students to understand the
     impact of their programs
    All assessments’ mission is program improvement by
     understanding students’ perceptions of their program
EBI and FYP Improvement…

 Foundations of Excellence Student and
  Faculty/Staff Assessments
    Both studies measure the institutions’ delivery of the first year of
     college from the first-year students’ perceptions and the
     faculty/staff’s perceptions
    Assessments are integrated into a full self-study of the
     institution’s first year of college overseen by the Policy Center.
EBI and FYP Improvement…

 MAP-Works!
    Study measures first-year students’ expectations of their first-
     year and their perceptions of their environment during that initial
     transition
    Online reports are sent to each student and their advisor, hall
     director, and/or first-year course instructor.
        Students’ reports focus on the top areas of transition (i.e. study time, self-
         management, making friends, residence hall transition, etc.)
        Faculty/staff’s reports focus on areas of risk self-reported by student (i.e.
         no support system, few expected study hours, expected graduation, etc.)
        Institution’s reports focus on areas of low performance by large segments
         of population to support improvement of first-year programs.
Learning Outcomes
Self-Reported Outcomes…

 Can be trusted so long as…
      Questions are clear, with singular meanings and
       interpretations.
      The information requested is known to respondents.
      Respondents believe the questions merit a thoughtful
       response.
      Response options are complete and appropriate.
      Questions avoid risk to the respondent, and do not
       encourage the respondent to respond in socially
       desirable ways.
NSSE and Learning Outcomes…

 Provides student learning process and outcomes
   indicators
    To what extent are FY students developing
     their writing skills?
         % FY writing papers > 5 pgs.
         % FY students who prepare 2 or more drafts

         % FY reporting substantial gains in writing
          effectively
         % FY worked on paper requiring integrating ideas,
          sources
NSSE and Learning Outcomes…
NSSE & Educational and Personal Gains (% “very much” or “quite a bit”)
       Self-Reported Educational and Personal Gains              First-Year
                                                                              Seniors
       from College                                              Students
       Thinking critically and analytically                        81%         87%
       Acquiring a broad general education                         82%         86%
       Working effectively with others                             66%         78%
       Writing clearly and effectively                             72%         77%
       Learning effectively on your own                            70%         77%
       Using computing and information technology                  65%         76%
       Acquiring job or work-related knowledge and skills          57%         72%
       Speaking clearly and effectively                            60%         72%
       Understanding yourself                                      60%         66%
       Analyzing quantitative problems                             55%         65%
       Solving complex real-world problems                         49%         58%
       Understanding people of other racial/ethnic backgrounds     50%         52%
       Voting in local, state, or national elections               24%         23%
CIRP and Learning Outcomes…
 Using TFS & YFCY: Academic & Cognitive Indicators
    Grade point average
       Goes down by ½ point during first year
    Longitudinal change in self-ratings of skills
       Writing ability (+8%)
       Mathematical ability (-3%)
       Academic ability (-5%)
    Self-perceived changes
       General knowledge (21% report much stronger)
       Critical thinking skills (19% report much stronger)
       Analytical/problem-solving skills (17% report much stronger
CIRP and Learning Outcomes…
 Using TFS YFCY: Other Indicators
    Identity development
       +4% in personal rating of self-understanding
    Interpersonal skills
       +13% report their ability to work as part of a team
    Civic engagement
       +8% in goal to keep up to date with political affairs
    Awareness and acceptance of diversity and
     multiculturalism
       +3 in goal to improve understanding of other
        countries/cultures
EBI and Learning Outcomes…
    Academic Assessments are linked to professional learning
     outcomes standards…
            Undergraduate Business Exit Assessment (AACSB Standards)
            Nursing Exit Assessment (AACN Standards)
            Teacher Exit Assessment (NCATE Standards)
            Engineering Exit Assessment (ABET Standards)

    Student Affairs studies are linked to professional learning
     outcomes standards, for instance…
            ACUHO-I/EBI Resident Assessment (ACUHO-I, CAS, and
             NASPA/ACPA Learning Reconsidered)
            AFA/EBI Fraternity/Sorority Assessment (AFA, CAS, NASPA/ACPA
             Learning and Leadership Reconsidered)
            ACUI/EBI College Union Assessment (ACUI and CAS Standards)
            ACUI/EBI Student Activities Assessment (Learning Reconsidered,
             Leadership Reconsidered, CAS Standards)
Using Multiple Data
Sources
Linking Data…
   In-house surveys
   National surveys (a growing list)
       CIRP
            TFS/ YFCY / CSS
     NSSE
     CSEQ / CSXQ
     EBI Benchmarking surveys
     Noel Levitz Student Satisfaction Inventory
     ETS Major Field Tests
     ACT Collegiate Assessment of Academic
      Proficiency
   Institutional data: GPA, financial aid,
    transcripts, retention, certification
    tests, etc.
NSSE, EBI, and Engagement…
 Washington State University study: What is the
 relationship between residence hall environment and
 student engagement? Is there a difference by gender?
 (Robert Tattershall, Director, Housing and Conference Services, Washington State
 University)

      Used   EBI data to characterize those residence hall
       floors with high and low interaction
      Identified correlations between NSSE’s
       engagement measures and the residence hall
       interaction levels
      Found gender differences
NSSE, CIRP, and Retention…


    Disaggregated NSSE results from seniors by those
     who started at the institution as first-year students and
     those who entered as transfer students
    Better understand the transfer student experience.
    Combines with CIRP to develop a fuller portrait of the
     transfer student experience.
CIRP, NSSE, and FYP Improvement at Elon…

     Elon University
         High institutional ethos of assessment
         Building a student database
         Value-added assessment methodology guides studies
     Both CIRP/YFCY and NSSE…
       Lead to additional research
       Are utilized for longitudinal study & benchmarking
       Provide valuable measures of cognitive skills
       Assess student interaction with faculty
       Help to understand overall educational experience
CIRP, NSSE, and FYP Improvement at Elon…

     Key uses of YFCY
         Time allocation
         Benchmark for issues of campus community
         Cognitive and skill changes
         Measuring use and satisfaction with campus programs and
          services
         Facilities evaluation (classrooms, labs, library)
         Social activities and behaviors
     Primarily used by Student Life &, more recently,
      General Education Office
CIRP, NSSE, and FYP Improvement at Elon…

     Key uses of NSSE
         Activities inside the classroom (e.g., speaking up,
          presentations, working with students)
         Activities outside the classroom (e.g., plays, museums,
          events)
         Evaluation of various functions (e.g., administrators,
          academic advising)
         Academic rigor
     Primarily used by Student Life & Academic Affairs
Survey Administration Cycles…
 Some Examples…
    Administer all surveys to all students and merge them together
     with campus data to create an extensive student information
     system
    NSSE every 3 years, alternate CIRP, EBI institutional surveys
    Administer NSSE and EBI two years in a row to establish
     baseline data and assess needs for first-year learning
     communities
    Administer NSSE to a random sample of first-year students
     and YFCY to those remaining
    Administer CIRP TFS and YFCY and/or BCSSE and NSSE for
     program participation patterns and outcomes;
     supplement more detailed programmatic information
     from EBI
Q&A
Contact Us…
     Cooperative Institutional Research Program (CIRP)
     Higher Education Research Institute at UCLA www.heri.ucla.edu




     National Survey of Student Engagement (NSSE)
     Center for Postsecondary Research, Indiana University
     www.nsse.iub.edu



     Educational Benchmarking (EBI)
     EBI Service Center, Springfield, MO
     www.webebi.com

								
To top