Slide 1 - Northeastern Illinois University by ewghwehws

VIEWS: 6 PAGES: 21

									 Assessment of research:
Where do we go from here?


   Linda Rueckert, Northeastern Illinois University
     (http://www.neiu.edu/~lruecker/assess.htm)
         Kathleen Morgan, Wheaton College
Who are the stakeholders in undergraduate research assessment?



    *Institutions/agencies

    *Departments and Programs

    *Faculty

    *Students
Stakeholders in Undergraduate Research: why assess?

 How you ask will depend greatly on what you want to know, and why…


  Institutions

  *retention

  *grants-seeking

  *grant stewardship/accountability
Stakeholders in Undergraduate Research: why assess?
  How you ask will depend greatly on what you want to know, and why…




Departments/Programs

*program improvement
Stakeholders in Undergraduate Research: why assess?
   How you ask will depend greatly on what you want to know, and why…


 Faculty

 *training improvement

 *are they learning what you think you’re teaching?

 *(n=61 CUR attendants)

 Top 3:

           1. critical thinking
           2. communication skills
           3. applied knowledge/disciplinary literacy
Stakeholders in Undergraduate Research: why assess?
   How you ask will depend greatly on what you want to know, and why…


 Students

 *improve grad school prospects

 *improve employability

 *fun

 *develop closer relationships with faculty

 *we’d like to find out more! (see survey on yellow handout)—and you can help!
Stakeholders in Undergraduate Research: how assess?
  A few examples…                             ..can you provide others?

 Institutions

 *Oak Ridge Associated Universities: custom-made survey instruments

 *University of Michigan—structured interviews
 (http:///www.umich.edu/~urop/facres/evalurop.html)

 *University of Delaware Research-based Education:
 (http://www.udel.edu/RAIRE)
Stakeholders in Undergraduate Research: how assess?
   A few examples…                                ..can you provide others?

Departments/Programs

 *Northern Illinois University Psychology Dept.
     NEIU capstone analyses
• Students do independent research, field
  experience, or enhanced class as
  capstone project.
• All are required to write a final APA-style
  paper.
        Capstone scoring rubric
• Content                    • Style
  –   Theoretical basis        –   Flow
  –   Breadth and depth        –   Spelling & grammar
  –   Creativity               –   Clarity
  –   Design
                               –   APA format
  –   Statistics
  –   Conclusions
  –   Relate to intro.
  –   Broader implications
    Capstone scores for content
       and style (old data)
        4
      3.8
      3.6
      3.4
Score 3.2
        3                     Class
      2.8                     Independent
      2.6
      2.4
      2.2
        2
            Content   Style
    Capstone scores on content
       subscales (old data)
     4


    3.5
S
c
o    3
r
e
    2.5


     2
          Theor.basis   Breadth   Creativity   Concl.   Relates   Implications
    Capstone scores for content
       and style (new data)
      3.4
      3.2
       3
Score 2.8
                              Class Exp.
      2.6                     Independent
      2.4
      2.2
       2
            Content   Style
Capstone scores on content subscales
            (new data)

          4



         3.5

     S
     c
     o    3
     r                                                                                          Class Exp.
     e                                                                                          Indep.

         2.5



          2
                                   C


                                                D


                                                        St


                                                                 C
               Th


                        Br




                                                                            R


                                                                                       Im
                                   re


                                                es




                                                                 on


                                                                            el
                                                          at
                          ea




                                                                                        pl
                  e




                                                                                at
                                      a




                                                            s.
                                                  ig
                 or




                                                                     cl




                                                                                          ic
                                       tiv
                             dt




                                                                                   e
                                                    n




                                                                        .
                   .b




                                                                                            .
                                                                                  s
                               h


                                          ity
                      as
                        is
Stakeholders in Undergraduate Research: how assess?
   A few examples…                                ..can you provide others?

  Faculty

 Critical Thinking Assessment Tools:
 *Watson Glaser Critical Thinking Appraisal RANRA

 *California Critical Thinking Skills Test

 *Tim van Gelder’s Critical Thinking Assessment Site (lots of links to various
 tools here): http://www.austhink.org/critical/pages/assessing.html

 Communication Skills Assessment:
 *Liverpool Communication Skills Assessment Scale (LCSAS)

 *Biography of Assessment Alternatives: Oral Communication. Innovative
 Assessment. ERIC document ED 422595

 *Assessment of Basic Oral Communication Skills: A Selected, Annotated
 Bibliography. ERIC document ED 319086
Stakeholders in Undergraduate Research: how assess?
   A few examples…                              ..can you provide others?

  Students
 *Kardash (2000). Self-rating instrument

          --used a 1-5 scale, asked research interns to rate themselves on items like

          “To what extent do you feel you can……

             …understand contemporary concepts in your field?
             …make use of the primary scientific literature in your field?
             …formulate a research hypothesis based on a specific question?”, etc.

 *reflective judgment essays

 *goal setting and ranking

 *our student survey
Sample goal setting document:

                LEARNING FROM YOUR RESEARCH EXPERIENCE*
        Before you begin your research work, take a few minutes and complete the
       statements below. This will help you to begin thinking about your learning
       goals for participating in research. Although at this point, you can only
       speculate about the research experience, the more you think about it now,
       the more you will learn from it.

    A. I am engaging in research because:

    B. My goals for participating in research include:

    C. I think I will be performing the following duties/tasks:

    D. I hope to develop the following skills:

    E. In addition to testing an hypothesis, I think I might learn about:

    F. What skills or perspectives do you think you already have to contribute to
       your research experience?

      Turn your completed handout in to your supervising faculty member.
Sample guided reflective essay:

                            Principle Investigator (PI) Self-Evaluation
  Student Name:__________________________________ Semester_______ Yr_________
  Title of research project you worked on:______________________________________________
  ______________________________________________________________________________
  Estimated # of hours spent on this project during the entire semester____________________

  DIRECTIONS: The academic credit that you will receive upon satisfactory completion of your role as a PI
      reflects the college’s understanding that learning here also takes place outside the conventional
      classroom. To that end, it is useful to think about just what it is that you have learned. Using the
      guidelines below, write a 1-3 page reflection essay on your experience. You may find it useful to refer to
      your research journal as you write. Attach your essay to this cover sheet and return it to Dr. Morgan.

  I.     Briefly describe the research project you developed or are developing and your primary responsibilities
         in that development. What did you do (or what do you plan to do)? What have you found out?

  II.    Reflect (write about what you have learned):

         *Some of the skills that you have developed and how you learned them
         *What you have learned from your supervising professor, your research assistants (if any),
          or others who might have been involved in the project’s development
         *What you expected to do and what you actually did
         *The relationship, if any, between this out-of-the-classroom learning experience and your
          academic work

  III.   Have you changed as a result of this experience? If so, in what ways?

  IV. What grade do you think you should receive for your efforts? Why?
   A Few Last Words on Assessing Undergraduate Research Experiences

 If you are just beginning an assessment program—whether of your classroom
     or your research program—here are a few suggestions to keep in mind
     (courtesy of Tom Angelo again):

1. Know what it is you want to know, BEFORE you begin planning your
   assessment. Do you want to know if student researchers go on to graduate
   schools more often than students who do not participate in research? Do you
   want to know if your student researchers are sufficiently trained in execution of a
   protocol to be able to manage it on their own? Do you want to know if student
   researchers' self-confidence increases after their experiences? Knowing what
   outcome you want to assess is essential to designing and implementing the right
   assessment.


2. Keep it simple and short. Don't make assessment into a chore or burden for
   either you or your students. Collect only as much data are you can reasonably
   use and respond to.
  A Few Last Words on Assessing Undergraduate Research Experiences


3. Keep it focused. (This harkens back to knowing what you want to know!) While
   there are many interesting questions you might ask, focus on those most likely to
   improve the achievement of your own teaching, research, or program goals.
   Make sure that your students understand the purpose and value of your
   assessment. This too is an important component of assessment that we often
   forget.

4. Don't ask if you don't want to know, or won't be able to respond.

5. Make sure your aim is true. ALWAYS try out a new assessment technique on a
colleague before trying it out on your students.


6. Close the feedback loop. Make sure to let students or other stakeholders know
the outcomes of the assessment and what changes (if any) those outcomes
suggest for you and them.


 7. Let us know about your assessment program!
Send your assessment ideas to:

L-Rueckert@neiu.edu
 Or

 kmorgan@wheatonma.edu



                             Thanks!

								
To top