Product Institutional Formulation Guide

Document Sample
Product Institutional Formulation Guide Powered By Docstoc
					   Rush University
   Assessment Plan
      Developed by Rose Suhayada, Ph.D., RN

Director, University Assessment and Student Learning

                  Rush University

                   February 2008
                  Rush University Assessment Plan
The assessment plan of Rush University is based on an adaptation of the widely used
Stufflebeam‟s CIPP (Context, Input, Process, and Product) model, which is a comprehensive
framework for systematically evaluating program effectiveness (Stufflebeam, 2003). The CIPP
model fits well with the Higher Learning Commission‟s commitment to continuous quality
improvement. Whereas some evaluation plans focus primarily on program outcomes, the CIPP
model includes input, process, and context evaluation to inform change at all levels of

Figure 1 is an adaptation of Stufflebeam‟s CIPP model as it applies to the Rush University
Assessment Plan (Stufflebeam, 2003, pg. 6). The inner wheel includes the core values that guide
decisions of Rush faculty. The middle wheel reflects assessment foci. The outer wheel indicates
the type of evaluation being conducted.

Each assessment focus in the middle wheel shares a reciprocal relationship with its evaluation
focus in the outer wheel; and all evaluation foci in the outer wheel share reciprocal relationships
with each other. For example, assessment of curriculum implementation and evaluation
procedures might reveal that faculty have a poor understanding of direct and indirect measures of
student learning. These findings raise questions about process effectiveness. Ineffective process
might be attributed to inadequate input (faculty development) and could explain a substandard
product (student performance). In this scenario, assessment of the process component informs
decisions about input (faculty development on measures of learning), that results in changed
process (application of credible learning measures that guide teaching methods), that impacts
product (improved student performance).

Not all components of the CIPP model need to be evaluated at the same time. For example,
context components are generally more stable and may need less frequent evaluation. Input,
process, and product evaluation, on other hand, are more dynamic and may need more frequent
assessment to inform decisions about program quality and effectiveness.

Two tools that will be used to monitor ongoing improvement are the Rush University Action
Plan and the Program Assessment Report. The Action Plan is requested when problems or needs
are identified resulting from the assessment process or when faculty suggest major change
initiatives that affect the educational programs. The Program Assessment Report will be
completed by program directors or faculty to identify program outcomes, direct and indirect
measures of learning, and program or course changes that were made based on assessment data.
In addition, each college will be asked to submit a Program Review and Outcomes Assessment
to include but not limited to: certification and licensure rates; graduation and attrition rates;
scholarly and professional accomplishments of students, faculty, and alumni; satisfaction rates of
students and employers; and job placement rates of graduates.

Rush University                        Assessment Plan                                            2
 CIPP Model for Program Evaluation
Adapted from Stufflebeam, D. L.( 2003). Professional Standards and Principles for Evaluations. In T. Kellaghan and
      D. L. Stufflebeam (Eds.), International Handbook of Educational Evaluation. Norwell, MA.: Kluwer.

Rush University                              Assessment Plan                                                     3
                                       RUSH UNIVERSITY ASSESSMENT PLAN
                                                  ASSESSMENT CRITERIA                                 DATA SOURCES                 ASSIGNED

                                  - Mission documents of each college are linked to the             - Mission statements:    - College faculty and
                                    mission of the university, accurate, and accessible.              RUMC and RU              leadership groups
                                  - Faculty, students, and staff of the university indicate           websites: RU catalog   - University Assessment and
                                    familiarity and understanding of the mission, vision, values,   - Minutes: University      Student Learning
                                    and goals of their college and of the university.                 and faculty
                                  - Assessment plans for each of the colleges are linked to their     councils/senate and
                                    mission, goals, and objectives for student learning and           committees

                                    academic achievement.                                           - Surveys: faculty and
                                  - University promotional materials accurately reflect program       students
                                    offerings, outcomes, accreditation/approval status, academic    - College assessment
                                    calendar, admission and progression policies, degree              plans
                                    requirements, tuition and fees.
                                  - Faculty students and staff are informed of and given            - Rules for Governance   - College faculty, leadership

                                    opportunity to participate in and influence decisions and       - Minutes: University      groups and committees
                                    policies that impact the university and its educational           and faculty            - University Assessment and
                                    programs                                                          councils/senate and      Student Learning
                                                                                                    - Surveys: faculty and

Rush University                                                Assessment Plan                                                                               4
                                               ASSESSMENT CRITERIA                                  DATA SOURCES                    ASSIGNED
                               - Strategic initiatives at the college level align with the        - University and college    - College faculty, faculty
                                 university‟s mission, are approved through designated              level strategic plans       leadership groups ,and
                                 channels, and give consideration to resource demands of the        and budgeting               committees
                                 university.                                                        documents
                               - Evaluation of strategic initiatives is reflected in university   - Minutes: University
                                 and college documents.                                             and faculty
                               - Faculty are given an opportunity to participate in                 councils/senate and
                                 programmatic and resource planning, whether for on-site or         committees; Provost
                                 distance offerings.                                                council.

                               - Quality, availability, functionality, and adequacy of learning   - Annual capacity           - Staff: library, labs
                                 labs, equipment, educational technologies, practice sites and       analysis of university   - Faculty: clinical sites

                                 academic support services for both on-site and distance             resources and services   - Rush University Student
                                 offerings are assessed.                                             (library, METC,            Services: student

                                                                                                     RUSL, student              counseling, academic
                                                                                                     counseling and             support, other student
                                                                                                     academic support)          services
                                                                                                  - Surveys: faculty and
                               - Faculty, including short term faculty, receive an orientation    - University Council:       - Program and college
                                 to the university and its educational programs.                     Faculty orientation        administrators

                               - Faculty involved in the clinical and/or didactic teaching of       program                   - University and
                                 students, whether on-site or through electronic format, meet     - Faculty vitae               Program/College faculty
             FACULTY and

                                 the degree, credential and/or licensing requirements             - Minutes: faculty            development committees
                                 appropriate to the discipline in which they teach.                 development               - University Assessment and
                               - Faculty development programs are responsive to faculty             committees                  Student Learning
                                 needs and designed to facilitate improved teaching and           - Surveys: faculty
                                 evaluation strategies with diverse student bodies and varied
                                 learning environments, including on-line and clinical
                               - Faculty development programs are responsive to the
                                 scholarship and research development needs of faculty

Rush University                                            Assessment Plan                                                                                  5
                                          ASSESSMENT CRITERIA                               DATA SOURCES                  ASSIGNED
                           - Each degree program has clearly stated learning outcomes     - Curriculum review       - Program/course
                             to guide assessment of student learning.                       policies and procedures faculty/directors
                           - Faculty use direct and indirect measures that align with     - Program Annual          - University Assessment and
                             course and program objectives to assess student learning       Assessment Reports        Student Learning
          CURRICULUM and

                             and teaching effectiveness.

                           - Course materials, including course syllabi, clearly convey
                             student expectations for learning, evaluation criteria,
                             course content, and learning activities that align with
                             course objectives.

                           - Each college maintains a system of quality assurance
                             which includes regular review of the curriculum, course
                             offerings, teaching effectiveness, and student learning.
                           - Each program provides evidence of how assessment data
                             are used for program improvement and enhanced student
                           - The university‟s outreach programs and co-curricular         - # and description of   - Directors of continuing

                             activities support the education of its students.              community outreach       education programs
                                                                                            programs in which      - Program /course faculty and
                                                                                            students, staff, and     directors
                                                                                            faculty participate

Rush University                                       Assessment Plan                                                                              6
                                                      ASSESSMENT CRITERIA                                 DATA SOURCES                   ASSIGNED
                                      Colleges / programs of the university meet or exceed their        - Survey: faculty,      - College / program
                                      benchmarks relative to:                                             students, alumni        administrators
                                      - student and faculty diversity                                   - Reports: College      - University Assessment and
                                      - enrollment, graduation, and attrition rates                       Review and Outcomes     Student Learning
                                                                                                          Assessment            - Office of the Registrar

                                      - certification and licensure rates of graduates
                                      - job placement rates
                                      - faculty, student, and alumni scholarship, research,
                                        professional accomplishments and community service
                                      - satisfaction rates of faculty, students, staff, and employers
                                        of Rush graduates

Rush University                                                     Assessment Plan                                                                           7
Rush University   Assessment Plan   8
This form is to be completed for each proposed major educational or support program enrichment,
enhancement, addition, retrenchment/curtailment or deletion. Major curricular changes, new degrees,
discontinuance of a program, etc., should generate submissions of this form. Changes within individual classes
are not expected to be documented with this form, but the process may be useful to follow within a department
for such actions.
1     Indicate the “Problem/basis for change.” If the problem was identified through a review of data, please
      provide the data, a summary of the data, or indicate where the data can be reviewed. Check all
      constituencies which you expect to be affected by this problem/proposed change.
2     Indicate the “Proposed change/initiative.” Describe this as clearly and briefly as possible. Note the
      departments/units that will be to be involved in addressing this proposal.
3     Note the initiator/source for the proposed change (e.g., name and title, name of college or university
      committee, or department/unit).
3a    Note the date that the proposal process was initiated (e.g., the date submitted to the Office of the Vice-
When items 1-3 are complete, submit the form to the Office of the Vice-Provost in hard copy or via
4     The Vice-Provost or designee will make an initial assignment of responsibility for this action plan. This
      may be to a dean or to the person that initiated the project.
4a    The Vice-provost (or designee) will enter the date that initial assignment was made.
5     This individual assigned responsibility will need to assume the role of team leader for this plan or
      designate someone else to serve in that capacity. Identify the team leader/contact person here. Indicate
      other members of the action plan team. Each team should have two or more members (faculty,
      administration, and/or staff) to provide necessary resources/research/data/objective critiques/input for
      formulation and feasibility of the plan.
5a    Note the date work on the proposal started.
6     Briefly summarize action(s) taken to address implementation of the change/initiative. Notes
      resources/funding which will be needed to implant. (Note: This form is not a funding request).
6a Briefly summarize barriers/roadblocks or issues for implementation of the change/initiative.
7     Identify the target date for accomplishing this change.
8     Describe the anticipated project outcome(s) your team expects from implementation of these actions.
      Submit the form to University Assessment in Student Learning (UASL) in hard copy or as an email
9     Evaluation strategies should be identified by your team. If this is not possible, document what
      observable results would be useful to assess if the action plan addressed the problem or basis for change.
      UASL staff members are available for consultation on evaluation strategies. Please check if your team
      needs consultation on this step (this step may be left blank pending your consultation with UASL).
When items 1-9 are complete, submit the form to the UASL in hard copy or as an email attachment.
10    What is the evaluation period for this project (include pretest/baseline through follow-up)?
11    Evaluation findings: Provide a brief summary of what the evaluation showed and if there is need for
      additional evaluation to further assess the outcome(s) of the process. How has this affected Rush?
11a Date of findings: Note the date(s) that the findings were collected.
12     The action plan team should provide a brief statement regarding decision/action/recommendation(s)
       regarding the change. If a report was issued, it may be referenced here and attached. Include next steps
       and/or recommendations for further/other changes, additional pilot projects, new action plans, self-
       studies, etc.
 12a Date of decision/action: Note the date the team reached its decisions, actions, and/or recommendations
     for this project. This should also be the date the materials are submitted to the UASL.
When all items are complete, submit the form to the UASL in hard copy or via email with all
supporting documents.

Please remember that University Assessment and Student Learning (UASL) staff members are
available to help clarify aspects of the Action Plan process.

Rush University                                  Assessment Plan                                                  10
                                          The Action Plan Process
Initial structure:

A Rush University Action plan can be initiated in two ways:

a) top down – an individual or group at the leadership level (e.g., the Provost‟s Council, HLC Steering Committee,
Vice Provost) identifies a problem and proposes a change or initiative. The Office of the Vice Provost will then
initially assign responsibility for developing an action plan team and following this project.

b) bottom up – an administrator or college/university committee may identify a problem or initiative. Items 1
through 3a of the action plan form are to be completed and the form is then sent to the Office of the Vice Provost for
review. The Vice Provost may then consult with the Provost‟s Council or the HLC Steering Committee or specific
administrators. If the proposal is accepted as an action plan, an initial assignment is made to an individual who will
be responsible for developing the team and following the project.


University Assessment and Student Learning (UASL) has been assigned the charge of tracking these action plans.
This will allow their staff to collect evidence of how Rush follows our change processes for future HLC self-studies,
to provide consultation on evaluation strategies and interpreting evaluation findings, and to identify action plans
which appear to have stalled out in implementation.

A spreadsheet will be created to log in action plans, including the dates in the timeline for each document. This will
be kept on a shared drive which can also be accessed by the Vice-Provost.

The action plan team is expected to complete their initial assessment of how this problem will be tackled or how
change implemented. Then, the team will submit the action plan sheet to the UASL after items 1-9 have been

The UASL staff will review the document for completeness and will provide consultation as requested from the
action plan team.

The team will continue working on their task and when they have completed the project and have a set of
decision/actions/or recommendations to offer, they complete the remaining items on the form and resubmit the
document to the UASL. The UASL staff will review the document for completeness, look over the evaluation
findings to determine if it appears the evaluation strategy was successfully implemented and that the analysis
presented is appropriate for the kinds of data collected.

After the document has been accepted by the UASL, it is sent to the Vice-Provost for review and dissemination to
the appropriate parties.

The UASL staff will also periodically review the spreadsheet and alert the Vice-Provost to any action plans which
have gone past their target date for accomplishing the change and/or are beyond their planned evaluation period.
They will also communicate with the designated contact person/team leader for these action plans to seek an update
and see if the team requires consultation from the UASL to help them proceed.

Evolution of the process:

One of the major initial tasks of the UASL will be to develop a university assessment plan (a requirement Rush must
meet for HLC accreditation). The action plan process will then be integrated as one component of the university
assessment plan. Templates of action plans for optional internal use by colleges/programs will also be provided.

University Council will be asked establish a subcommittee which would review these action plans and final reports.

If the work of this subcommittee is successful, efforts will be made to institutionalize this group as an elected
committee of faculty under the Rules of Governance

Rush University                                     Assessment Plan                                                  11
                                      Rush University
                  Office of University Assessment and Student Learning

Academic Assessment
        Academic assessment is intended to help programs develop and/or improve the process of
measuring student learning. The Academic Assessment Plan includes a description of a
program‟s goals and intended student learning outcomes and establishes indicators or measures
by which a department can measure its success based on student learning outcomes.
        While assessment is an overall institutional concern, as reflected in the various standards
for accreditation, its primary focus is the teaching-learning experience. To the greatest extent
possible, therefore, each program should describe the explicit achievements expected of its
students and adopt reliable procedures for assessing those achievements. Ultimately, assessment
and accreditation share the common goal of enabling the University to reach its fullest academic
potential by providing the highest quality education possible.
        A commitment to continuous improvement through ongoing assessment and
communication of educational objectives and achievements to our various constituencies
contributes to the credibility of the university and directly addresses accountability questions.

Questions and Answers about Assessment at Rush University.

What are we assessing?
Academic assessment involves the following:
   1. Identifying the desired student learning goals or outcomes for a curricular sequence or
      major program;
   2. Measuring quantitatively and qualitatively the degree to which these goals have been
   3. Using the measurement results to improve the academic program.

Why do we need assessment?
  1. Improve the quality of programs
  2. Engage in introspective evaluation
  3. Demonstrate accountability
  4. Maintain accreditation

What do the accrediting organizations require of us?
  1. How are your stated student learning outcomes linked to the mission of the University
      and specific College?
  2. What evidence do you have that students achieve your stated learning outcomes?
  3. In what ways do you analyze and use evidence of student learning?
  4. How do you ensure shared responsibility for student learning and assessment of student
  5. How do you evaluate and improve the effectiveness of your efforts to assess and improve
      student learning?

How do we formulate student learning goals?
It could be helpful to break down student learning goals into three areas:

Rush University                            Assessment Plan                                       12
    1. Cognitive knowledge of content area
    2. Skills and Competencies
    3. Student attitudes (intellectual curiosity, ethics, etc.)

Where are particular learning objectives addressed?
  1. Courses
  2. Programs
  3. Services
  4. Internships, clerkships
  5. Community service projects
  6. Independent study

What kinds of evidence can be used to demonstrate student learning?
Assessment measures can be categorized as direct or indirect indicators of student learning. It is
recommended that at least half of the measures should provide direct evidence of student
Direct evidence: students show their mastery of learning goals through the demonstration of
knowledge, skills, and quality of their work (e.g., creativity, analysis, synthesis, or objectivity).
        1. Scores and pass rates of licensure/certificate/standardized exams
        2. Capstone or synthesis experiences (research projects, presentations, performances,
        3. Ratings of skills or performance provided by internship/clinical supervisors
        4. Portfolio assessments
        5. Comprehensive assignments or exams
        6. Simulations
        7. Embedded in classroom assessment (in-class writing sample, in-class analysis of a
           problem, in-class collaborative problem solving project)
        8. Instructor designed pre-test posttests
        9. Video-audiotape evaluation

Indirect evidence: generally perception or comparison data that allows you to make inferences
about learning, but does not directly demonstrate actual learning.
    1. Student self-assessments (logs and journals)
    2. Surveys (students, alumni, employer, course evaluations)
    3. Exit interviews
    4. Course grades
    5. Placement data
    6. Comparison with peer institutions

Rush University                              Assessment Plan                                        13
                                        Rush University
                                  Program Assessment Report
    Date: ____________
    College: ___ RMC ___ CON ___ CHS ___ GC
    Program/Division: __________________________
    Name of person submitting report, if not the course director: ______________________

  1. List the terminal learning outcomes/objectives for your program in the far left column.
  2. Under „Assessment Methods‟ list all types of learning assessments that you use to evaluate
     student learning. Include both direct and indirect measures, e.g., capstone/synthesis project,
     comprehensive exam, standardized test, clinical/laboratory observation, written
     assignments, certification/licensure results, portfolio review, surveys, etc.
  3. For each learning objective, place an X in the box that corresponds to the assessment
     method(s) used to evaluate learning.
  4. Email the completed form to Where possible, attach syllabi.

                                                              Assessment Method

Program                              Learning

Rush University                            Assessment Plan                                       14
    For this past academic year:

    1. Describe substantive changes, if any, that you have made to either your courses or
       program (e.g., changed teaching and/or evaluation methodologies, learning experiences,
       sequencing of courses, addition of courses, etc.)

    2. If applicable, what was the basis for the change, i.e., what kind of assessment data
       (student or faculty feedback, performance on standardized tests, course exams, etc.)

    3. Have you evaluated the results of your change? If yes, describe the results. If no, how and
       when do you plan to evaluate the effect of your change?

Rush University                           Assessment Plan                                       15

Shared By:
Description: Product Institutional Formulation Guide document sample