A Technical Guideline for A Plan of Outcomes Assessment by Nowandforever


									             A Technical Guideline for A Plan of Outcomes Assessment

                                        Brian Hu
                            Director of Institutional Research

Purposes of Outcomes Assessment
• Implementation of a comprehensive program of institutional effectiveness established
   to meet regional or other accreditation criteria and to increase accountability to the
   public and clients (students, parents, alumni and donors).
• Evaluation of the Institutional Mission Statement.
• Assessment for strategic planning and budgeting to implement the Mission.
• Response to state- mandated assessment programs for accountability to the public.
• Assessment of student academic achievement for the purpose of regional or
   professional accreditation.
• Assessment of student academic and social supporting services for improvement and
   institutional effectiveness
• Pure/intrinsic student outcomes assessment for identifying strengths and weaknesses
   in programs and theoretical research.
• Moreover, according to WASC, assessment of effectiveness is an institutional
   concern, which sets up standards on planning with collecting data as the means, and
   improving institutional planning and functioning as the end.

Entities of Outcomes Assessment
• Chief executive officers – Vice Presidents, Deans and Department Chairs/Directors at
   different levels.
• The individual or a group charged by the Chief Executive Officer with the
   responsibility for planning and implementation —The Outcome Assessment
• The academic and nonacademic departments/units where planning and
   implementation will actually take place -- faculty and staff involved.

    Under the leadership of the Chief Executive Officer and the Outcome Assessment
Committee, academic and nonacademic departments/units are the basis for outcome
assessment, where planning, processing and implementations are actually to occur. The
purpose of departmental implementation is to demonstrate at that level a set of
assessment criteria and procedures, which the campus has relative freedom to develop,
rather than to meet externally determined student performance standards. The
department/unit will ultimately need to demonstrate and record its use of assessment data
to improve student learning/or its services and operations.

Process of Outcomes Assessment
       The administration of the process of outcomes assessment should be a cyclic one
from top-down to bottom- up and then top-down and bottom-up again to guarantee the
realism and validity of the Mission, and the feasibility and reliability of the Expanded

Statement of Institutional Purposes. The actual outcomes assessment should occur at the
departmental/unit level. The followings are the major steps in outcomes assessment.

•   Establish a group or committee/subcommittee for outcomes assessment and
    institutional effectiveness by chief executive officers at different levels.
•   Make a plan and set up a timeline for outcomes assessment.
•   Review the University Mission by the Board of Trustees and The Chief Executive
    Officer and revise it, if necessary.
•   Review and revise the divisional/departmental Expanded Statement of Institutional
    Purposes/intended goals and objectives to match the University Mission.
•   Work out no more than five most important expected outcomes according to
    divisional/departmental intended goals and objectives by faculty and staff under the
    leadership of the divisional/departmental outcomes assessment committee.
•   The University Outcomes Assessment Committee coordinates all
    divisional/departmental subcommittees to provide consultation and monitors the
    process of outcome assessment.
•   Establish the strategies, criteria, measures and means to evaluate expected outcomes
    according to available resources.
•   Implement the evaluation of expected outcomes.
•   Compare the results of outcomes assessment with expected outcomes, and analyze the
    data for reports and record keeping.
•   Write reports about the analysis of assessment and use of results and make necessary
    adjustment for improvement of programs or services.
•   File reports and submit reports to the University Outcome Committee for review,
    feedback and suggestions for improvement.

Major Areas of Outcomes Assessment
    The following list is only for reference. It is not used to suggest an exhaustive data
analysis for all areas. The Department/Unit should choose those areas that may be
closely related to its expected outcomes.

1. Student Enrollment Assessment
• Demographics – characteristics such as gender, ethnicity, age, GPA, high school
   GPA, high school rank, SAT/ACT scores, trend and change
• Socioeconomic Status: household income, parental education, and parental
   occupation, trend and change
• Transfer enrollment: origins, entering GPA, cumulative GPA, trend and change
• Enrollment – by gender, ethnicity, major and college, degree GPA, attempted credit
   hours, earned credit hours, trend and change
• Persistence/retention – rate by cohort, gender, ethnicity/gender, college, trend and
• Graduation – rate by cohort, gender, ethnicity/gender, major, college, trend and
   changes, transfer from entering major to degree major, trend and change
• Financial Aid – encouragement from friends and family, financial support through
   financial aid programs, total/average amount received, tuition discounting pricing

2. Academic Integration
• Classroom Experiences – study hours spent in and out of classroom, library hours,
   computer hours
• Faculty contact/relations/concerns
• Academic conscientiousness/study habits and learning initiatives and motivation
• Academic performance (GPA and other performance tests)
• Perception of self abilities
• Satisfaction with academic program to meet students’ needs

3.   Social Integration
•    Peer relations
•    Social involvement/extracurricular activities
•    Employment opportunities (on and off campus)
•    Perceptions of prejudice/diversity/campus climate
•    Leadership involvement
•    Satisfaction with the help/contact from faculty and staff
•    Satisfaction with the community as a whole
•    Satisfaction with the effectiveness of administrative offices
•    Satisfaction with advice/counseling in academic programs
•    Satisfaction with advice/counseling in career development
•    Satisfaction with campus housing environment and staff
•    Commitment to campus spiritual/religious life
•    Function and assistance of financial aid in student life

4.   Institutional Goal Commitment
•    Financial status/endowment/budgeting procedure/controlling procedure
•    Multicultural/intercultural environment
•    Efficiency and effectiveness of academic support services (Library, Learning
     Resource Center, Academic Advisement, Academic Computing, Admissions,
     Graduate Admissions, Financial Aid, Grants and Faculty/Staff Development,
     Institutional Research)
•    Administrative Support Services (Information Services, Finance and Administration,
     Facilities, Public Relations, Alumni Relations, Security and Safety Services, Food
     and Catering Services, Mail Services, Plants & Grounds)
•    Student Services (Housing & Food Services, Student Life, Athletics, Career
     Development, and Clinic, etc.)

Major Indicators in Outcomes Assessment
• Persistence rate
• Graduation rate
• Intellectual growth
   - General skills (reading, writing, and math)
   - Knowledge (general education, major field, inter-discipline)

    -   Higher order cognitive skills (critical thinking, problem solving, communication
        and reasoning)
•   Personal Growth
    - Attitudes/values/beliefs (responsibility & self-control, openness and tolerance,
        goal clarity)
    - Knowledge building skills (library skills, time management, study skills, note-
        taking and outlining, independent learning, computer application skills)
    - Behavior (attendance, interpersonal skills, additional degrees, transfer
        performance, employment)
    - Religious belief/spiritual life
•   Aspirations (life- long learning, graduate studies/degrees, social activities)
•   Satisfaction with education experiences (climate, environment, supporting services,
    faculty involvement, career development, class activities, teaching methodology and
    student activities)
•   Alumni outcomes (employment, career development, career achievement, activity
    involvement, gift giving)

General Means of Outcomes Assessment

Purpose                              Suggested Strategies and Methods
Meeting Standards                    Department Comprehensive Exam
                                     Standard Competency Examinations
                                     Senior Thesis/Research Project
                                     Student Performance/Exhibit
                                     Practice Teaching
                                     Certification/license Exams
                                     External Examiners/Readers
Comparing to Others                  National Comprehensive Exams (ACT, ETS)
                                     National Surveys (CIRP, CSS, Faculty Survey)
                                     Program Review by External Experts
                                     Peer Assessment
                                     Comparative School and Placement Informant
Measuring Goal Attainment            Senior Thesis/Research Project
                                     Student Portfolio
                                     Student Self- Assessment
                                     Focus Groups
                                     Alumni Surveys
                                     Exhibits and Performances
Developing Talent & Program          All of the Above, especially:
Improvement                          Senior Essay and Interview
                                     Focus Groups
                                     Course Embedded Assessment
                                     Learning Portfolio
                                     Alumni Surveys
                                     External Program Reviews.
Measuring Efficiency and             Cost-benefit/effect analysis

Effectiveness                      Ratio Analysis
                                   Work Volume/Load Records
                                   Trend and Change Analysis
                                   Employee and Client Satisfaction Surveys
                                   Program Performance Evaluation
                                   Focus Groups
                                   Retention/Turnover rate

Available Data for Outcomes Assessment:

•   CIRP Freshmen Survey – 1989-1999
•   College Student Survey – 1998 (for senior students only)
•   Statistical Handbook – 1993-1999
•   Baccalaureate Origin Studies for Doctoral Recipients – 1998
•   Retention and Graduation Rates- 1988-present
•   LMU Intercultural Campus Survey - 1997 and 2000
•   IPEDS reports: Completion, Faculty Salary, Staff, Graduation, Finance, Library,
•   Some of HEDS’ comparative reports with the group of the Higher Education Data
    Sharing System (enrollment, faculty, financial aid, graduation rate, etc.)
•   CRSRDE retention reports by the Consortium for Student Retention Data Exchange
•   US News reports
•   Other


To top