REPORT OF THE WASC VISITING TEAM

Document Sample
REPORT OF THE WASC VISITING TEAM Powered By Docstoc
					              REPORT OF THE WASC VISITING TEAM

             CAPACITY AND PREPARATORY REVIEW

                         To PITZER COLLEGE



                           October 29-31, 2008



              In Partial Fulfillment of the Requirements for

                      Reaffirmation of Accreditation



                               Team Roster

                       Elaine Tuttle Hansen, Chair

                    Rita Patel Thakur, Assistant Chair

                           Andrew Grosovsky

                               Lori Varlotta




The evaluation team in conducting its review was able to evaluate the institution
under the WASC Commission Standards and the Core Commitment for
Institutional Capacity and therefore submits this Report to the Accrediting
Commission for Senior Colleges and Universities of the Western Association of
Schools and Colleges for action and to the institution for consideration.
Pitzer College Capacity and Preparatory Review Team Report                                                                              2




                                              TABLE OF CONTENTS



      SECTION I. OVERVIEW AND CONTEXT.................................................................3

              A. Description of the Institution and Visit .............................................................3

              B. The Institution’s Capacity and Preparatory Report: ..........................................6

                        •     Alignment with the Proposal

                        •     Quality and Rigor of the Review and Report

              C. Response to Previous Commission Issues ........................................................8

      SECTION II. EVALUATION OF INSTITUTIONAL CAPACITY UNDER THE

              STANDARDS ......................................................................................................10

              Theme One: Linking the Academic with Residential Life ...................................10

              Theme Two: Comprehensive Reviews of Academic Programs ...........................17

              Theme Three: Connecting the Global and the Local ............................................26

      SECTION III. FINDINGS AND RECOMMENDATIONS .......................................31

              Commendations .....................................................................................................31

              Recommendations..................................................................................................33

      SECTION IV. PREPARATION FOR EDUCATIONAL EFFECTIVENESS

              REVIEW...............................................................................................................34
Pitzer College Capacity and Preparatory Review Team Report                                         3


      SECTION I – OVERVIEW AND CONTEXT



      A. Description of Institution and Visit



      Pitzer College is a private, coeducational liberal arts college offering the BA degree in

      more than 40 major fields. With a student body of about 1000 undergraduates, the

      college emphasizes interdisciplinarity, intercultural understanding, and social

      responsibility. There are few requirements; students are expected to work with an

      advisor to plan an individualized course of study that meets the College’s six educational

      objectives. Founded in 1963, Pitzer is one of The Claremont Colleges, a consortium of

      five undergraduate and two graduate institutions. The close physical proximity of all

      seven institutions creates a rich and broad educational environment, and the institutions

      have taken advantage of many opportunities for both shared facilities (such as the central

      library and medical center) and joint programs in both academic and co-curricular areas.



      The visiting team conducted its evaluation based on prior study of extensive written

      materials as well as on-campus interviews, meetings, and further review of documents

      between October 28 and October 31, 2008. The team is grateful to the Pitzer faculty,

      staff members, students, and trustees for all the hard work that went into providing the

      team with a remarkable learning experience. The team was met by individuals and

      groups with attentiveness, openness, helpfulness and enthusiasm.
Pitzer College Capacity and Preparatory Review Team Report                                           4


      The aspiring, purposeful, and engaged culture of Pitzer was evident in every aspect of the

      team’s visit. This is an educational community that is justly proud of its special character

      and accomplishment, and one that has stayed deeply and pervasively true to its founding

      mission. The faculty is devoted to producing engaged and socially responsible citizens

      through excellent teaching and scholarly work. The students are enthusiastic and

      committed, loyal to the institution and deeply appreciative of their close working

      relationships with faculty and staff members. The college is blessed with strong leaders

      and staff members at every level who strive to sustain and enhance this special place, and

      the team was impressed by the talent, achievement and vision of the people it met.



      Pitzer faculty members are dedicated to their students and to the College as whole, and

      the distinctive structural organization of the faculty and the curriculum into decentralized

      “field groups” as opposed to traditional departments reflects the innovative,

      interdisciplinary and individualized approach to learning throughout the College. The

      Admissions Office has been remarkably successful in attracting and selecting

      increasingly strong applicants in recent years. A strategic planning process begun in

      2001 has worked to align the physical elements of the campus, especially the residence

      halls, more fully with the academic experience. Justifiably proud of its commitment to

      intercultural understanding and social responsibility, Pitzer has developed its own study

      abroad programs, and this year, over 65% of Pitzer students studying abroad will be

      enrolled in these far-flung programs, which combine classroom study with engaged

      learning in the local community. Study abroad has also been expanded and enriched by

      an increase in the number of exchange programs begun in 2004, so that currently 72% of
Pitzer College Capacity and Preparatory Review Team Report                                         5


      the graduating class spends time abroad. Closer to home, Pitzer has robust programs for

      both community-based learning and volunteer work in the local communities, especially

      Ontario, CA. External grant support for many of Pitzer’s innovative programs testifies to

      their innovation and achievement. Governance at Pitzer is designed to be highly

      inclusive, with students represented on all the standing committees of the College. Major

      policy recommendations coming out of these committees are taken to College Council, a

      legislative body consisting of the faculty, staff representatives, and 16 students.

      Members of the Board of Trustees with whom the team met, finally, were passionate

      about the mission of Pitzer and committed to strong fiduciary oversight and philanthropic

      support.



      Pitzer College first received accreditation by the Western Association of Schools and

      Colleges in 1965 and was most recently reaccredited in 1998. The commission noted the

      “exemplary” infusion of Pitzer’s mission into the institutional programs and culture and

      commended the integration of the College’s distinctive educational objectives. Many of

      the visiting team’s suggestions for further attention focused on the need for more

      assessment of student learning, systematic institutional research, and the development of

      a more robust culture of evidence. While Pitzer was guided by these recommendations in

      preparing for the current reaccreditation process, the visiting team observed that many of

      the responses (such as the hiring of a full-time Director of Institutional Research and

      Assessment and the development of a process for regular external academic program

      reviews) are still at early stages and need continued attention.
Pitzer College Capacity and Preparatory Review Team Report                                          6


      B. The Capacity and Preparatory Review Report: Alignment with the Proposal and

      Quality and Rigor of the Review and Report



      Pitzer’s CPR report is consistent with its proposal, presented and approved in the summer

      of 2006, to focus on three themes chosen to explore the goal of designing educational

      experiences that reflect the College’s educational objectives. The report presents details

      about the institutional context, the College’s approach for the Capacity and Preparatory

      Review, and the process of preparing the review. It provides descriptions, objective and

      goals, indicators, evidence, conclusions, and future goals for each of the three themes:

      1.     Linking the Academic with Residential Life,

      2.     Comprehensive Reviews of Academic Programs, and

      3.     Connecting the Global and the Local



      The CPR report is well organized, clearly written and presented, and based on the team’s

      study of documents and interviews, it accurately portrays the condition of the institution.



      Three Thematic Subcommittees worked with the Office of Institutional Research, the

      Dean of Faculty, and the Academic Planning Committee, meeting regularly in the 2006-

      07 and 2007-08 academic years to design and implement measures for the collection of

      data and assessments for each of the special themes (in keeping with WASC Standards

      and Criteria For Review 2.1,2.4,2.6,2.7,2.10,2.11,4.1, 4.4,4.5,4.6,4.7) and to review

      interim findings and to report back to the relevant committees. As noted in subsequent
Pitzer College Capacity and Preparatory Review Team Report                                           7


      parts of this report, members of the visiting team observed that most of the data collected

      was obtained by indirect measures.



      According to the CPR report, the College reviewed its infrastructures and budgets as they

      relate to the sustainability of programs and assessments of three themes (CFRs 3.1, 3.4,

      3.5, 3.7); reported on the findings to the relevant committees, and implemented any

      recommended actions that resulted from the initial assessments of student learning,

      organizational capacity, and budgets (CFR 4.1).



      During conversations with various campus constituencies, there were some indications

      that even though input was solicited from students, faculty, and staff, the report was

      prepared and reviewed by a few people. The College community seems to have been

      involved in the initial discussion phase of the report, and substantial efforts were made to

      gather responses from students, faculty, and staff. The College created a comprehensive

      WASC webpage easily accessed from the college homepage. Consistent emails and oral

      reports to faculty council and college council encouraged all campus constituents to study

      the WASC pages in detail and to comment on the report. Email calls and in-person

      presentations at faculty council exhorted faculty to participate in the process. Despite

      these efforts, while some people the visiting team met with were familiar with the process

      and the report, others were not and reported that they had not been given an opportunity

      to be part of an interactive process.
Pitzer College Capacity and Preparatory Review Team Report                                            8


      The institution considered each theme thoughtfully and rigorously, asking good questions

      and seeking evidence to answer them. The data support the claims made by the

      institution in addressing its Core Commitment to Institutional Capacity. The self-review

      also led to greater understanding of the institution’s capacity and its infrastructure for

      data collection and analysis. To cite one prominent example of this improvement, the

      team noted that the data collection process identified the need for a full time Institutional

      Research position. This position was created and filled, and the new Director of

      Institutional Research and Assessment is working with several other staff members to

      expand and coordinate data collection, data analysis, and the use of data tracked and

      evidence collected in making informed decision. Although the development of the

      process has been somewhat protracted, the College has also recognized the need to

      expand and improve methodologies for assessing programs and student learning

      outcomes through formative and summative evaluations.



      C. Institutional Response to Previous WASC Action Letter



      In its letter to the College's President after the 1998 reaccreditation, the evaluation team

      found much to commend in its visit and noted in particular how Pitzer had developed its

      distinctive educational objectives so that they formed a seamless whole. The team also

      offered several specific recommendations for further building on the institution’s

      strengths, and the WASC action letter of March 9, 1999 both endorsed the team’s

      recommendations and highlighted three areas in particular need of further attention:
Pitzer College Capacity and Preparatory Review Team Report                                       9


      Educational effectiveness and program review: Noting that Pitzer was in an early stage

      of formal assessment efforts, the team and the commission expressed concern about the

      absence of qualitative and quantitative data and recommended development of

      assessment methodologies leading to a systematic understanding of the extent and

      manner in which it is achieving its educational objectives. Special mention was made of

      the need for program reviews with guidelines tying them to student learning as an

      important vehicle for more extensive discussion of issues of quality.



      Retention: With a ten-year retention rate between 48.86% and 66.67%, Pitzer compared

      unfavorably with comparable liberal arts colleges, and WASC stressed the importance of

      studying and addressing this concern.



      Natural Science and Mathematics: The commission underscored the importance of the

      team’s recommendations concerning Pitzer’s involvement in the Joint Science Program

      and also voiced concern that the natural science and mathematics requirement may be

      met with courses outside mathematics and natural science.



      According to the CPR report, these three recommendations guided the current

      reaccreditation process and were to be addressed through the three themes, respectively.

      The extent to which the institution has adequately responded to all three previous WASC

      recommendations will therefore be addressed in Section II of the team report, which is

      organized by institutional theme.
Pitzer College Capacity and Preparatory Review Team Report                                           10


      SECTION II – EVALUATION OF INSTITUTIONAL CAPACITY UNDER THE

      STANDARDS



      As noted above, Pitzer College used the Special Themes approach in its CPR report and

      the visiting team has also chosen to organize its report according to those themes, with

      separate sections for each. The team has therefore not described how Pitzer meets each

      WASC Standard or Criterion for Review, but the report cites appropriate Standards and

      CFRs in the discussion of how Pitzer has defined its educational objectives, is achieving

      them through core functions, is developing and applying resources and structures to

      sustain its objectives, and is committed to learning and improvement.



      THEME ONE: LINKING THE ACADEMIC WITH RESIDENTIAL LIFE

      In framing its inquiry for the first theme, Linking Academics with Residence Life, Pitzer

      elicits the goals and objectives of Residential Life—create social spaces to enhance

      community, diversity and social responsibility; enact environmental sensitivity through

      ecological design that teaches and conserves; and integrate academic and residential life

      by incorporating instructional spaces into facilities, hosting faculty in residence (FIR),

      and enhancing co-curricular programs. In addition to residential goals and objectives, the

      campus also revisits the three questions it posed during the Institutional Proposal stage of

      its reaccreditation to add direction to this inquiry:
Pitzer College Capacity and Preparatory Review Team Report                                          11


      1.     What impact will the new residence halls have on student satisfaction and success

             (CPR 2.10)? In particular will the integration of academic and residential life

             improve student learning (CPR 2.11)?

      2.     Will the on-going residential life project improve the environmental practices of

             the campus and raise environmental awareness (CPR 4.2)?

      3.     Does the residential life experience foster student awareness and appreciation for

             the Pitzer mission and its educational objectives (CPR 1.2)?



      After reviewing the CPR and spending time with colleagues during the campus visit, the

      visiting team believes that the campus has laid a foundation for strengthening the

      connection between academics and residential life. The five indicators delineated on

      pages 7-8 of the institution’s CPR report coupled with the evidence they are generating

      serve as good starting points for the desired linkages. The campus has directed its efforts

      in various ways with respect to this theme. Discussed in this portion of the report are the

      implementation of three grant-funded social responsibility/civic engagement programs

      and College assessment efforts that feature the use of focus groups and survey

      instruments.



      The College has initiated three social responsibility/civic engagement programs:

      Changemakers, Faculty-in-Residence (FIR) and the Agnes Moreland Jackson Diversity

      Programs. Each of the programs is aligned directly with the mission of the University

      and with the culture of the campus (CFR 1.2).
Pitzer College Capacity and Preparatory Review Team Report                                         12


      Changemakers is overseen by The Center for California Culture and Social Issues

      (CCCSI), with the financial support of a Weingart Foundation Grant. Through its public

      speakers series, emergent community-engagement courses, community-partnerships, and

      student projects are meant to increase students’ awareness of social responsibility and

      their likelihood of being civically engaged.



      Faculty in Residence (FIR), funded by the Office of Student Affairs and overseen by the

      ad hoc FIR committee and the campus life committee, has great potential to help create a

      more seamless undergraduate experience for Pitzer students, although to do so it must do

      more than simply expect faculty to host a few programs. Based on conversations with

      one of the faculty members who lives in the halls and from some of her residents, the

      Pitzer FIRs indeed do much more than offer programs and are in fact fully engaged

      residents themselves. The Pitzer FIRs seem to be integrated both casually and

      structurally into the halls, sharing meals with their residents, spending informal time in

      public study spaces, walking around getting to know their student neighbors, and

      occupying informal “gathering” spaces like the common laundry room. The structural

      integration, via expectations for hosting Sunday salons and other types of academic

      conversations, also exists, although it seems somewhat less developed. Perhaps this is

      intentional given the spirit of informality and fluidity that marks the Pitzer experience.

      The hosted programs that have taken place (such as lectures and Salons) have had

      relatively low attendance, (4-15 attendees) even given the small college setting.
Pitzer College Capacity and Preparatory Review Team Report                                           13


      “The Agnes Moreland Jackson Diversity Program,” an endowed fund, is overseen by the

      campus life committee who accepts funding proposals for funding requests and makes the

      awards.



      Since these projects are very new the campus has not yet systematically evaluated their

      impact on the overall Pitzer experience and on student learning. As it moves forward

      with this set of related initiatives, the College might think about how they are connected

      to or build upon existing structures and programs and how the campus might develop and

      assess direct student learning outcomes that verify that learning has occurred (as

      compared to indirect student learning outcomes or satisfaction-based outcomes that report

      perceived increases in awareness or learning).



      Each of the programs noted above shows great promise for integrating academics and

      residence life (CFR 2.9). To discern the extent to which they accomplish this goal,

      however, their correlative assessments should include at least some direct learning

      measures in addition to the indirect measures and satisfaction based measures described

      in the CPR report.



      A second major thrust used by Pitzer to bolster the academic-residence life connection is

      a series of tools and approaches designed to assess academic-residence life connections.

      The annual student survey figures prominently as a means that is used to monitor

      students’ attitudes and behaviors, particularly as items on the survey are related to social

      responsibility, environmental awareness, and intercultural understanding (CFRs 2.3-2.6).
Pitzer College Capacity and Preparatory Review Team Report                                            14


      A second instrument used to evaluate residential life at the college and its connection to

      the academic experience is the survey of residence hall mentors. It is not entirely clear to

      the team how this group of student leaders differs in role and responsibility from the

      residence hall assistants, even after reading the CPR report and discussions with

      residence life staff (CFR 2.13), and clarification would be useful for both internal and

      external constituencies. As suggested above, the College might consider how the hall

      mentor position is integrated into the existing infrastructure (e.g., the RA staff) and how a

      survey of the individuals in this role informs the broader discussion of effective

      academic-residence life connections.



      Holding focus groups with residents is still another method used to discern the strength of

      the academic-residential life connection. To date, residents’ responses to questions posed

      during these groups are aligned with what one might expect to hear from any successful

      residential life program. The team is not sure whether or not these responses shed any

      additional light on the impact of the new facilities on learning or on bolstering the

      residential life-academic connection. One would expect, for example, that residents

      would use and appreciate the new, nicely appointed study rooms, but the extent to which

      these rooms impact student learning and curricular—co-curricular integration remains

      unknown. Likewise the focus group questions and responses did not confirm a

      relationship between the LEED Gold certification and increased awareness in

      sustainability or behavior modifications that promoted better use of energy. As the

      writers of the Pitzer CPR report suggest, demonstrating causal relationships between the
Pitzer College Capacity and Preparatory Review Team Report                                              15


      new dorms and improved study habits, increased commitment to social responsibility and

      diversity, and respect for physical property will be difficult to demonstrate.



      Collecting compelling evidence to demonstrate the academic-residence life connections

      that the institution is attempting to nurture and develop will likely require residential staff

      and their faculty colleagues to include more direct measures of learning outcomes.

      Surveys that solely or primarily focus on student satisfaction responses and indirect

      outcomes (e.g., self-reported increases in learning or appreciation) rather than direct

      measures that can “verify” what students have learned or how they have changed (e.g.,

      instruments that ask students to demonstrate knowledge acquisition, information

      retention, attitudinal or behavior change) provide much weaker evidence for “results.”

      The College might consider how to support the inclusion of meaningful direct measures

      in their assessment of the academic-residential life connection.



      Finally, the analysis of academic success/attrition data form a third path used to explore

      the College’s actualization of Theme One (CPR 2.10. While these data can be used to

      tell part of the student success story, they cannot be used to reveal any “cause and effect”

      of the new residence halls and their corollary programs on retention and graduation rates.

      To use these data as partial indicators of success or as being correlated with success, the

      College may find it useful to put them in a broader context. In other words, what else

      might have been changed or added to (subtracted from) the Pitzer experience to increase

      continuation, retention, and graduation rates? The more fully the retention story is told

      (e.g., increased selectivity of Pitzer students, improved reputation of the entire
Pitzer College Capacity and Preparatory Review Team Report                                         16


      consortium, bolstered marketing initiatives, improved website) the more honest and

      accurate a picture one sees.



      The visiting team agrees with the writers of the CPR report that one form of valuable data

      –though it is hard to collect—can be gleaned through exit surveys with students who do

      not return. The team was impressed with Pitzer’s attempts to speak with those students

      who are requesting transcripts from the registrar’s office and suggests that the College

      explore ways that more departing students might be encouraged to complete this survey.



      While the campus is setting the groundwork for connecting academics with residential

      life, to fortify this linkage, the team has the following observations and suggestions: It

      may make sense in the context of this theme to think more broadly about the links

      between the academic and co-curricular, or alternatively to explore how the new

      residence halls have promoted involvement with other co-curricular experiences.. An

      interesting and related question is to what extent these new facilities “teach”? If the

      assumption, expressed in the CPR report, is that the new residence halls teach students

      about sustainability, clean energy, and environmental awareness more generally, how

      does this teaching unfold and how might the resultant learning be measured? Finally,

      thinking intentionally about how new projects are connected to and build upon other

      campus initiatives in support of academic and co-curricular connectivity might help to

      assist the institution in conceptualizing broader linkages alluded to above.
Pitzer College Capacity and Preparatory Review Team Report                                         17


      As noted several times in the narrative above, the campus has relied heavily in this theme

      upon student satisfaction surveys and indirect outcomes (e.g., self reported increases in

      learning or appreciation) rather than a reliance on direct measures that can “verify” what

      students have learned (e.g., instruments that ask students to demonstrate knowledge

      acquisition, information retention, attitudinal or behavior change). Explicit attention to

      developing direct measures and the means to evaluate them will provide more powerful

      and compelling evidence for the efforts that the College is clearly making toward

      strengthening the ties between academic and student

      clearly making toward strengthening the ties between academic and student life. While

      the striking improvements in student retention are commendable, there is little data

      actually explaining how changes in campus life (including the new residence halls) have

      led to increased retention or learning.



      THEME TWO: COMPREHENSIVE REVIEW OF ACADEMIC PROGRAMS



      Educational effectiveness and systematic program review were major areas of concern on

      the part of the Commission following the Pitzer reaccreditation review in 1998. It is

      notable that the Commission linked these two issues together in formulating their

      statement to the College. The Commission was concerned about the absence of data in

      efforts to evaluate and draw conclusions concerning the extent to which educational

      objectives were achieved. The Commission recommended that program reviews, which

      had been initiated on a five-year cycle in 1995, would benefit from guidelines tying them

      to the assessment of student learning. Pitzer therefore considered its own commitment to
Pitzer College Capacity and Preparatory Review Team Report                                          18


      rigorous evaluation of academic programs, and its desire to address Commission

      recommendations from the 1998 review, and developed Theme 2 for the present

      reaffirmation review.



      The revised program review process was adopted in 2005, and at this point four field

      groups (Environmental Studies, Economics, Psychology, and History) have gone through

      the external team review phase. Three more field groups (Math, Sociology, and

      Anthropology) have recently completed self-studies. In general, the periodicity of the

      reviews is eight years.



      The current program review process was revised in several important ways from the

      previously existing self-study mechanism. The revision was informed by a Pitzer’s study

      of program review procedures at 12 aspirant peer institutions. Pitzer found it was the only

      one not systematically including external advisors in its program review process. Most

      significantly therefore, the new process now includes the participation of an external

      team of peer reviewers.



      The program review is structured as a three-phase process extending over three years as

      follows:

      1.     The development of a self-study document by the field group faculty;

      2.     The visit of the external review team and the submission of their report;

      3.     The consideration of the report by the field group faculty, and the development of

             their response and action plan.
Pitzer College Capacity and Preparatory Review Team Report                                             19




      This iterative study model could lead to thoughtful actions in aligning program structure

      and curriculum design with student learning outcome objectives. For example, the

      Environmental Studies field group responded with great interest to the external

      reviewers’ suggestions that they add a stronger science component to the curriculum.

      Curriculum modifications are accordingly being developed, significantly, with strong

      support from the Dean. The logical next step in this process is the development of explicit

      student learning outcomes that might be expected to result from the inclusion of the new

      science component.



      However, the long period of development and the relatively recent adoption of the

      revised process means that it is still in its initial phases of implementation. At the time of

      the CPR team visit, only the Environmental Studies group had completed the entire

      process, and was yet to implement recommendations. Therefore, the College faculty

      have formed only preliminary judgments about the effectiveness of the new mechanism.

      As documented in the CPR report and based on numerous discussions with faculty

      groups during the WASC visiting team’s visit, the College is going through an

      understandable learning process in implementing the new review mechanism.



      The overall objective for Theme 2 is to evaluate the implementation of the new Program

      Review process. The expressed interest in implementing modest changes to the process

      as programs go through it, to provide feedback about what is working and what is not, is

      consistent with WASC expectations (CFR 2.7).
Pitzer College Capacity and Preparatory Review Team Report                                          20




      Some positive and important commitments in the development of an assessment capacity

      were noted, such as the recent recruitment of a full-time Director of Institutional

      Research and Assessment. Although the new Director is still relatively new to the

      College, the team was told and observed that she played an important role in preparing

      data and documents for the WASC visiting team. In the near future, the Director should

      be able to support a more robust and effective use of data and the development of a

      culture of evidence.



       Several faculty members informed the WASC visiting team of their participation in

      discipline based assessment and educational conferences. Additionally, the WASC

      visiting team notes the creation of the Teaching and Learning Committee (TLC). In

      keeping with Pitzer’s inclusive culture, TLC brings together faculty, students, and staff.

      The committee has sponsored a variety of efforts to build a culture of teaching and

      learning excellence. These include workshops, luncheons with talks by faculty or

      external speakers, a book-reading group, and new faculty orientation. Topics have

      included classroom instruction, pedagogy, student interaction, diversity, and advising.

      Although TLC has not yet focused on establishment of learning outcomes and

      assessment, the group forms a natural locus of interest and dissemination of best practices

      on these vital topics. It is encouraging that a permanent budget line has been recently

      committed to support the work of TLC. We note that further resources may need to be

      dedicated in building a more broad-based, ongoing assessment effort in the field groups.
Pitzer College Capacity and Preparatory Review Team Report                                       21


      For example, teams might more systematically attend assessment workshops sponsored

      by WASC or other organizations that transcend disciplinary boundaries.



      The team was most encouraged to note that the recently adopted Program Review policy

      requires field groups to address the following questions in their self-studies:

      1.     What approaches are taken within the Field Group to assess student learning?

      2.    How does the Field Group assess student learning?



      Despite these positive developments, the WASC visiting team members are concerned

      that the objectives for Theme 2 are too process-oriented and not strongly enough

      anchored within an assessment framework. The team noted a lack of specificity in

      setting course, curriculum, and program goals for student learning outcomes, which

      would help focus efforts in assessment and data analysis (CFR 4.3). For example, the

      required data exhibit 7.1, “Inventory of Educational Effectiveness Indicators” presented

      learning outcomes, “educational objectives,” only at the institutional level: breadth of

      knowledge; understanding in depth; critical thinking, formal analysis, and effective

      expression; interdisciplinary perspective; intercultural understanding; and concern with

      social responsibility and the ethical implications of knowledge and action. When asked

      for additional information for program level learning outcomes, the same six (i.e.,

      institutional level) outcomes were the institution’s response. Syllabi examined in the

      team room, similarly, did not explicitly contain course level learning outcomes. Nor was

      there any evidence of mapping programmatic to course level learning outcomes.
Pitzer College Capacity and Preparatory Review Team Report                                          22


      In discussions with many faculty groups, and in review of field group self-studies, the

      WASC Visiting Team found that student learning outcome data are not routinely used for

      Program Review and improvement (CFR 2.4 and 2.7).



      One example of a pilot assessment framework did come to the WASC team’s attention

      during the campus visit. This example was provided by the chemistry major within the

      Joint Sciences program. The team looks forward to reviewing additional documentation

      and evidence for the implementation of assessment models during the EE Review.

      Program reviews should serve as a natural vehicle for a more robust development of an

      assessment framework. In fact, Pitzer’s CPR report provides a similar sentiment:



            “While self-studies have improved in depth and quality, it is clear from the

      feedback we have received from the external evaluators that we still have further to go in

      making these reviews as useful as they could be in terms of facilitating student

      learning…We hope that as we continue, we will have an even greater focus on the

      educational outcomes and their assessment by the field groups.”



       While it should eventually form the backbone of College assessment of educational

      effectiveness, the potential of the new program review process may still be mostly

      unrealized at the time of the upcoming educational effectiveness review. Few field

      groups will have completed the program review process, so that systematic comparisons

      and analyses across field groups are likely to be premature. In addition, given the current

      capacity for serious assessment efforts, and the seeming cultural biases against
Pitzer College Capacity and Preparatory Review Team Report                                          23


      developing definition of student learning outcomes at the course and program level,

      collection of direct and indirect evidence concerning student achievement of the learning

      outcomes, and “closing the loop” to make programmatic and course changes in response

      will not be likely to have been completed.



      The Role of Science and Mathematics in Liberal Education



      In its 1998 action letter, the Commission expressed concern that “…natural science and

      mathematics requirements may be met with courses outside mathematics and natural

      science”. The Commission further noted that “A lack of preparation in these areas is

      significant…” and encouraged the institution to examine the roles of “mathematics and

      science in achieving Pitzer’s goal of providing students with an excellent liberal arts

      education”.



      Pitzer embedded their response to this concern within the revised program review process

      (Theme 2), Therefore, the WASC visiting team also considered whether the

      implementation of a new program review procedure had been effective in ameliorating

      this issue.



      Progress appears to have been made with respect to embedding the natural sciences into

      the broader liberal arts curriculum. The Environmental Studies field group, as mentioned

      above, is moving to integrate a stronger emphasis on science into its curriculum. Pitzer is

      currently in the design phase for a new science building in the heart of the campus,
Pitzer College Capacity and Preparatory Review Team Report                                           24


      providing a physical demonstration of commitment to and engagement in science. And

      we learned that science will be an important discussion point in the upcoming strategic

      planning process. A visit with the Joint Sciences faculty convinced the team of their

      commitment and accessibility to Pitzer students. The graduation requirements for Pitzer

      students include one full credit, a semester length course in natural science. While not all

      of the accepted courses include a laboratory component, the team was satisfied that all

      Pitzer students participated in at least one course with substantive science content.



      More concern remains regarding the mathematics issue. Students at Pitzer may satisfy

      graduation requirements by one full credit course in either formal reasoning (logic) or

      completion of any natural science course that has a college-level math course as a

      prerequisite The team was provided data that indicated approximately 40% of Pitzer

      students in recent years opted for a formal reasoning course rather than one in

      mathematics. While the team was told that the overwhelming majority meet the formal

      reasoning requirement through a statistics or economics course, the formal reasoning

      courses do not appear to satisfy WASC general education requirements (CFR 2.2a),

      which state that undergraduate programs should “ensure the development of core learning

      abilities and competencies including… college-level quantitative skills”.



      In summary, there has been progress in integrating natural science into the educational

      environment of the College, but development of a college-level quantitative reasoning

      curriculum requirement appears to require additional focused attention.
Pitzer College Capacity and Preparatory Review Team Report                                          25


      Further, although the new program review process was intended to consider the role of

      science and mathematics on a major-by-major basis, the visiting team learned that this

      issue has not been consistently addressed in program reviews to date. In fact, it seems

      unlikely that disciplinary review teams would take up this issue in depth during a two-day

      campus visit. The team suggests that Pitzer faculty must themselves reconsider the

      question of whether Pitzer students are satisfactorily prepared in mathematics at a level

      appropriate for liberal arts baccalaureate degree graduates. External reference would be

      welcome, however, through a survey of peer institutions, or by assembling a dedicated

      external advisory team to review the current Pitzer requirements.



      In summary, the team appreciated Pitzer’s admirable work in implementing a new

      external review process. It is concerned, however, that the slow implementation schedule

      following the previous reaccreditation review will limit the availability of results at the

      time of the upcoming Educational Effectiveness Review. The team is also concerned that

      Pitzer’s current implementation of comprehensive program review is not consistently and

      strongly enough anchored in an assessment framework. The objectives in Theme 2 are

      generally process-oriented, and not strongly enough focused on developing the program

      review process as a vehicle for direct assessment of student learning outcomes. Finally,

      concerns were expressed as a result of the last reaccreditation review regarding

      mathematics and natural science graduation requirements. Pitzer intended to address this

      concern through the revised review process, but to date this has not been consistently

      implemented. Indeed, it seems unlikely that disciplinary external review teams will
Pitzer College Capacity and Preparatory Review Team Report                                        26


      address this issue with depth or effectiveness, given factors noted above including short

      visit times and an inclination to focus on disciplinary issues.



      THEME THREE: CONNECTING THE GLOBAL AND THE LOCAL



      This theme is an excellent example of how Pitzer aligns educational objectives with

      programs designed to insure student success and curricular and co-curricular (CFR 2.1,

      1.1, 1.3, 2.11, 2.13). Support by external funding is just one indicator that Pitzer

      programs are well known and respected in this innovative area.



      Theme Three also embodies Pitzer’s pervasive institutional commitment to its stated

      purposes and educational objectives.(CFR 1.1, 1.2). This theme both directly reflects and

      seeks to connect two of those objectives in particular, “Intercultural Understanding” and

      “Concern with Social Responsibility and the Ethical Implications of Knowledge and

      Action.” It further connects two previously well-developed but separate efforts:

      international study, chiefly through the Pitzer Study Abroad Program; and local service-

      learning work, community service through volunteering, and community-based

      research. Recognizing that understanding of “difference” at an international level has

      typically been poorly connected to understanding of “difference” at the local level, the

      focus of this theme is on “bridging the divide” and realizing the synergies between the

      global and local through courses and pedagogy.



      The overarching goal is to enrich student learning through experiential, interactive, and
Pitzer College Capacity and Preparatory Review Team Report                                          27


      research opportunities at home and abroad that engage them in cultures and perspectives

      different from their own and also ask them to reflect on how globalization affects both

      international and local communities. (CFR 2.3, 2.4, 2.8, 2.11) Four different programs

      are brought together under this creative umbrella where, as the team was told in interview

      with its leaders, “intercultural meets multicultural,” study abroad.



      Pitzer College is understandably proud of its history of encouraging students to spend

      a semester or a year studying outside the College. The majority (in the current year, over

      65% of students who go abroad) pursue this opportunity on Pitzer programs in Botswana,

      China, Costa Rica, Ecuador, Italy, Japan, and Nepal. The Pitzer-run programs feature

      a combination of curricular and “active” learning including intensive language study,

      home stays with host families, and independent study projects.



      Students are required to synthesize and reflect on their classroom and community

      engagement learning in a “field book,” a writing portfolio. Faculty and staff use textual

      analysis of the students’ writing as a way of assessing the skills they intend to promote

      through this type of learning. (CFR 2.3, 2.5, 2.6) In a pilot project some years ago, there

      was an attempt to analyze and aggregate the findings from the data reflected in the field

      books, and staff members plan next year to work with IR and graduate students to return

      to the rich data for further, more comprehensive analysis.



      Students are encouraged to integrate their study abroad, moreover, into their four year

      plans through preparatory courses, orientation, and advising in their first and second
Pitzer College Capacity and Preparatory Review Team Report                                          28


      years, and a few students (on average 30 a year) also enroll in a colloquium on study

      abroad when they return. Pitzer is rightly proud of the large number of its graduates

      who win national fellowships through Fulbright, Watson, Rotary and Coro awards, and

      since students who study abroad make up 85% of those who win these awards, Pitzer

      views its emphasis on study abroad as the cause of its outstanding achievement in

      student fellowships.



      In the last few years, the College has expanded its study abroad opportunities, which

      were utilized by about 50% of Pitzer students, even further to the current 72% by adding

      reciprocal exchange programs with 39 international and 13 domestic institutions.

      According to the faculty and staff with whom the team met, perceived needs in the study

      abroad area include support for more current faculty to travel to both Pitzer program sites

      and exchange sites to provide continuity to the vital connections that underpin these

      programs and to create a stronger experience at the exchange sites. The WASC visiting

      team was not able to determine how the international students who come to Pitzer are

      faring, or how their presence and engagement in the campus community promotes both

      college-wide goals and the more specific aims of courses, programs and individuals,

      though this seems a logical area to explore in assessment of the College’s goal of

      furthering “intercultural understanding.”



      The second program analyzed under this theme is the Community-Based

      Spanish Program, which began in 2000 with support from Atlantic Philanthropies. This

      program matches Pitzer Spanish students with host families in neighboring communities.
Pitzer College Capacity and Preparatory Review Team Report                                         29


      It affords one of the best examples that the WASC visiting team found of a developed

      and systematically applied student learning outcome assessment, using both direct

      measures (CFR 2.2, 2.4) and well-established expectations for how well students

      achieve outcomes (CFR 2.1, 2.4, 2.5). Entrance and exit oral proficiency interviews

      (OPI), exit questionnaires, observations, journals, focus groups (see Pitzer CPR report,

      appendix 32) all indicate that the program is successful in achieving its goals.



      The third component of this theme is the newest, the “paired courses” program.

      Supported by Mellon and Christian Johnson Endeavor, this initiative begins when faculty

      are awarded travel grants to sites abroad, where they begin to develop a new course. That

      course is then taught in the spring semester at Pitzer, followed by a month-

      long companion course taught in the summer at one of the study abroad sites, on the same

      topic but situated in a different context. Three paired courses have been offered at this

      point. Preliminary assessment has been carried out through student evaluations,

      focus groups and staff interviews.



      Paired courses are ultimately seen as something to be incorporated in plans for a Center

      for the Study of Global Communities, which will be directed by an endowed

      faculty chair. Fund-raising for both the center and the chair is ongoing, and faculty are

      involved now in planning for the new Center. The Academic Planning Committee has

      asked the original global-local faculty group to help define the mission and goals for the

      center and to make recommendations for how this new Center will work with the existing

      Study Abroad Program and CCCSI (CFR 4.1) The broad objectives of this program are
Pitzer College Capacity and Preparatory Review Team Report                                          30


      to examine how global phenomena like urbanization, pollution, and health issues are

      differently expressed in different cultural contexts.



      The WASC visiting team was not able to determine whether, and if so, how more specific

      aspects of student learning are also being identified for assessment in this program. The

      paired courses are labor-intensive and therefore expensive, so it will be important for

      Pitzer to find direct and indirect measures of value added, to be sure that the benefits in

      terms of student and faculty learning are commensurate with the resources required to

      sustain this work.



      The final component of this theme is the Center for California Social and Cultural Issues.

      Founded in 1999 with support from the Keck and Irvine foundations, the Center is

      designed to further the objectives of intercultural understanding and social responsibility

      though community partnerships in various problem-solving projects. Programs include

      the semester-long “Pitzer in Ontario” program, integrating student internships in

      various agencies and not-for-profit organizations with interdisciplinary coursework at

      Pitzer. The Center also provides support for long-term (more than 5 years) relationships

      with community partners, overseen by a faculty member and offering students action

      research projects. Assessment of projects uses field books, entrance and exit

      surveys, interviews, and writing assignments (Pitzer CPR report, appendix 38). Results

      have led to changes (such as providing a more theoretical foundation for community-

      based learning, more extensive orientations and debriefings for specific partner sites, and

      more orientations for staff members).
Pitzer College Capacity and Preparatory Review Team Report                                         31


      Based on the evidence cited above found in the institution’s CPR report and

      its appendices and other materials in team room, as well as information learned from

      interviews with staff and faculty and students, the WASC visiting team concludes that

      Theme Three presents some of the most persuasive support for Pitzer’s capacity for the

      Educational Effectiveness Review. Under dedicated and hard-working leadership, most

      programs show the emerging uses of assessment to determine and reflect on

      how successfully learning outcomes consistent with Pitzer’s educational objectives are

      being achieved. Throughout this theme, the team finds programs using performance data

      and analysis and cultivating a culture of inquiry leading to improvement.



      SECTION III – FINDINGS AND RECOMMENDATIONS



      The WASC visiting team has identified the following findings based on the evidence and

      analysis in section II.



      Commendations:



      1.      The visiting team observed and experienced an outstanding level of commitment

              of the faculty and staff to teaching, scholarship and creative activity, student

              learning and the widely shared educational objectives of the institution (Standard

              II). Faculty and staff spend large portion of their time working and supporting

              students. Students recognize that they are in a very special place and have great
Pitzer College Capacity and Preparatory Review Team Report                                          32


             appreciation for the faculty and staff that create this learning environment and the

             close working relationships they have established with them.



      2.     The visiting team found much to praise in Pitzer’s institutional purposes and

             integrity (Standard I). All members of the community, faculty, students and staff

             understand and are committed to the institutional purposes. They live the Pitzer

             mission in their campus life, in both academic and co-curricular activities.



      3.     Theme Three: Connecting the Global and the Local presents some of the most

             persuasive indications of Pitzer’s commitment to learning and improvement

             (Standard IV) and the College’s capacity for the Educational

             Effectiveness Review. Under dedicated and hard-working leadership, most

             programs show the emerging uses of assessment to determine and reflect on

             how successfully learning outcomes consistent with Pitzer’s

             educational objectives are being achieved. Throughout this theme, the team

             found programs using performance data and analysis and cultivating a culture of

             inquiry leading to improvement.



      4.     In the same vein, Pitzer has recognized the need for data collection and

             demonstrating learning effectiveness supported with data (Standard IV).

             Institutional research function has evolved and the college has hired a full-time IR

             director.
Pitzer College Capacity and Preparatory Review Team Report                                        33


      5.     Pitzer has made considerable stride in admission and retention of students in last

             few years. The student acceptance rate has moved from 75% to 25%. This is a

             big accomplishment in a very short time.




      Recommendations:



      1.     Pitzer’s current implementation of comprehensive program review is not

             consistently and strongly enough based in an assessment framework. There is

             need for specificity in setting course, curriculum, and program goals for student

             learning outcomes, which will help focus efforts in assessment and data analysis

             (CFR 4.3).



      2.     Concerns were expressed in the last reaccreditation review regarding mathematics

             and natural science graduation requirements (CFR 2.2a). There has been progress

             in integrating natural science into the educational environment of the College and

             Pitzer undergraduates, but the development of a college-level quantitative

             reasoning curriculum requirement appears to still require focused attention. The

             visiting team learned that the evaluation of natural science and mathematics in a

             broad Pitzer liberal arts education has not been consistently addressed in program

             reviews to date, and it does not appear that the assessment of science and

             mathematical education, or general education more generically, will form part of

             the program review process. The team suggests that Pitzer faculty must

             themselves reconsider the question of whether Pitzer students are satisfactorily
Pitzer College Capacity and Preparatory Review Team Report                                           34


             prepared in mathematics and natural sciences at a level appropriate for liberal arts

             baccalaureate degree graduates.




      3.     As the faculty who have been at Pitzer since its beginning in 1963 retire and new

             faculty join the campus community, the College needs to attend to their

             understanding of the educational objectives and institutional purposes of Pitzer

             (CFRs 3.2, 3.3, 3.4). The College is remarkable at present for the level of faculty

             commitment to its distinctive mission and its inclusive governance structure.

             Faculty are responsible for individual student advising in addition to teaching,

             program development, and various service commitments. In recent years, there is

             a perception among some faculty that more emphasis is being placed on faculty

             scholarship, and this has generated an understandable concern, felt on many

             campuses today, about how Pitzer is going to sustain this labor-intensive model in

             future. Faculty and administrative leaders are aware of this issue, and the team

             encourages continued attention to the critical alignment of faculty and staff efforts

             and rewards with the institutional purposes of Pitzer.



      SECTION IV – PREPARATIONS FOR THE EDUCATIONAL EFFECTIVENESS

      REVIEW



      Throughout this report, the visiting team has noted positive and important steps in the

      development of an assessment capacity of the institution, such as the recent recruitment

      of a full-time Director of Institutional Research and Assessment and the expanded and
Pitzer College Capacity and Preparatory Review Team Report                                          35


      more rigorous process of academic program reviews that was recently adopted. As the

      team also has observed, however, at the program and course level student learning

      outcomes have not yet been consistently established or communicated in syllabi and

      publications. In discussions with many faculty groups, and in review of field group self-

      studies, the team found that student learning outcomes data are not routinely used for

      Program Review and improvement (CFR 2.4 and 2.7). Program reviews, moreover, are

      not looking at General Education, and therefore they are not addressing the concern about

      math and science. At the institutional level, the six educational objectives are very broad

      and hence difficult to measure. The institution needs to give further serious thought as to

      how they can be defined in ways that they can be measured.



      Members of the visiting team also noted some resistance to the assessment of student

      learning outcomes as a formal enterprise, perhaps tied to a belief that WASC expectations

      require quantitative data in all cases. Most of the data collected so far uses indirect

      measures such as self reported surveys. The team is therefore concerned about the

      institution’s capacity for assessment and ability to demonstrate organizational learning

      using results of the assessment of student learning before the EER visit.



      To summarize and reiterate, Pitzer has begun programmatic work designed to address the

      themes that form the focus of their institutional proposal. Faculty and staff members

      have begun the program review process, which provides some foundation for assessment.

      But they do not have a well developed assessment framework and they are not yet doing

      much work with student learning outcome data. As mentioned above, Themes 1 and 3
Pitzer College Capacity and Preparatory Review Team Report                                         36


      are furthest developed at this point. The EER will need to focus more strongly on data

      that reflects direct assessment of learning outcomes rather than indirect measures such as

      satisfaction surveys and self-assessment of learning gains. The capacity and

      infrastructure to complete the EER can be described as in its infancy or “emerging” and

      will require more time to collect data that will then need to be compiled, organized, and

      analyzed.