2002-03 Service-Learning Assessment Report Service-Learning Assessment Committee Bobby Matthews, Chair Previous Assessments: In 2001-02 the Assessment Committee conducted a preliminary overall assessment of LSU’s commitment to service, based on Barbara Holland’s grid “Levels of Commitment to Service, Characterized by Key Organizational Factors Evidencing Relevance to Institutional Mission.” Then, triangulating syllabus analysis, faculty focus sessions, and faculty reports analysis, the committee determined that the service-learning components of courses awarded incentive grants were generally in line with best practices as indicated by the mission statement of the ServiceLearning Council. Overall results were quite positive. The findings indicated, however, that a more rigorous rfp, including greater expectations and incentives for extended planning and preparation, would further enhance student learning and quality of service.
The results of the 2001-02 report were used to revise the next rfp and planning process for 200203. The results were also shared with grantees and other faculty, both current service-learning faculty and those planning new courses, thereby shaping current and proposed courses. In addition, a $5,000 Faculty Fellow was established to deepen the level of SL scholarship at LSU and to mentor other faculty regarding nationally accepted best practices and results of our own assessments. Following are the results of the 2002-03 year of assessment.
Methodology: Methodology for 2002-03 included five stages of formative assessment, all of which recognize and value faculty reflection by all stakeholders to promote continuous improvement: A comparison of each course’s syllabus and other materials related to the service-learning component of the course to three criteria from Bringle and Hatcher’s definition of servicelearning, as printed in the LSU Service-Learning Faculty Manual: (1) “a credit bearing experience”; (2) involving “an organized service activity that meets identified community needs”; (3) reflection “on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of civic responsibility.”
1
Focus sessions in which faculty grantees were questioned about their implementation of the above elements during their courses. They were specifically asked to identify learning goals and to show how they were enhanced by the service-learning project, to indicate the importance of the service-learning to overall grade in the course, to show how the project met community needs, and to describe their process for evaluation and reflection. Focus sessions in which students were questioned about the implementation and effectiveness of the above elements during their courses, with an emphasis on enhanced academic learning and civic understanding. Analysis of reports prepared by grantees at completion of the courses which addressed the following: (1) course goals and activities, (2) partnership with community, (3) reflection component, (4) evaluation process, and (5) professional scholarship and public impact. Survey of community partners. Though not included in the Assessment Committee’s formal methodology, all service-learning faculty, including those without grants, were provided with forms for assessing impacts on student learning. Results of these surveys are used by individual faculty for formative evaluation and revision of individual courses.
Conclusions: Faculty have designed a variety of creative service projects that effectively enhanced course content and engaged students. All projects met significant course learning goals identified in syllabi. A variety of significant community-defined needs were met though these service-learning projects. Faculty showed commitment to the reflective process and to participate in assessment. All faculty participants indicated commitment to service-learning pedagogy and all plan to revise when they teach the course again. Many 2001-02 grantees and all 2002-03 grantees have participated in at least one presentation during which they shared what they had learned with other faculty. Several have conducted formal research designed to assess learning in their particular disciplines. 2002-03 grantees have incorporated professional reflection into a panel presentation called “Faculty as Reflective Practitioners” and have made a panel
2
presentation for CELT’s faculty development series and the International Conference on Civic Education Research. Faculty believed students learned more. Faculty also reported that they, too, found themselves more engaged, their teaching more interesting, and their methods positively changed. Many reported that their teaching in non-service-learning courses had improved as a result of what they had learned through using service-learning pedagogy. 2001-02 faculty recognized that some service projects, while beneficial, needed further integration into the courses…either through more rigorous, more frequent, or more structured reflection; more time devoted to the project; more weight placed on the service project in the evaluation process; clearer articulation of service as an expressed objective in the syllabus; more explanation of connections between learning and service goals; or more thorough evaluation of learning outcomes in terms of both civic and academic goals accomplished through service. 2002-03 assessment indicated improvements in all these areas, 2001-02 faculty recognized the need for stronger community partnerships and most planned to further develop and sustain their partnerships with community sites through formal agreements, more communication, site visits, better orientation and training of students, and more sharing of the instruction role with community members. 2002-03 assessment indicated improvements in all these areas. Students in one class, however, found that they needed more communication with their partner. This finding foregrounds the importance of clear understandings and contracts between faculty member, students, and partners. It also indicates that students need to spend some time on site, even for project models, and that a single partner for large classes may not work as well as one partner for each group. Some faculty expressed some disjunction between need to carefully select a partner and need to give students ownership and access to partners. 2001-02 faculty found reflective activities beneficial to the learning process. Most indicated plans to focus their reflective assignments more specifically on learning goals, to reflect more frequently, and to provide more frequent and timely feedback on these assignments. 2002-03 assessment indicated improvements in all these areas, though one class was challenged by large numbers of students and faculty inexperience with the reflective process.
3
This faculty member found that the evaluation criteria for reflective activities did not always match his assignment and that students did not get enough formative feedback. This faculty member recognized the problems and planned to use a different approach next time. Faculty noted significant improvement in critical thinking skills following regular feedback to reflective activities. i.e. Critical thinking can be learned. Most faculty moved toward assigning focused prompts and interactive contextualized discussions, rather than summative reports. They all indicated that peers should be involved in feedback. One of the most positive outcomes is the evidence of increased use of “writing to learn” strategies, even in courses that had not previously used such strategies. In addition, group oral presentations were often used to develop students’ collaborative, analytical, and communication skills…competencies called for in the flagship agenda. Most 2001-02 faculty relied either on anecdotal evidence or traditional evaluation procedures such as content tests, but several recognized the need for pre and post evaluation directed more specifically to the specifics of the service-learning process. 2002-03 assessment indicated more attention is being paid to pre and post formal assessment and to articulating academic and civic learning goals in the syllabi. Faculty recognized that civic learning goals are not automatically accomplished. Students need structured strategies to accomplish civic goals, just as they do for academic goals. Faculty and students sometimes tried to do too much and both acknowledged that servicelearning is time consuming if done well. Depth, rather than breath, might be a more effective approach in many service-learning classes to avoid overwhelming both students and faculty with work. Students were quite positive about the service-learning experience…more engaged, more empowered, more responsible. Students felt that the service-projects helped them accomplish course learning goals, that their learning mattered, and that they learned more. Many reported that they worked harder in these courses, not because the courses demanded more but because of the authenticity of their work and their commitment to community partners. Many students indicated that they would like to take other courses but that few SL courses were available at upper levels.
4
Students reported increased understanding of issues of diversity. One student indicated that the full value of the course was not apparent until later when he had a roommate from another country and he felt much better able to deal with difference because of his servicelearning experience. Students indicated closer ties to faculty as a result of the course. Students reported increased understanding and awareness of social problems and the challenges faced by marginalized citizens.
Recommendations: Because our initial assessment indicates that service-learning significantly enhances student learning, helps to fulfill LSU’s flagship agenda, and more fully engages both students and faculty, the university should devote substantial resources to advancing the program and developing quality classes. This commitment would help to guarantee that every communitybased course is consistent with university learning goals and promotes ethical and mutually beneficial partnerships with the community. Such resources would include, but not be limited to, sustainable funding for faculty fellow to mentor new faculty, additional support staff and operations for summer session, additional office space, supplies for recruitment and support. Sustainable funding should be appropriated for at least 8 annual $3,000 awards and resources and materials for 10 hours of planning sessions during the semester before implementation, followed by formative group assessment during and after implementation. The University should develop a mechanism and structure to coordinate the many University-related outreach activities and learning initiatives. Such coordination would realize much greater efficiency for all stakeholders including community partners, students, faculty, and administration. The semester of planning activities and formative reflection activities for faculty proved fruitful and should be continued. The faculty who worked most closely with the servicelearning office seemed to have the most successful classes. Planning should continue to emphasize strategies for developing both civic and academic goals, full alignment of the service and learning goals, structured and frequent reflection activities, criteria for student reflection activities, full engagement of partners throughout planning and implementation, and rigorous measurement of outcomes.
5
Grants should be awarded only to classes that meet regularly together as a group. Grants should target departments with plans to integrate service-learning into required curricula. This strategy would eventually guarantee that every LSU graduate has at least one service-learning course. In the short term, it would make courses more accessible to students in a variety of majors. Faculty with larger classes might maintain strong partnerships while giving students ownership by dividing the class into teams and allowing each team to work with a different partner or by allowing students to work with a list of partners who have signed a carefully articulated agreement between faculty member and partner. More resources should be aimed at developing interdisciplinary service-learning work. Every effort should be made to make sure that all partners understand goals and expectations. Staff should aim for adequate oversight, risk management, and assessment without burdening students and/or faculty with time-consuming paperwork.
Survey of Fall 2002 and Spring 2003 Community Partners
Methodology: The survey of all community partners (including non-funded courses) was adapted from one in Assessing Service-Learning and Civic Engagement: Principles and Techniques (Campus Compact 2001). Staff used the survey results to Provide individual faculty members with partner results so that they can determine the effectiveness of their service-learning relationships with community organizations Determine how the program can better serve community partners and promote student learning Improve the way LSU shares its resources with the community Enhance the instructional role of community partners
Demographics: The community partners who responded to the survey include a cross section of the organizations or agencies that service-learning classes work with in East Baton Rouge Parish. All partner organizations are non-profit, including 85 %, which are public organizations, and 15 %,
6
which are private. The organizations responding self-identified as education (46%), safety (18%), health (12 %), environment (9%), housing (9 %), and public services (6 %).
7
Results: Forty-nine percent of the community partners responded to the survey. Their level of satisfaction with the program is indicated by the following results: 100% indicated that they would continue working with the university in this or another activity. 94 % were satisfied with their overall communication with students and faculty. 87% were satisfied with the level and quality of interaction with students and faculty. 94% were satisfied with the quality of student work. 71% were satisfied with the feedback and input into planning of experiences. 88% were satisfied with the scope and timing of the activity. 94% were satisfied with their level of trust with faculty and students.
In responding to their experience with our university, community partners responded positively, as is indicated in answers to the following questions. Please note that some respondents checked more than one choice and items that resulted in less than 10 % responses are not included. How did your interactions with the university influence your capacity to fulfill the mission of your organization? Enhanced offerings of services (22 %) Increased number of clients served (16 %) Increased leverage/networks with other community groups (13 %) No influence (13 %)
What were some of the economic effects of your work with the university? Identification of additional volunteers (27 %) Increased value of services (14 %) Increased organizational resources (14 %) Completion of projects (11 %)
In what ways do you believe that you are able to influence as a result of your connection with one of our courses? Influence on student learning experience (65 %) Influence on faculty awareness of community (23 %)
8
As a result of your connection to this university course, how has your awareness of the university changed? I learned more about university programs and services (35 %) I know whom to call upon for information and assistance (24 %) I have an increased knowledge of university resources (17 %) I have more interactions with faculty and administrators (14 %)
Conclusions: While the results were overwhelmingly positive, there are two areas that the program and individual professors, who were given copies of the results from the agencies with which they worked, can work to improve—the level and quality of interaction with students and faculty and the feedback and input into planning of experiences. Since these two areas are closely connected, improving the preplanning relationship with the partner should result in improved satisfaction with the quality of interaction of community partners with students and faculty.
Recommendations: Faculty and community partner training and literature should continue to emphasize the importance of developing strong, sustainable partnerships. Resources should be allocated to provide support for orientation, facilitating, and monitoring partnerships.
9