Designing and Implementing Program and Project Evaluations DONALD YARBROUGH, PH.D. MELISSA CHAPMAN, PH.D. CANDIDATE U.I. CENTER FOR EVALUATION AND ASSESSMENT HTTP://WWW.EDUCATION.UIOWA.EDU/CEA/ Importance of Evaluation Evaluation can inform programs, projects, and policies “…evaluation consists of a set of activities developed to help correct, support, and extend the way that people, individually and collectively, naturally make sense of policies and programs implemented to meet human needs” Program Evaluation Guidelines Guiding Principles for Evaluators (American Evaluation Association, http://www.eval.org/) Systematic inquiry Competence Integrity/honesty Respect for people Responsibilities for General and Public Welfare Program Evaluation Guidelines Program Evaluation Standards (Joint Committee on Standards for Educational Evaluations, 1994, http://www.wmich.edu/evalctr/jc) Feasibility Propriety Accuracy Utility Metaevaluation Purposes of Evaluation: General Formative Summative Purposes of Evaluation: Specific Description and Monitoring Investigate Causality Determining merit and/or worth Improving organizational decisions (ex – accountability) Contributing to knowledge Purposes of Evaluation: Questions Context and problems Theories and beliefs Purposes of Evaluation: Questions Intended clients, targets, learners, participants, beneficiaries Proposed Goals Proposed Solutions, interventions, activities Purposes of Evaluation: Questions Resources Proposed Used Actual activities during implementation Purposes of Evaluation: Questions Theories Intervention theories and beliefs How should these activities result in the proposed changes or improvements? Management and Service delivery theories and beliefs Purposes of Evaluation: Questions Products of the program or project Materials Participation Processes Etc. Outcomes, impacts, efficiencies, effectiveness achieved Scheduling and Staffing Uses and Reporting Supporting Documentation Tables References Vita and resumes Budget Prior support Resources (transcription, technological, physical space) Expanded Model Prior Situation & Context Focus for the Program/Project 1. Context & 2. Purposes that the P/P 3. Targeted Users of the Environment of the might serve, including P/P, Including Process and program/project Needs, Problems, Areas Instrumental Users, and for Growth Other Beneficiaries, Including Staff 4. Diagnostic Theory (part of the inclusive Program Theory) informing how the problem and needs that the P/P will address are determined. This theory resides in/with the P/P designers, staff & stakeholders and is based on scholarship, practice wisdom & beliefs (see also Causal Hypothesis) Expanded Model Intervention Focus Planning Implementation 5. Solutions 6. Process 7. Outcome 8. Resources 9. Activities, 10. Outputs & Strategies Goals & and Impact & Inputs Methods & of the P/P For the P/P to Objectives Goals & Actually Procedures Implement for Planning Objectives Used in the Actually the P/P for Planning P/P Used in the the P/P P/P 11. Program theory informing the P/P impact model (Impact Theory & Intervention Hypothesis). Based on scholarship, practice wisdom & beliefs 12. Program theory informing the evaluation P/P service delivery, administration & management (Process theory, organizational and service delivery plans). Based on scholarship, practice wisdom & beliefs Expanded Model Post Implementation Situation & Results Focus 13. Post P/P 14. Outcomes of 15. Impacts of the 16. Costs & Context & the P/P P/P Efficiencies. What Environment for are they and how the evaluation. could they be best How has it managed? changed? 17. Program theory explaining causal conclusions required in arguing that the P/P did or did not meet its purposes as they are understood at the end of the P/P. (Impact theory and intervention hypothesis). Based on scholarship, practice wisdom & beliefs Questions?
Pages to are hidden for
"Designing and Implementing Program and Project Evaluations"Please download to view full document