Designing and Implementing Program and Project Evaluations by djz44927

VIEWS: 8 PAGES: 18

									  Designing and Implementing
Program and Project Evaluations

        DONALD YARBROUGH, PH.D.
    MELISSA CHAPMAN, PH.D. CANDIDATE

U.I. CENTER FOR EVALUATION AND ASSESSMENT
  HTTP://WWW.EDUCATION.UIOWA.EDU/CEA/
            Importance of Evaluation

 Evaluation can inform programs, projects, and
  policies
 “…evaluation consists of a set of activities developed
  to help correct, support, and extend the way that
  people, individually and collectively, naturally make
  sense of policies and programs implemented to meet
  human needs”
           Program Evaluation Guidelines

 Guiding Principles for Evaluators
  (American Evaluation Association, http://www.eval.org/)

    Systematic inquiry

    Competence

    Integrity/honesty

    Respect for people

    Responsibilities for General and Public Welfare
          Program Evaluation Guidelines

 Program Evaluation Standards
 (Joint Committee on Standards for Educational Evaluations, 1994,
 http://www.wmich.edu/evalctr/jc)
    Feasibility
    Propriety
    Accuracy
    Utility
    Metaevaluation
      Purposes of Evaluation: General

 Formative


 Summative
       Purposes of Evaluation: Specific

 Description and Monitoring
 Investigate Causality
 Determining merit and/or worth
 Improving organizational decisions (ex –
  accountability)
 Contributing to knowledge
      Purposes of Evaluation: Questions

 Context and problems


 Theories and beliefs
      Purposes of Evaluation: Questions

 Intended clients, targets, learners, participants,
 beneficiaries

 Proposed Goals


 Proposed Solutions, interventions, activities
      Purposes of Evaluation: Questions

 Resources
   Proposed

   Used



 Actual activities during implementation
           Purposes of Evaluation: Questions

 Theories
   Intervention theories and beliefs
         How should these activities result in the proposed changes or
          improvements?


     Management and Service delivery theories and beliefs
      Purposes of Evaluation: Questions

 Products of the program or project
   Materials

   Participation

   Processes

   Etc.



 Outcomes, impacts, efficiencies, effectiveness
 achieved
Scheduling and Staffing
Uses and Reporting
           Supporting Documentation

 Tables
 References
 Vita and resumes
 Budget
 Prior support
 Resources (transcription, technological, physical
 space)
                          Expanded Model

Prior Situation & Context Focus for the Program/Project              

1. Context &                2. Purposes that the P/P     3. Targeted Users of the
Environment of the          might serve, including       P/P, Including Process and
program/project             Needs, Problems, Areas       Instrumental Users, and
                            for Growth                   Other Beneficiaries,
                                                         Including Staff

4. Diagnostic Theory (part of the inclusive Program Theory) informing how the
problem and needs that the P/P will address are determined. This theory resides
in/with the P/P designers, staff & stakeholders and is based on scholarship, practice
wisdom & beliefs (see also Causal Hypothesis)
                                Expanded Model

Intervention Focus               
                   Planning                                           Implementation
5. Solutions     6. Process      7. Outcome     8. Resources   9. Activities, 10. Outputs
& Strategies     Goals &         and Impact     & Inputs       Methods & of the P/P
For the P/P to   Objectives      Goals &        Actually       Procedures
Implement        for Planning    Objectives     Used in the
                                                               Actually
                 the P/P         for Planning   P/P
                                                               Used in the
                                 the P/P
                                                               P/P

11. Program theory informing the P/P impact model (Impact Theory & Intervention
Hypothesis). Based on scholarship, practice wisdom & beliefs

12. Program theory informing the evaluation P/P service delivery, administration &
management (Process theory, organizational and service delivery plans). Based on
scholarship, practice wisdom & beliefs
                         Expanded Model

Post Implementation Situation & Results Focus             


13. Post P/P        14. Outcomes of      15. Impacts of the   16. Costs &
Context &           the P/P              P/P                  Efficiencies. What
Environment for                                               are they and how
the evaluation.                                               could they be best
How has it                                                    managed?
changed?

17. Program theory explaining causal conclusions required in arguing that the P/P
did or did not meet its purposes as they are understood at the end of the P/P.
(Impact theory and intervention hypothesis). Based on scholarship, practice
wisdom & beliefs
Questions?

								
To top