Program evaluation is…

Document Sample
Program evaluation is… Powered By Docstoc
					 INTRODUCTION TO
UTILIZATION FOCUSED
     EVALUATION
      12 STEPS
    Evaluation Conclave
      October , 2010

     Sonal Zaveri, PhD
     IDRC Consultant
                     Outline

1. Why is evaluation use important?

2. UFE      –           What    is     it?

3. How          is         it   different?

4. Understanding the 12 steps
 Evaluation Use – Why is it important?

         Many evaluations land on
           dusty bookshelves
BUT…..
• Monies have been spent $$$$$
• Do not know what is effective or not
SO …
How do we do evaluations that can be
  used?
           What is evaluation?
         What do we want to know?
•   What is the worth, value, significance?
•   How effective was program?
•   Was it implemented as planned?
•   Were goals achieved?
•   Did participants benefit?
•   Any unexpected consequences ?
•   What worked? How to improve?
•   Do next?
  3 Questions for evaluation
WHAT?                What to know?

                     What to act on?
SO WHAT?

                     How to use findings?
NOW WHAT?

  Challenge is not getting information,
            it is USING it!
Utilization focused evaluation is…



 A decision-making framework for
  enhancing the utility and actual
   use of evaluations. (Patton, 2008a: slide 9)
                         Premises of UFE

1. Evaluations should be JUDGED by their utility and ACTUAL USE.

2. No evaluation should go forward unless and until there are primary intended
   users who will use the information that can be produced.

1. Primary intended users are involved in the process - commitment.

2. Evaluation is part of initial program/evaluation design. The primary
   intended users want information to help answer a question or questions.

3. Evaluator’s role is to help intended users clarify their purpose and
   objectives.

4. Implications for use are part of every decision throughout the evaluation –
   it is the driving force of the evaluation process. (Patton, 2008a)
We don’t want to end up in this situation,
  so…
                   UFE in 12 steps
1.    Project / network readiness assessment.
2.    Evaluator readiness and capability assessment.
3.    Identification of primary intended users.
4.    Situational analysis.
5.    Identification of primary intended uses.
6.    Focusing on evaluation.
7.    Evaluation design.
8.    Simulation of use.
9.    Data collection.
10.   Data analysis.
11.   Facilitate use.
12.   Meta-evaluation.
       Project and Evaluator Readiness
                 Steps 1 & 2
   Are all involved parties supportive of adopting
    a UFE approach? Are key persons interested
    in UFE?

   Are resources available?

   Can the evaluator be a facilitator? Is the
    evaluator willing to assess effectiveness of
    evaluation according to how intended users
    use the findings?
                 Users and Uses
                 Steps 3, 4, & 5
   Identifying Users – who has high power and
    high interest in the findings?

   Situational Assessment – what are the
    barriers and opportunities in the evaluation
    use?

   Identifying Use – choosing what to evaluate
                 A challenge !
            Focus and Design of Evaluation
                     Steps 6 & 7
       What are the KEQ – key evaluation questions
        that will best provide findings for use?

       Which evaluation design is best? Practical?

        What evaluation methods should be used?

        The primary users are involved in the decision
                          making!
            Simulating use …Step 8
   Before collecting data - prepare possible
    ‘positive and negative’ findings

   Ask – what do these results mean? How it will
    affect use?

   Primary users get a chance to check
    expectations against the ‘real world’ and
    review KEQ and focus.
               A unique UFE step!
           Data Collection & Analysis
                 Steps 9 & 10


   Primary users are involved in data collection
    and know progress

   Users are involved actively in analysis

   How can findings be used for use?
              Ultimate Test is USE
                    Step 11


   How does the primary user use the findings?

   Disseminate according to plan but also look at
    other opportunities

   Have you found unexpected results – how will
    you use them?
              Evaluating the UFE
                   Step 12


   Follow up – how was intended use by user
    achieved?
   Did additional users/uses emerge?
   Reflect – what actually happened?

       UFE IS A LEARNING PLATFORM –
         TO DO BETTER NEXT TIME
                   UFE in 12 steps
1.    Project / network readiness assessment.
2.    Evaluator readiness and capability assessment.
3.    Identification of primary intended users.
4.    Situational analysis.
5.    Identification of primary intended uses.
6.    Focusing on evaluation.
7.    Evaluation design.
8.    Simulation of use.
9.    Data collection.
10.   Data analysis.
11.   Facilitate use.
12.   Meta-evaluation.
    THANK YOU!


    Contact details
sonalzaveri@gmail.com

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:10/26/2011
language:English
pages:18