Monitoring and evaluation by ewghwehws

VIEWS: 6 PAGES: 32

									MONITORING AND EVALUATION
OUR APPROACH
1.   What is monitoring and evaluation? Conceptual
     differences and terminologies.
2.   Approaches to Monitoring and Evaluation.
3.   Establishing M&E System.
4.   How to do Monitoring and Evaluation.
WHAT IS MONITORING?
   Day-to-day follow up of activities during implementation
    to measure progress and identify deviations
   Routine follow up to ensure activities are proceeding as
    planned and are on schedule
   Routine assessment of activities and results
   Answers the question, “what are we doing?”
WHY TO       MONITOR ACTIVITIES?

   Tracks inputs and outputs and compares them to plan
   Identifies and addresses problems
   Ensures effective use of resources
   Ensures quality and learning to improve activities and
    services
   Strengthens accountability
   Program management tool
WHAT IS EVALUATION?
 It is a time-bound exercise that attempts to assess
  systematically and objectively the relevance, performance
  and success of ongoing and completed programmes and
  projects.
 Designed specifically with intention to attribute changes to
  intervention itself
 Answers the question, “what have we achieved and what
   impact have we made”
 Evaluation commonly aims to determine the
  relevance, efficiency, effectiveness, impact and
  sustainability of a programme or project.
 Relevance: The degree to which the outputs,
  outcomes or goals of a programme remain valid
  and pertinent as originally planned or as subsequently
  modified owing to changing circumstances within the
  immediate context and external environment of that
  programme.
 Efficiency: A measure of how economically or
  optimally inputs (financial, human, technical
  and material resources) are used to produce outputs.
 Effectiveness: A measure of the extent to which a
  programme achieves its planned results
  (outputs, outcomes and goals).
   Impact: Positive and negative long term effects
    on identifiable population groups produced by
    a development intervention, directly or indirectly,
    intended or unintended. These effects can be
    economic, socio-cultural, institutional, environmental,
    technological or of other types.
   Sustainability: Durability of programme results
    after the termination of the technical
    cooperation channelled through the programme.
   Static sustainability – the continuous flow of the
    same benefits, set in motion by the completed
    programme, to the same target groups;
   Dynamic sustainability – the use or adaptation of
    programme results to a different context or changing
    environment by the original target groups and/or
    other groups.
WHY EVALUATE            ACTIVITIES


   Determines program effectiveness
   Shows impact
   Strengthens financial responsibility and accountability
   Promotes a learning culture focused on service
    improvement
   Promotes replication of successful interventions.
TYPES OF EVALUATION
   Ex-ante Evaluation: An evaluation that is performed
    before implementation of a development.
    intervention. Related term: appraisal.
   Ex-post Evaluation: A type of summative evaluation
    of an intervention usually conducted after it has
    been completed.
   External Evaluation: An evaluation conducted by
    individuals or entities free of control by those
    responsible for the design and implementation of the
    development intervention to be evaluated (synonym:
    independent evaluation).
   Internal Evaluation: Evaluation of a development
    intervention conducted by a unit and /or
    individual/s reporting to the donor, partner, or
    implementing organization for the intervention.
 Formative Evaluation: A type of process
  evaluation undertaken during programme
  implementation to furnish information that
  will guide programme improvement.
 Impact Evaluation: A type of outcome
  evaluation that focuses on the broad,
  longer-term impact or results of a programme.
 Joint Evaluation: An evaluation conducted
  with other partners, bilateral donors or
  international development banks.
 Meta-evaluation: A type of evaluation that
  aggregates findings from a series of
  evaluations.
 Process Evaluation: A type of evaluation that
  examines the extent to which a programme is
  operating as intended by assessing ongoing
  programme operations. A process evaluation helps
  programme managers identify what changes are needed
  in design, strategies and operations to improve
  performance.
 Qualitative Evaluation: A type of evaluation that
  is primarily descriptive and interpretative, and
  may or may not lend itself to quantification.
 Quantitative Evaluation: A type of evaluation
  involving the use of numerical measurement and
  data analysis based on statistical methods.
 Summative Evaluation: A type of outcome and
  impact evaluation that assesses the overall
  effectiveness of a programme.
 Thematic Evaluation: Evaluation of selected
  aspects or cross-cutting issues in different types
  of interventions.
CONFUSING TERMS
 Audit
 Appraisal

 Inspection
Approach          Major Focus         Typical Question            Likely Methodology


Goal Based        Assessing           Were the goals              Comparing baseline and progress
(Strategic        achievement of      achieved? Efficiently?      data; finding ways to measure
Approach)         goals               Were they the right         indicators.
                  and objectives.     goals?



Decision Making   Provide             Is the project effective?   Assessing range of options related to
(System           Information         Should it continue?         the project context, inputs, process, and
Approach)                             How might it be             product. Establishing some kind
                                      modified?                   of decision-making consensus.

Goal Free         Assessing full      What are all the            Independent determination of needs
(Inductive        range of project    outcomes? What value        and standards to judge project worth.
Approach)         impacts, intended   do they have?               Qualitative and quantitative
                  and not intended                                techniques to uncover any
                                                                  possible results.


Expert            Use Expertise       How does an outside         Critical review based on experience,
                                      professional rate this      informal surveying, and subjective
                                      project?                    insights.



Participatory     Stakeholder         How do the                  Participatory Workshops.
Approach          Satisfaction        stakeholders rate the
                                      project.
    M&E TOOLS
 Evaluating programme strategy and direction: Log-
  frames, Stakeholder Analysis
 Evaluating programme management: Horizontal
  Evaluation; Appreciative Inquiry
 Evaluating programme outputs: Evaluating academic
  articles and research reports; Evaluating websites; After
  Action Reviews
 Evaluating outcomes and impacts: Outcome Mapping,
  Most Significant Change; Episode Studies.
M&E TOOLS
Following is a non – exhaustive list of M&E Tools:
1. Performance indicators

2. Formal surveys

3. Rapid appraisal methods

4. Participatory methods

5. Cost-benefit and cost-effectiveness analysis

6. Impact evaluation
     1. PERFORMANCE      INDICATORS
 Performance indicators are measures of inputs, processes,
  outputs, outcomes, and impacts for development projects,
  programs, or strategies.
 Uses:
       Setting performance targets and assessing progress toward
        achieving them.
       Identifying problems via an early warning system to allow
        corrective action to be taken.
   Problems:
       Poorly defined indicators are not good measures of success.
       Tendency to define too many indicators, or those without accessible
        data sources,
       Often a trade-off between picking the optimal or desired indicators
        and having to accept the indicators which can be measured using
     HOW TO MAKE INDICATORS
1.   Identify the problem situation you are trying to address.
2.   Develop a vision for how you would like the problem areas
     to be/look. This will give you impact indicators.
3.   Develop a process vision for how you want things to be
     achieved. This will give you process indicators.
4.   Develop indicators for effectiveness.
5.   Develop indicators for efficiency .
    2- FORMAL SURVEYS
 Formal surveys can be used to collect standardized
  information from a carefully selected sample of people or
  households.
 Uses:
       Providing baseline data against which the performance of the
        strategy, program, or project can be compared.
       Comparing different groups at a given point in time.
       Comparing changes over time in the same group.
       Comparing actual conditions with the targets established in a
        program or project design.
     2- FORMAL SURVEYS
   Problems:
       With the exception of CWIQ, results are often not available for a
        long period of time.
       The processing and analysis of data can be a major bottleneck for
        the larger surveys even where computers are available.
       LSMS & household surveys are expensive & time-consuming.
       Many kinds of information are difficult to obtain through formal
        interviews.
     DIFFERENT TYPES OF SURVEY
1.   Multi-Topic Household Survey (also known as Living
     Standards Measurement Survey—LSMS).
2.   Core Welfare Indicators Questionnaire (CWIQ).
3.   Client Satisfaction (or Service Delivery) Survey.
4.   Citizen Report Card.
    3- RAPID APPRAISAL             METHODS

 Rapid appraisal methods are quick, low-cost ways to gather
  the views and feedback of beneficiaries and other
  stakeholders, in order to respond to decision-makers’ needs
  for information.
 Uses:
       Providing rapid information for management decision-making,
        especially at the project or program level.
       Providing qualitative understanding of complex socioeconomic
        changes, highly interactive social situations, or people’s values,
        motivations, and reactions.
       Providing context and interpretation for quantitative data collected
        by more formal methods.
3- RAPID APPRAISAL              METHODS

   Problems:
       Findings usually relate to specific communities or localities—
        thus difficult to generalize from findings.
       Less valid, reliable, and credible than formal surveys.
4- RAPID APPRAISAL    METHODS

1.   Key informant interview
2.   Community group interview
3.   Focus group discussion
4.   Direct Observation
5.   Mini surveys
4- PARTICIPATORY             METHODS

 Participatory methods provide active involvement in
  decision-making for those with a stake in a project,
  program, or strategy and generate a sense of ownership in
  the M&E results and recommendations.
 Uses:
       Learning about local conditions and local people’s perspectives
        and priorities to design more responsive and sustainable
        interventions.
        Evaluating a project, program, or policy.
       Providing knowledge and skills to empower poor people.
4- PARTICIPATORY             METHODS

   Problems:
       Sometimes regarded as less objective.
       Time-consuming if key stakeholders are involved in a
        meaningful way.
       Potential for domination and misuse by some stakeholders to
        further their own interests.
4- PARTICIPATORY     METHODS

1.   Participatory rural appraisal
2.   Participatory monitoring and evaluation
5- COST-BENEFIT & COST-EFFECTIVENESS                          ANALYSIS

 Cost-benefit and cost-effectiveness analysis are tools for
  assessing whether or not the costs of an activity can be
  justified by the outcomes and impacts. Cost-benefit
  analysis measures both inputs and outputs in monetary
  terms. Cost-effectiveness analysis estimates inputs in
  monetary terms and outcomes in non-monetary quantitative
  terms.
 Uses:
       Informing decisions about the most efficient allocation of resources.
       Identifying projects that offer the highest rate of return on
        investment.
5- COST-BENEFIT & COST-EFFECTIVENESS                          ANALYSIS

   Problems:
       Fairly technical, requiring adequate financial and human
        resources available.
       Requisite data for cost-benefit calculations may not be available,
        and projected results may be highly dependent on assumptions
        made.
       Results must be interpreted with care, particularly in projects
        where benefits are difficult to quantify.
6 – IMPACT EVALUATION
 Impact evaluation is the systematic identification of the
  effects – positive or negative, intended or not – on individual
  households, institutions, and the environment caused by a
  given development activity such as a program or project.
 Uses:
       Measuring outcomes and impacts of an activity and distinguishing
        these from the influence of other, external factors.
       Helping to clarify whether costs for an activity are justified.
       Informing decisions on whether to expand, modify or eliminate
        projects, programs or policies.
    Problems:
1.    Some approaches are very expensive and time-consuming
2.    Reduced utility when decision-makers need information
      quickly.
3.    Difficulties in identifying an appropriate counter-factual.

								
To top