Training course on Participatory Monitoring and Evaluation (PME) - PDF - PDF

Document Sample
Training course on Participatory Monitoring and Evaluation (PME) - PDF - PDF Powered By Docstoc
					                             Report on


  Training course on Participatory Monitoring and
                Evaluation (PM&E)




                 Conducted from 21 - 25 August, 2006
                    At ELD, Jawalakhel, Lalitpur


The Empowerment and Learning Development Centre (ELD)


                           www.eld.org.uk
Introduction
This report highlights the key achievements and documents the process of a training course on
Participatory Monitoring and Evaluation (PM&E). The Empowerment and Learning Development
Centre (ELD) organised the five-day training from August 21 - 25, 2006 at the ELD training centre,
Jawalakhel, Lalitpur.

The objectives of the training were to:

•   Develop clarity on various concepts related to PM&E
•   Equip participants with a range of hands-on tools for facilitating PM&E
•   Enable participants to facilitate PM&E in their own
    respective working areas

Uttam Uprety led the course as main facilitator with two
sessions being led by an external expert. Altogether there
were seven participants representing:

•   ADRA
•   ICIMOD
•   International Nepal Fellowship
•   SAP Nepal
•   Search for Common Ground
•   The Mountain Institute

While preparing this report, the facilitator has included the responses of participants from daily
feedback and final training evaluation results, as well as personal observation.

This report is organised as follows:

    •   Conclusions
    •   Training Evaluation
    •   Daily Narrative
    •   Annexes
            o Workshop Schedule
            o Workshop Products




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   -2-
                                                www.eld.org.uk
Conclusions
The training was successful for several reasons. Firstly the participants, by the end of the course,
had acquired conceptual clarity of various aspects of Monitoring and Evaluation including
terminologies as used by various development organisations. Furthermore, they got basic
knowledge of Social Auditing - a recent concept in development management. They learned
various hands-on tools to facilitate Participatory Monitoring and Evaluation, ranging from PRA, AIA
and TOP (Technology of Participation) to ZOPP. Finally, sufficient opportunities for practicing the
tools enabled them to reflect on their working context and plan how to use such tools in their
respective fields.

The training was responsive and up-to-date with new concepts and tools. The facilitator also
customised the course and included the contents to address participants’ individual expectations.
The course was adapted in places to meet the multi-cultural environment.

Participatory discussion during the workshop widened the participants' horizons. They agreed to
use these tools as their situation demands in planning, Monitoring and Evaluation. From the first
day, participants were actively participating in group interaction. They raised practical concerns and
brought their field experiences into the discussions. The facilitator observed that participants
wanted to reframe every session from the course in their own context.

The training maintained a balance between theory and practical sessions. However, except for
some tools (as participants wanted to know about more tools during the course), the emphasis was
on practical applications rather than theory. Participants enjoyed the sessions very much, leading
to increased confidence in applying such tools. The final day’s evaluation indicated that they had
identified areas to apply such tools not only in PM&E, but also in planning, review, and other
activities.

Participants praised the training team for managing the workshop effectively. They appreciated the
facilitators’ knowledge and skills on the subject matter as well as the training management. Enough
reference materials were provided to the participants for each session. Facilitators provided digital
copies of slides developed during the training together with other reference materials on CD.




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   -3-
                                                www.eld.org.uk
Training Evaluation
The training was evaluated using a Rapid Rural Evaluation scattergram. From the results we can
see that, overall, participants were extremely happy with all aspects of the course.

Criteria
                            -20              -10               0                +10               +20
Objectives met?                                                                 ☺☺☺               ☺☺☺☺
Presenter’s skills?                                                             ☺☺                ☺☺☺☺
                                                                                                  ☺
Her/his knowledge of                                                            ☺                 ☺☺☺☺
topic?                                                                                            ☺☺
Logistics                                                      ☺                                  ☺☺☺☺
                                                                                                  ☺☺
Your Skills                                                                     ☺☺☺               ☺☺☺☺
improved?
Confidence gained?                                                              ☺☺☺☺              ☺☺
                                                                                ☺
Worth coming?                                                                   ☺☺☺               ☺☺☺☺

Individual comments collected included:

Positive

•   Overall, I enjoyed the course very much and also appreciated the facilitator’s skills and
    competence. It would be even better if the application (practical) of the tool was given after a
    very brief description of the tool followed by the presentation on its theory.
•   It was an excellent training for me, especially as I was never exposed to these tools and have
    the responsibility to develop a monitoring system for my organisation.
•   Use of LCD, marker and cards, white boards simultaneously helped to fresh participants mood
•   I will try to participate more in such kinds of training

Possibilities for Improvement

•   It would have been more useful beneficial for me if the course was delivered in only Nepali
•   More exercise on indicators developments needed
•   Such kind of training should be residential




       Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006    -4-
                                               www.eld.org.uk
Narrative

Day One

Day One was useful for the participants in                                        Your Expectations…
helping them understand:
                                                                                              Competency in using PM&E tools
                                                             Skills
•   Various terminologies related to project                                                  Develop a PM&E system for radio program and
                                                                                             community peace building activities
    Monitoring and Evaluation                                                                 Proficiency in M&E methodologies

•   How participatory M&E differs with
    conventional M&E in various aspects.
                                                                                              Principles, norms of PM&E
                                                             Knowledge
                                                                                              PM&E tools
The training began half an hour later than the                                                To get the clear vision of PM&E system to run the
                                                                                             project smoothly
scheduled time as three participants were late.
As a result, we had to drop some of the slides
from the presentation. Regarding Monitoring and
Evaluation, the session duration was bit short               ELD training on Participatory Monitoring and Evaluation                              1


considering the number of slides to cover.

Sharing expectations of both the participants and
their line managers made them reflect on how                     Line Manager’s Expectations..
their line managers perceived the importance of
                                                                  Knowledge on overall M&E system to bring greater impact on
the training.                                                     project’
                                                                  project’s in coming days
                                                                  Skills to develop the M&E tools and apply in the projects
While discussing the ‘what and why’ of                                                                                           in
                                                                  Skill to facilitate and use effective participatory approaches in the
                                                                  project
participation, Uttam shared on the group                          Learn about the theory and practice of PM&E
dynamics of decision making. Key points from                      Learn best practices

the discussion were:                                              Learn effective ways of implementing a monitoring system
                                                                  Train staff and community focal points
                                                                  Learn about project M&E process
•   Generally when a group starts a discussion,                                                                                    retreat
                                                                  Facilitate group meetings for Mountain Forum Staff and staff retreat

    we expect participants to share their
    ideas/opinions in a way without confrontation            ELD training on Participatory Monitoring and Evaluation                              6

    and reach a conclusion – however, this
    rarely happens.
•   When opinion is so diverse as to make it difficult to come to closure or premature decisions are
    both undesirable.
•   Participation is ensured only if the facilitator is skilled and knows the group dynamics well.




       Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006                              -5-
                                               www.eld.org.uk
         Group Dynamics in Decision Making




                                                        Groan Zone
                                                        Groan Zone
                                                                     Shared Frame       Decision
    topic




    ELD training on Participatory Monitoring and Evaluation                                    13




Discussing on Monitoring and Evaluation, the facilitator led a discussion and later summarised the
discussion to agree a definition. The group identified some of the criteria to distinguish Monitoring
and Evaluation as:

    •     Intention
    •     Time/ frequency
    •     Scope
    •     Involvement of project/program staff and community

Day-end Reflection

Happy about …                                                                Concerned about …
• Good content of the program                                                • Program started late
• Good team                                                                  • Did we go off track as we wanted to be
• Everything nice                                                               more participatory?
• Interesting material                                                       • Poor time management-should stick to
• Format of course                                                              planned schedule
• Group dynamics in decision making                                          • Appreciative Inquiry approach
• Key aspects of M&E                                                         • More talking on English
• Introduction to M&E still to be covered more
• Philosophy of participation
• Introduction of M&E
• Energisers
• Lunch and refreshments
• Time and topics covered




         Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006                   -6-
                                                 www.eld.org.uk
Day Two
Following a review of the first day, the facilitator gave a brief overview of various participatory tools
to facilitate PM&E. He highlighted the uniqueness and use of tools from PRA, Appreciative Inquiry
Approach (AIA), LFA, and other approaches that can be used in facilitating various stages of
participatory processes.




The overview of various participatory tools made participants aware of the varieties of tools
available. The tools listed are not even comprehensive, as different organisations are using
different sets of tools in PM&E.




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   -7-
                                                www.eld.org.uk
The group then discussed and shared their experiences with Social/Resource Mapping and key
issues discussed were:

    •   Handing over the stick is key for empowerment
    •   Questioning skill is best tool for the facilitator
    •   Identifying key informants is very important aspect in any facilitation
    •   What is the difference between making a map on the spot and bringing key informants
        somewhere else?
    •   Date, key informants and facilitators are to be included in any map
    •   Local symbols are very important
    •   Social/Resource maps can be used for PM&E to make the changes visual.
    •   The tool by itself doesn't bring lot of information; it should be used as a springboard for
        discussion

The next tools discussed were the Venn diagram and Communication Linkage Matrix. The
facilitator introduced the Venn diagram as a PM&E tool. Participants discussed a lot on the size of
the circles and relative distances. Unlike their previous concept of using this tool in organisational
assessment, they realized that it could also be used in assessing satisfaction with program / project
components.

The tool was followed by the Communication Linkage Matrix. The facilitator explained how this tool
can be used in various situations. Participants appreciated how the Communication Linkage Matrix
could be used in triangulating the Venn diagram tool in a participatory way.




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   -8-
                                                www.eld.org.uk
In response to the issues raised by participants such as ‘How to develop criteria for matrix?’ and
‘Are there any universal criteria?’, the facilitator clarified that people’s responses during Venn
diagramming regarding the size and distance of the circles should be taken as criteria for
communication linkage matrix. Further, he emphasised the need of setting criteria common for all
components under evaluation.

The session on Discussion method was also highly interactive. We discussed on and concluded
that common problems include:

    •   Not going to the deep level interaction:
        only superficial level
    •   Going off-track
    •   Drop out
    •   Know-it-all types people
    •   Discussion for the sake of discussion
    •   Not coming to the closure zone
    •   No fresh ideas
    •   Time management

The group further agreed that:

    •   The facilitator has to make sure that these qualities are maintained in any discussion
    •   S/he needs to use facilitative listening skills to make sure that everybody gets the point in
        the way the presenter intended it
    •   There needs to be a structured method of discussion.

Day-end Reflection

Happy about …                                            Concerned about …
• Good tools shared                                      • How am I going to use them in my project?
• Good knowledge gained                                  • Not confident at skills level
• All good today only some more translation              • Not very clear about social/resource
   needed                                                   mapping and its use in M&E of project
• ORID methods                                           • How to retain all the information provided
• Organisation of the presentation                          (lot of concepts)
• Venn diagram
• All topics covered and social map
• Communication linkage matrix
• Time management
• Food




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   -9-
                                                www.eld.org.uk
Day Three
Participants were happy with the review session,                                 Environmental Scanning
even though it was very extensive and took more                                         Method
time than scheduled. After the review, the facilitator            Influence Time                        1995                         2000              2005

highlighted the art of designing questions for the 4-                                                                                   acity
                                                                                                                                     Cap ing
level discussion method known as ORID. Participants                  National
                                                                                                        dation
                                                                                                                                      Build
                                                                                                                                                                ram
                                                                                                                                                           Prog ion
                                                                                                   Foun                                                        ns
had the opportunity to identify the questions for the                                       What do we see
                                                                                            happening most
                                                                                                                             How can we name
                                                                                                                                                           Exte


four levels from scramble cards that gave them
                                                                     Regional                                                  this period?
                                                                                             in this period?


confidence in setting questions.                                      District



The next activity was discussion design. Participants
were grouped and given an assignment to design                     ELD training on Participatory Monitoring and Evaluation                                       68




questions to facilitate the discussion. One
representative from each group facilitated the                           Workshop Method: Overview
discussion. Participants and the facilitator gave                                                         Focus Question of the Workshop               ?

feedback on both the questions themselves as well
as individuals’ facilitation styles.

Due to time constraints, only the theoretical aspects
of Environmental Scanning were covered on Day
Two, with the practical part rescheduled for the final                                                                           A              B      C


day.
                                                                                                                                 B
                                                                                                                                            C
The practical session on Workshop Method met with                  ELD training on Participatory Monitoring and Evaluation
                                                                                                                                 A
                                                                                                                                                                65

good response from participants. The workshop
product is annexed.


Day-end Reflection

Happy about …                                            Concerned about …
• Workshop practice                                      • Some participants arriving late and leaving
• Discussion practice                                       early
• ORID discussion method and practical




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006                                                - 10 -
                                                www.eld.org.uk
Day Four
The first session explored the tool the Empowerment Circle, which was new for participants.
Discussion on this tool was useful, and participants shared a lot of their experiences.

While discussing the Empowerment Circle, the
facilitator highlighted its uses as:                                      Hierarchy of              Types of Indicators        Indicators
                                                                          Objectives

•   Identifying local criteria for any changes - not                      Goal                      Impact
    only results
                                                                                                    Sustainability
•   Assessing changes from both a personal and
    group perspective                                                     Objectives                Effect
•   Triangulating the evaluation results from other                                                 Usages

    tools                                                                                           Productivity

•   Making the evaluation aspects visual and                              Results                   Availability
    simplifying terms like empowerment into various                                                 Coverage

    specific components
                                                                    ELD training on Participatory Monitoring and Evaluation                                     164

The next session looked at the Ranking Method. Two
types of ranking - matrix and pair-wise – were
discussed.                                                              Result Chain: Exercise
The post-lunch session was on LFA in Project                    Society with fewer drug
                                                                   Related problem
                                                                                                                     IMPACT

management. As different organisations use different
terminologies, participants had some difficulty at first,                                                          OUTCOME               Increase awareness
                                                                                                                                          Reduction of drug

but the facilitator clarified some of the terminologies          Mobile team activities,
and their similarities and differences.                           Radio broadcasting                               OUTPUTS



                                                                                                                  ACTIVITIES        Mobile team in school formed and trained

The exercise on Results Chain was highly                                                                                                  Radio training and production



appreciated by the participants. What the group            Funds, staff, stakeholders
                                                                                      INPUTS
                                                              Radio equipment
realised was unless we analyse the problems and
objectives logically, we can begin designing the
                                                                    ELD training on Participatory Monitoring and Evaluation                                      95




project hierarchy from goal, which leads to confusion, illogical sequences and describing the
hierarchy upside down.

The session on designing indicators on various aspects of a project intervention was a very useful
exercise for participants. It made them aware of the different types of indicators and the
implications on Monitoring and Evaluation.

Day-end Reflection

Happy about …                                            Concerned about …
   • Practical exercises                                    • Training is perfect
   • Logical framework                                      • No need to evaluate. Keep it up
   • Empowerment circle


        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006                                - 11 -
                                                www.eld.org.uk
Day Five
The final day was an intensive and
useful day in many respects.                    PM&E Plan: Group Work
Animated slides on the various
stages of Environmental Scanning                          Key
                                                                       Methods of
and the sharing of one example from         Evaluation    Evaluation   Data         Key               Facilitator    When
                                            Aspects                    Collection   Informant
one participatory strategic planning                      Questions
                                                                       and Analysis
ensured participants grasped the
essence of this tool.                          ---


                                               ---
The last session of the day reviewed
the entire course and developed                ---
individual action plans on PM&E.
The facilitator presented a format              ---
and requested participant to think
how they would like to use the
knowledge and skills from this
training in their field.

Finally the workshop was evaluated and participants were awarded with certificates and given
resource CDs containing slides, photos and supplementary materials.




       Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006         - 12 -
                                               www.eld.org.uk
Annexes of Group Work
1 Discussion Design

Group One
Topic/Agenda: Evaluation of ELD Training Hall
R.O: Participants come up with suggestions fro ELD to improve its hall
E.O: The participants feel that they have contributed in improving their training hall
Objective:
         What did you notice when you come into the hall for the first time?
         What are the facilities do you observed?
         What is the capacity of this hall?
Reflective:
         What do you like and dislike about this hall?
         How do you feel when you are sitting in this hall?
Interpretative:
         How would you arrange it differently and why?
         What changes would you like to see and why?
Decisional:
         What suggestions and recommendations would you like to provide to ELD?

Group Two
Topic/Agenda: Community peace building
R.O: Identify the role of youth in community peace building
E.O: The participants realise that they have crucial role to play

Objective:
         What are you doing in community peace building?
         What groups are affected by this conflict?
         How would you see the relationship between youth and conflict?
Reflective:
         What do you think youth have lost during last ten years?
         Who do you think has the prominent role in peace building?
Interpretative:
         How can youth contribute in community peace building?
Decisional:
         Where to start from?




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006   - 13 -
                                                www.eld.org.uk
2 Workshop Method Practical

Focus Question: What roles can youth play in community peace building?

Campaign and          Advocacy and            Leadership             Monitor HR              Reconstruction
procession            awareness               development            violation
                      raising
Formulate             Media advocacy          Taking leadership      Monitor human           Reconstruction of
pressure group        for peace building      roles in resolving     right violation by      infrastructure
for peace building                            conflict in their      the conflicting
                      Awareness               community              parties                 Resource
Interaction           raising on                                                             mobilisation for
programs at           constitution            Playing role                                   rural
different levels      assembly                model of peace                                 development
                                              builders
Organising peace Performing street
processions      dramas that carry Bringing people
                 peace message     together across
Formation of                       dividing lines
community peace Competition on
groups           peace
                 development
Youth campaign topics
for peace
development      Educating people
                 in the community
                 about peaceful
                 ways of solving
                 conflict

                      Advocacy against
                      HR violation

                      Community
                      awareness
                      programs




        Report on training course on Participatory Monitoring and Evaluation, Kathmandu, August 2006     - 14 -
                                                www.eld.org.uk