Read this report - Institute of Development Studies - Institute of by zhangyun

VIEWS: 5 PAGES: 54

									Improving the Impact of     PROGRAMME DOCUMENT NO. 4
 Microfinance on Poverty

        Action Research
             Programme      Asia Region Workshop on: Microfinance and Impact
                            Assessment Methodology, Bhuvaneswar, India




                            July 2001




                            CMF, Nepal




                                                                           Imp-Act
                                                      The Institute of Development Studies,
                                          at the University of Sussex, Brighton BN1 9RE, UK
                             Tel: +44 (0) 1273 873733, Fax: +44 (0) 1273 621202/691647
                                       Email: Imp-Act@ids.ac.uk     Web: www.Imp-Act.org

               Charitable Company no.877338 limited by guarantee and registered in England
CONTENTS

1. List of Acronyms ____________________________________ 3
2. Foreword__________________________________________ 4
3. Executive summary __________________________________ 4
4. Objectives of the workshop ___________________________ 6
5. AIMS (Assessing Impact of Microenterprise Services) tools __ 7
   5.1.1. Impact Survey _________________________________________________ 7
   5.1.2. Client Exit Survey ______________________________________________ 8
   5.1.3. Client satisfaction survey _________________________________________ 8
   5.1.4. Client empowerment survey ______________________________________ 8
   5.1.5. Loan and savings use. ___________________________________________ 8
   5.1.6. Experiences of different organizations with the use of AIMS tools _________ 9
     5.1.6.1. CARD ____________________________________________________ 9
     5.1.6.2. SHARE __________________________________________________ 10
     5.1.6.3. CMF ____________________________________________________ 12
 5.2. Focus Group Discussions ______________________________ 13
 5.3. Semi-structured Dialogue _____________________________ 16
 5.4. Participatory Rapid Appraisal __________________________ 18
   5.4.1.   Seasonality of Income, Expenditure, Savings and Credit _______________ 20
   5.4.2.   Product Attribute Ranking _______________________________________ 21
   5.4.3.   Financial Services Matrix ________________________________________ 22
   5.4.4.   Life-cycle Profile. ______________________________________________ 23
 5.5. Development Audit: __________________________________ 24
 5.6. Internal Learning System: _____________________________ 26
 5.6. Impact Monitoring ___________________________________ 28
6. Comments on field trip ______________________________ 29
7. BRAC’S Presentation ________________________________ 31
8. Quantitative Issues in surveying: ______________________ 34
9. Qualitative issues __________________________________ 37
10. Future plans of partner organizations: _________________ 40
 10.1. PRADHAN: ________________________________________ 40
 10.2. CMF _____________________________________________ 41
 10.3. CYSD ____________________________________________ 43
 10.4. CARD ____________________________________________ 44
11. Regional planning for what next: _____________________ 45
12. Final comments on the seminar ______________________ 46
Annex:_____________________________________________ 50
 Agenda of the Workshop: _________________________________ 50
 List of participants ______________________________________ 51
1. List of Acronyms

AIMS: Assessing Impact of Microenterprise Services
ASA: Activists For Social Alternatives
BRAC: Bangladesh Rural Advancement Committee
CARD: Centre for Agricultural Rural Development
CMF: Center for Microfinance, Nepal
CYSD: Center for Youth and Social Development
DA: Development Audit
FGD: Focused Group Interviews
IA: Impact Assessment
IDS: Institute of Development Studies
ILS: Internal Learning System
MFC: Micro Finance Council of the Philippines, Inc.
MFI: Microfinance Institution
NABARD: National Bank for Agriculture & Rural Development
PRA: Participatory Rapid Appraisal
SIDBI: Small Industries Development Bank of India
2. Foreword

Imp-Act is a three-year action research to assess the impact of microfinance
programs and institutions in the lives of the poor people. It is being conducted in
different parts of the world simultaneously with financial support from the Ford
Foundation and a collaborative implementation effort of about twenty
organizations over the world. The global process is being coordinated by three
universities in the UK, with the Institute of Development Studies, Sussex taking
the lead coordinating role. The different organizations are further divided into
five regions. This report is based on the second Asian regional workshop that
was held from July 3-9, 2001 at Bhubaneswar, India. This workshop was focused
on developing a methodology for Impact Assessment that will be suitable for
each partner organization in Asia. The Asian regional partners in this action
research are SHARE, CYSD, and PRADHAN from India, CARD from the
Philippines, and CMF from Nepal. CMF is also the regional coordinator for the
program.

I would like to acknowledge the individuals that have contributed to making the
workshop successful. I would like to thank Ani Rudra Silwal, who is working as a
summer intern at CMF, Nepal, for writing this report. I would also like to thank
Tripti Uprety, another intern at CMF, for assisting in the preparation of the
report.

3. Executive summary

Imp-Act Asia is a part of the Global Impact Assessment Project funded by the
Ford Foundation. The First Asia meeting of Impact was held in June 2001. After
the first meeting, the participants proposed a seminar on impact assessment
methodologies which CYSD, India volunteered to host in July 2001 in
Bhuvaneswar, India. The workshop would be organized by CMF in coordination
with the CYSD and UK teams. The agenda for the workshop was developed and
finalized virtually by the Asia team.

With the multitude of tools available for assessing impact the goal of the
seminar was to expose the participants to a range of experience in IA
approaches, methods and tools, and to provide training in these in such a way to
improve the capacity of participants to identify and use approaches appropriate
to their work. The workshop was also intended to review the work-in-progress of
participating organizations, in the light of the available resources on impact
assessment methodologies, to develop a work plan for regional collaboration,
and to initiate discussions and develop a work plan for “wider impacts” work in
Asia.

The participants of the seminar (see Annex) included representatives of the five
Imp-Act Asia organizations – CARD, CMF, CYSD, PRADHAN, and SHARE. The
resource persons for the seminar ranged from different countries and
organization both within partners and outside. They included Mr. Anton
Simanowitz, Mr. Martin Greeley and Ms. Naila Kabeer from the Imp-Act UK
team, recognized resource persons including Mr. M. Hussain(BRAC), Ms. Helzi
Naponnen, and Ms. Francis Sinha. There were also observers from different
organizations in the region that are involved in interesting impact assessment
work.
This is a report on the proceedings of the Bhuvaneswar seminar. It will first
discuss the tools that were presented by the partners who had experience using
them and other tools that were presented by experts. It will summarize each
tool by discussing the approach it takes, the context it is applicable in and the
issues/concerns that were raised about it during the seminar. Thereafter, it will
discuss the qualitative and quantitative issues regarding the tools that were
discussed and presented during the seminar.

With the variety of tools and methodologies available for assessing impact, it
was agreed upon that a good research should involve three stages. The first
stage should involve qualitative work, aimed at developing an understanding of
the socio-economic conditions of the area through qualitative techniques such as
diagrams, matrices, wealth ranking, social-mapping. The goal of this process is
also to develop relationships of trust with the villagers and familiarizing on both
ends. The second stage should involve quantitative work, which involves
selecting and adapting appropriate tools for the research and conducting the
survey to get the data. This stage also involves doing the quantitative analysis of
the collected data. The third and the final stage should involve comprehension
via case studies. Since complicated issues and processes might underlie the
changes that are observed through quantitative results, qualitative methods
such as focus group discussions, case-studies, semi-structured interviews, and
mini surveys can be used to explore specific issues for richer, more informed
results.

Participants of the seminar realized that there is no universal “best practice”
package for assessing impact. What has to be done is to find the right
combination of tools and to adapt them to specific circumstances and contexts.
Ideally, a series of IA would be ideal so that every organization can learn from
its own impact assessment and IA goes beyond the three year Ford Foundation
project.

Participants also realized that packages such as AIMS and MSA tools can easily
be substitutes for thinking; some of them are general and others work in one
context but do not in others. Thus it is crucial to develop an understanding of
your specific circumstances in order to develop the appropriate tools. In trying to
understand the context of the area being studied, one should focus not only on
local circumstances, but also what is happening in nearby villages, macro
policies of the country, etc. Another warned that it is too easy to be blinded by
techniques and to forget what goals you are trying to achieve through IA: to
prove or to improve. Only after the organization is clear about its goals, should it
think about the appropriate tools.

As suggested in the concept paper for the virtual meeting on impact assessment
methodologies, there is a trend away from quantitative survey methods into
participatory tools that are that are mostly qualitative. So there was a lot of
discussion and curiosity about participatory tools during the workshop. However,
the resource persons were concerned that despite the interest in participatory
tools, partner organizations seemed to stick primarily to conventional survey-
based impact assessment. This could have occurred since survey-based impact
assessment methods are still predominant despite the rapid acceptance and use
of participatory tools.
With similar tools being used by most of the participating organizations, the
participants expressed the need for regional sharing of experience at the end of
the seminar. One way to do that would be study tours inside each region so that
organization can work together to learn from each other. When question was
raised about whether funding exists for such programs, the organizers
mentioned that funding for networking activity is within the budget of the
program and can be utilized when appropriate. The participants also suggested
sharing of individual work-plans so that everyone knows what others are doing.
Sharing would be especially useful with questionnaire since the same tools are
being used by most organizations. The list-serv was suggested as an ideal
platform for partner organizations to inform others of not only their needs and
plans but also preliminary findings from their study. The participants also
suggested that the absence of some experts such as the AIMS and MSA tools
was a disadvantage, although the presenters of those tools had sufficient
experience using them.

4. Objectives of the workshop

Goal of the overall Action Research

   To access the impact of the program on reduction of poverty and
    improvement in the quality of life of the clients.
   To identify strengths and weaknesses of the operations and thereby
    suggest/provide innovative measures on improving the quality of services.
   To assess the impact of the program on non- clients and the external
    environment.
   To assess the economic growth, entrepreneurial development, confidence
    level, decision-making and participatory level of the clients in all fields of life.
   To develop simple low cost and effective tools for „impact assessment‟ to be
    integrated into the day-to-day operations.
   To assess the quality of services and products offered and thereby improve
    the overall efficiency of the operations.
   To assess the wider impact of the program.
   To evaluate how far the objectives are being achieved.
   To assess the economic and social impact of the program on the clients.

    IMPACT ASSESSMENT is a participatory and negotiated process that tracks
    and captures significant and lasting changes in peoples lives, as a result of an
    intervention, done together with the people. The basic principles of Impact as
    we believe are:

   IA to be designed and implemented so as to ensure/enhance local ownership
    of purpose, process, methods and results.
   IA to be important part of the learning process for all those concerned:
    Organization (MFI), SHGs Enterprise/Beneficiaries, and also communities.
   IA to be a multiparty (Stakeholder) negotiated process in which people
    centered principles must predominate the research itself has to be
    empowering for people.
   The strategic framework of IA has to combine in a very creative way the
    three key elements of (our) research: Flexibility, A learning agenda, and
    scientific rigour.
   Instead of a standardized blueprint approach, IA has to be a dynamic and
    pluralize. (e.g. PRA, case study, Documentation Review, participant
    observation, survey methods etc.)         With appropriate trade-offs and
    triangulations. The question to be explored is: what mix of methods is most
    appropriate for this study and how should they be combined?

In order to capture all these principles and to try and be in the “right track” right
from the beginning the Asia team is organizing this workshop on methodology
with the following objectives:

Objectives of the workshop

   To expose participants to a range of experience in IA approaches, methods
    and tools, and to provide training in these in such a way to improve the
    capacity of participants to identify and use approaches appropriate to their
    work.
   To review the work in progress of participating organizations, in the light of
    the available resources on IA methodologies.
   To develop a work plan for regional collaboration
   To initiate discussions and develop a work plan for “wider impacts” work in
    Asia.


Participants in the workshop

   Five Imp-Act Asia organizations – CARD, CMF, CYSD, PRADHAN, SHARE
   Input from Imp-Act UK team – Dr. Anton Simanowitz, Dr Martin Greeley and
    Dr. Naila Kabeer.
   Maximum of 3 or 4 other organizations in the region that are involved in
    interesting IA work and who can fund for themselves. Interested
    organizations from other regions that have an interest in IA work could also
    be included provided they can fund their cost.
   Recognized resource persons including Dr. M Hussain, BRAC, Ms Helzi
    Neponen

5. AIMS (Assessing Impact of Microenterprise Services) tools

Sponsored by the Office of Microenterprise Development of the USAID since
1995, AIMS is a set of five assessment tools that practitioners can use to gather
information about their programs – information that is useful for impact
assessment and market research. On the one hand, MF practitioners need to
know what difference their programs are making and for whom. On the other
hand, meeting the high standards of academic research is beyond the reach of
most MF practitioners. Responding to these challenges, AIMS tries to find an
acceptable balance between the quality of an IA and its costs by setting a basic
standard of practitioner-led IA.

To meet the dual goal of proving impact and improving products/services AIMS
has developed five tools – three qualitative and two quantitative:

5.1.1. Impact Survey
This principal quantitative tool in the set comprises thirty-seven questions is
administered to a sample group of clients and a comparison group. To simplify
the task of selecting the latter group, a random sample is chosen from incoming
clients who have chosen to join the program but have not received any services
to date. The assumption is that those choosing to join the program are similar to
existing clients in terms of demographic characteristics, motivation, and
business experience, and thus offer an appropriate and easily identified
comparison group.

5.1.2. Client Exit Survey

This short survey, a quantitative tool, is administered to clients who recently
have left the program. Its purpose is to identify when and why clients left the
program and what they think its impact, as well as its strengths and
weaknesses, has been. This survey identifies client satisfaction with the
program. As such, it is a valuable management tool that can be applied regularly
as part of the institutional monitoring system or that can be used as a
component of a periodic evaluation. The advantages of the exit survey is that it
is simple to use since the answers are pre-coded and facilitate analysis using
EpiInfo data analysis software. It is appropriate both for one time and regular
use. It helps management identify problems that cause clients to leave, and
verifies staff impressions about why clients are leaving.

5.1.3. Client satisfaction survey

Given the dual aim of proving and improving programs, IA must go beyond fairly
easy measures of client satisfaction, ie number of drop-outs and/or sustained
participation. To fulfill this goal, this qualitative tool, which is a focus group
discussion, explores clients‟ opinions – what they like and dislike – of specific
features of the program, as well as their recommendations for improvement. In
doing so, this tool serves as a reality check to determine if the program is
indeed meeting clients‟ needs.

5.1.4. Client empowerment survey

This qualitative tool focuses on women clients and uses an in-depth interview to
determine if and how women have been empowered by their participation in the
program. Clients are asked a series of questions about themselves, their
enterprise, their family/household, and their community at different points in
time (past and present). The tool includes a methodological option to use self-
portraits as a way to initiate this discussion. Empowering the poor – to set goals,
organize themselves, make important decisions about their lives, and
communities – is a key part of alleviating poverty. This tool concentrates on
outward manifestations or concrete demonstrations of empowerment as a result
of participation in peer borrower groups and access to credit through
participation in the program.

5.1.5. Loan and savings use.

This qualitative tool is an in-depth individual interview focusing on how the client
has used his or her loans and business profits over time. Its multiple purposes
include determining how loan use and allocation decisions change over time, as
well as documenting changes in the individual borrower, enterprise,
family/household, and community that are associated with participation in the
program.

Good things about AIMS

   it is done by and for practitioners to reflect their needs and concerns
   it is practical and cost-effective but also meets scientific standard of validity
   it gives immediate results which can be used by management to inform
    policy and improve operations.

5.1.6. Experiences of different organizations with the use of AIMS tools

5.1.6.1. CARD

AIMS tools were first introduced in the Philippines during a CASHPOR workshop
in May 1999 where the participants of the workshop believed that AIMS could
improve the quality and outreach of MF services because it is cost-effective,
practical, is done by practitioners, and gives immediate results. It was then that
CARD decided to implement AIMS within its organization. Since then, it has used
the following tools:

a) Loan and Savings Use Tool
b) Empowerment Tool

In order to adapt AIMS tools to the Philippines, CARD has to make the following
customizations to the tools:

   Re-designing the questionnaire
   Translating the questionnaire into the local language
   Field testing the questionnaire to make sure the questions are appropriate.

Comments and questions from group discussion

Question:

What challenges did you face during the implementation of the AIMS tools?

Answer:

We faced the following problems during the implementation of the AIMS tools:

a) Institutionalizing IA. We realize that IA is more than just pilot projects
   initiated by donors and reports written for them. However, as IA is relatively
   new in the Philippines, we are having some difficulty making IA a part of the
   operations of our organization.
b) Finding the resources for IA. Since conducting an impact study is an extra
   burden for CARD, we are finding it difficult to convince the management that
   time/resources for IA is useful and necessary. Securing resources for
   assessing impact has been difficult.

Question:
What modifications have you had to make to the AIMS tools to adapt them to
the Philippines?

Answer:

Since AIMS tools are very general, they have to be customized to each country.
In our case, we had to make several modifications including translating into the
local language, and changing way of measuring annual income.


5.1.6.2. SHARE

SHARE used the following tools in its first phase of IA

   Client impact
   Client exit
   Client empowerment
   Client satisfaction
   Loan utilization

Their methodology was as follows

   Selection of Branches
   Sampling Data
   Selection and Training of Staff
   Designing Questionnaires
   Data Collection
   Input of Data
   Analysis of the Data
   Reporting

SHARE came across the following limitations during the impact study
that they conducted

   Coverage in terms of Members
   Costs involved
   Implementation in MIS
   Training to staff
   Exit clients not available at the time of survey
   Fund mobilization
   Identifying the samples

They have the following concerns regarding the use of AIMS tools

   Costs involved
   Selection of control group
   Selection of tools
   Time Period for the Study
   Staff Data Collection
   Acceptance by internal and external agencies
   Difficulties in procuring, reliable and honest data
   Selection of the proper time for the Study
   Expertise available with external consultants

Comments and questions from group discussion

Question:

Different people work in different time frames. Not everyone thinks in terms of
years and months – the terms we use in our studies to mark time. What
modifications did you make, if you made any, to take into account the different
timelines people operate in?

Answer:

We used major events such as elections, typhoons, important official visits, etc.
to remind people of time.

Comment:

   A combination of the following recall markers can be used to anchor time:
   Calendar years.
   Personal time markers, such as marriage, birth of children, death of family
    member, age of kids.
   Historical, such as assassination of important politicians, change of
    government, elections.
   Local markers such as typhoons, landslides, floods.

Comment:

   It is not always necessary to pinpoint the exact calendar year, or month to
    evaluate impact. Using relative time markers such as two years ago or the
    year before that will do the job of marking time.

Comment:

   Critical incidence analysis within the community can also be used to mark
    time. This technique will require different sets of timelines: important events
    in the community, organization, project, and wider society. Those events can
    be correlated to make sense of recall time.

Comment:

   It is of utmost importance to take into account local factors in identifying
    timeline. For example, since festivals are important part of social life in India,
    their aid should be taken while establishing timeline.

Question:

What changes did you have to make for to adapt the AIMS tools to your
organization?

Answer:
We had to make the following changes in order to adapt AIMS tools to our
organization:

   Translating the questionnaires into the local language.
   Formulating indirect questions (cross-questioning)        to   make   sure   the
    information received was reliable.
   Using local understandings for measuring wealth level.

5.1.6.3. CMF

CMF has used the following AIMS tools during the course of its activities:

a) Loan and Savings Use
   This tool was used to assess the risk and vulnerability of rural women of
   Nepal. It was conducted with two groups of 18 and 14 individuals in two
   geographic regions. The questionnaire was semi-structured thus had a
   qualitative twist to it. The goal of the tool was to identify risks and existing
   risk-coping mechanisms of rural women and design new loan products for the
   clients. CMF had to make the following adjustments to the questionnaire:

    Translation into the local language.
    The questionnaire had to be reformatted to add questions that are suitable
    for designing loan products for clients.


b) Women‟s Empowerment Project
   This was a structured questionnaire that was conducted in nine districts of
   Nepal with 500 individuals. The AIMS questionnaire had to be modified to
   take into account the goals of WEP to add questions on women literacy,
   entrepreneurship, empowerment, etc. The problem with this tool was that
   since it is quantitative, its analysis requires qualified people who know how to
   use data analysis software such as SPSS or EpiInfo.

Comments and questions from group discussion

Question:

When you use then-now analysis, how do attribute changes to specific projects?

Answer:

We have used base-line data as often as we can in order to get around the idea
of attribution.

Question:

One problem with AIMS is that, it uses then-now analysis most of the time to
assess impact. Since timeline is so important in such analysis, how do you make
sure that you capture the time-line correctly?

Answer:
We have not put too much effort into ensuring that the information we collect is
precise. We believe that doing IA and the reason behind doing IA are more
important that having lots of precise information and not knowing what to do
with it later on.


Question:

When using the Loan and Savings Use Tool, did you ever feel that your clients
had difficulty remembering the details on the loans? In such a case, does it
make more sense to evaluate the significant loans and the first/last loan?

Answer:

We did take those issues into consideration and made sure that our staff were
very well-trained. We also tried to use women interviewers more than men
because from experience we have realized that they get better responses in the
field than men.

Question:

Another problem with AIMS tools is that they are mostly cross-sectional and not
longitudinal. What problems does that create?

Answer:

Ideally, longitudinal study would make more sense since impact is long-term
change and not short. However, because of resource constraint, we sometimes
have to get around longitudinal study by using timelines and imagination based
on cross-sectional data.


Further resources

   SEEP/AIMS Manual, “Learning from Clients: Assessment Tools for
    Microfinance Practitioners” (January 2000), available on CD-Rom, printed
    from AIMS, or on web sites www.mip.org or www.cgap.org

5.2. Focus Group Discussions

Focus group research is certainly not static and the approach and methodology
is constantly changing. Focus groups have evolved over the past decade several
distinctive changes have become apparent. Focus groups have been a mainstay
in private sector marketing research. More recently, public sector organizations
are beginning to discover the potential of this procedure. The focus group is
unique from other research methods, i.e. in-depth interviews, face-to-face
interviews and using questionnaires to get information etc. as it allows for group
interaction and greater insight into why certain opinions are held. Focus groups
can improve the planning and design of new programs, provide means of
evaluating existing programs, and produce insights for developing marketing
strategies.
The focus group is a special type of group in terms of purpose, size,
composition, and procedures. A focus group is typically composed of 6 to 10
participants who are selected because they have certain characteristics in
common that relate to the topic of the focus group. In the past experience,
smaller groups show considerable potential. Smaller groups of 6, 7, or 8
participants not only offer more opportunity for individuals to talk but are
considerably more practical to set up and manage.

Characteristics of focus groups

Focus groups are useful in obtaining a particular kind of information –
information that would be difficult, if not impossible to obtain using other
methodological procedures. Focus group interviews typically have six
characteristics or features. These characteristics relate to the ingredients of a
focus group:

1.   people
2.   assembled in a series of groups,
3.   possess certain characteristics, and
4.   provide data
5.   of a qualitative nature
6.   in a focused discussion

Advantages of Focus Group Interviews

The Focus Group Interview offer several advantages:

    It is a socially oriented research procedure. People are social creatures who
     interact with others, influenced by the comments of others and make
     decisions after listening to the advice and counsel of people around them.
     Focus groups place people in natural, real-life situations as opposed to the
     controlled experimental situations typical of quantitative studies. Also, the
     one-to-one interviews are not able to capture the dynamic nature of this
     group interaction.
    The format allows the moderator to probe
    Focus group discussions have high face validity. The technique is easily
     understood and the results seem believable to those using the information.
     Results are not presented in complicated statistical charts but rather in lay
     terminology embellished with quotations from group participants.
    Focus group discussions can be relatively low cost.
    Focus group discussions can provide speedy results. For example, in
     emergency situations skilled moderators have been able to develop and test
     the questionnaire, conduct three to four discussions, analyze the results, and
     prepare a report in less than a week.
    Focus groups enable the researcher to increase the sample size of qualitative
     studies. Quantitative studies typically have limited sample sizes because of
     the time and cost constraints of individual interviewing. Focus groups enable
     the researcher to increase the sample size without dramatic increases in the
     time required of the interviewer.

Limitations of Focus Group Interviews
All techniques for gathering information have limitations and focus group
interviews are no exception. It is important to be aware of these limitations in
deciding whether to use this technique. Among the limitations are the following:

   The researcher has less control in the group interview as compared to the
    individual interview. The focus group discussion allows the participants to
    influence and interact with each other, and as a result, group members are
    able to influence the course of the discussion. This sharing of group control
    results in some inefficiency such as detours in the discussion, and the raising
    of irrelevant issues, thus requiring the interviewer to keep the discussion
    focused.
   Data are more difficult to analyze. Group interaction provides a social
    environment, and comments must be interpreted within that context. Care is
    needed to avoid lifting comments out of context and out of sequence or
    coming to premature conclusions. Occasionally participants will modify or
    even reverse their positions after interacting with others.
   The technique requires carefully trained interviewers. The open-ended
    questioning, the use of techniques such a pauses and probes and knowing
    when and how to move into new topic areas require a degree of expertise
    typically not possessed by untrained interviewers.
   Groups can vary considerably. Each focus group tends to have unique
    characteristics. One group might be lethargic, boring, and dull; the next
    selected in an identical manner might be exciting, energetic, and
    invigorating.
   Groups are difficult to assemble
   The discussion must be conducted in a n environment conducive to
    conversation. These factors often present logistical problems and may require
    participants incentive to participate. By contrast, an individual interview can
    be held ina location and at a time most convenient to the interviewee.

    Much of the success of the focus group depends on the quality of the
    questions and the skills of the moderator. Quality questions require
    forethought and planning. Successful focus groups begin with well-thought-
    out questions that are appropriately sequenced. Open-ended questions allow
    the respondent to determine the nature of the answer. Dichotomous
    questions and direct, interrogative “why” questions are to be avoided.
    Interviews are focused by providing participants with consistent and sufficient
    background information and by presenting the questions in a context.

    Focus groups have been found useful prior to, during, and after programs,
    events, or experiences. They have been helpful in assessing needs,
    developing plans, recruiting new clientele, finding out customer decision
    processes, testing new programs and ideas, improving existing programs,
    and generating information for constructing questionnaires.


Qualities of a good moderator

   An ability to get respondents to express themselves freely
   Conscious of others
   Enthusiastic & lively
   Good sense of humor
   Genuine interest in finding out what others think
   Friendly/Unthreatening
   Animated/confident
   Able to build rapport
   Respectful
   Out-going
   Observant
   Control tone of voice
   Neutral
   Answers the research question (more on this later)
   Selects an appropriate venue (more on this later)
   Listens/is patient

Comments from group discussion

- A source of tension during FGDs is whether the moderator is from inside or
outside.
- The fact that the facilitator asks questions although the group does not is a
bad practice.
- There is a difference between leading the discussion and drawing attention to
something: the moderator should not lead the group discussion neither should
be the one giving all the answers.
- Maybe the moderator should come at the very end of the discussion and add
the points that he/she thought were missed. Or after the discussion has been
going on for a while.
-Clearly spelling out your expectations and the objectives at the beginning of the
discussion is important.
- A problem might arise if there are people in the group who object to the
objectives of the moderator.
5.3. Semi-structured Dialogue

Semi-structured dialogue is a flexible two-way process where only some initial
topics are predetermined in the checklist to be applied. These topics can, and
should, be revised as the researcher learns or as the participants contribute to
the initial agenda. Furthermore, the questions raised and answers given depend
on the interests and knowledge of both the participants and the researchers.
Semi-structured dialogue is an informal process and it is therefore more
appropriate to talk of a dialogue than an interview.

Semi-structured dialogue is central to market research because it is the process
whereby trust and rapport are established and within which the checklist of
issues is addressed. Such trust and rapport are essential if reliable and relevant
learning is to be acquired.

Behavioral guidelines

Non-verbal communication

Research has shown that less than 10% of the message comes from the content
of what is said! More than 40% comes from tone and odulation of the voice and
the remaining 50% from non-verbal communication (mainly attitude and body-
posture).

The following may help to minimize the impact of your behavior on the dialogue
with participants:

   Maintain a comfortable social distance do not sit aboce the participant
   Do not show distaste/ disapproval about surrounding conditions
   Do not indicate contempt or disbelief in what is being said
   Do not refuse local hospitality
   Do not look and act too official

Attending and listening carefully

This helps establish good rapport and implies more than just physical presence –
it implies a certain intensity of presence. Some possible guidelines for attending
and listening are:

   Face the participant and adopt a posture indicating involvement
   Adopt an open posture
   Learn to be intimate (do not lean back or slouch)
   Maintain eye contact and do not look away too much.
   Be relaxed and comfortable (do not fidget too much)

Obviously such guidelines will vary from one culture to another. They will also be
context specific. One should remember to use one‟s best judgement at all times.
Rather than considering what you should do you can also consider what not to
do. Experience has shown that PRA facilitators tend to make the following errors
and that they are subject to the following biases:


Team errors

Common team errors include:

   Interrupting colleagues or participants
   Switching suddenly to a new topic
   Failing to plan and prepare properly
   Failing to give enough time to the discussion

Common Individual errors

Common individual errors include:

   Failure to listen closely
   Repeating questions
   Asking leading questions and suggesting answers
   Asking ambiguous questions
   Asking insensitive questions
   Failing to probe
   Failing to judge answers
   Allowing the dialogue to go on too long
Probing answers

Probing in necessary to obtain detail, elaboration and depth on a subject of
interest, until you are satisfied with the learning acquired or judge that it would
be counterproductive to go further. Making constant use of the six helpers:
what, where, when, who , why, and how; using key probes such as „what do you
mean?‟, „anything else‟, „ please tell me more‟, etc; using role-playing to probe
abstract issues, etc will help probing.

Triangulation

The learning acquired from any dialogue can only be part of a wider story. You
may also have been misled or may have been misinterpreted the answers given
to you. In addition to judging the answers given to you. In addition to judging
the answers as suggested above, it is, therefore, also very important that you
check your information from other sources of information and/or by using
different methods.

Recording the dialogue

It may be useful to take notes of the dialogue during and/or immediately
afterwards. If so, the use of small notepads is likely to be more appropriate than
using large official looking notebooks. Finally, it is helpful to make a clear
distinction between your thoughts and interpretations and what was actually
said by the participants. Use of both can then be made in writing the report.

Four common biases

Much development dialogue also suffers from a number of other biases. The
most common biases include the following:

   Elite bias: the tendency to give more weight to the knowledge, opinions and
    ideas of the educated and articulate
   Hypothesis confirmation bias: the tendency to focus selectively on
    information, knowledge and ideas which confirm the preconceived
    hypotheses, assumptions and beliefs of the PRA facilitators
   Concreteness bias: the tendency to generalize from the particular without
    probing or triangulating
   Consistency bias: the tendency to search prematurely for coherence in the
    learning acquired, in order to be able to draw meaningful conclusions as
    quickly as possible.

5.4. Participatory Rapid Appraisal

PRA is a tool by which participating communities, with the assistance of outside
facilitators, collect and analyse information themselves about their own lives and
community. PRA information is gathered and analysed “by the people, for the
people”. Results are processed and validated at the time of information
collection by peer groups.
Main Advantages of PRA

Ease of data collection, validation and analysis in a few community meetings.
Accuracy and coverage: Community provides accurate information when
validated in an open forum.
Ease of replication at a later date.
Strengthening of community planning capabilities.

Basic Principles of PRA

   offsetting biases: spatial, personal etc.
   rapid progressive learning: flexible, exploratory, interactive, inventive.
   reversal of roles : learning from, with and by local people
   appropriate imprecision: not finding out more than is needed and not
    measuring when comparing is enough
   triangulation: using different methods, sources and disciplines.
   direct learning: from and with local people.
   seeking diversity and differences: between different environments/people
    etc.
   facilitation: to enable local people do the mapping, modelling, diagramming,
    ranking, scoring, quantification, analysis etc.
   sharing: of information
   behaviour and attitudes: of external facilitators are of prime importance

Initial PRA Meetings and Consultations

   Arrange PRA meeting with the community ahead of time
   The purpose of the PRA exercises/ meeting: to develop improved financial
    services
   Flexibility is required by the facilitators in order to conduct the PRA
    successfully.
   The process of PRA and discussions between participants is as important as
    the output itself.

Practical TIPS for Facilitators

   Explain clearly the purpose and methods.
   Let informants/community do the work.
   Listen, learn and facilitate.
   Choose a place where informers feel relaxed and free to work.
   Discuss points of interest - probe, probe: why ? how ? when ? where ? what ?
   Encourage participation. Allow time for discussion, corrections and additions.
   Record observations.

Checklist of Materials Required

   “The stick” to moderate the flow of information/who is doing the talking
   Notebooks to record information in
   Manila paper: marker pens, crayons, scissors, glue and chalk etc.
   Blackboard: chalk
   The floor: stones/bottle-tops, sticks, chalk etc.
Menu of PRA Tools

   Seasonality of income, expenditure, savings and credit
   Seasonality of migration, casual employment, and goods/services provided
    by the poor
   Life-cycle profile to trace cash needs over time
   Venn diagram analysis of groups/organisations and their roles
   Ranking
   Relative Preference Ranking
   Pair-wise Ranking
   Wealth Ranking
   Cash Mobility Mapping
   Time Series of sickness, death, loss of employment, theft, natural disaster
    etc.
   Time Series of asset ownership
   Financial Services Matrix
   Financial Sector Trend Analysis

Analysing PRA Results

A Suggested Categories of Analyzing PRA Results and other data:

Results can be analysed using the categories below:

   Life cycle needs. E.g. Birth, death, marriage, education,
   Emergencies. E.g. fires, illness, accident, bereavement, desertion and
    divorce.
   Opportunities. e.g Starting or running businesses, acquiring productive
    assets, or buying consumer durables
   Consumption Smoothing - during times of low revenue

Some guidelines for Analysis of PRA

   Continuously writing-up and review the information as it is collected
   amending questions/ techniques - as need arises during the process
   Arrange and re-arrange findings according to a list of key issues.
   Be self-critical.
   Discuss each subtopic in turn,
   Tabulate the information to pullout key information
   Summarize the results, and
   Draw conclusions
   Cross-check your findings and conclusions e.g. with clients, field staff
   Findings have to be consistent and must not contradict each other.
   Two opposite statements cannot be true at the same time. If the findings
    contradict the secondary sources or other sources you must be able to
    explain why.

Below are descriptions of four of the PRA tools: Seasonality Analysis, Financial
Services Matrix, Product Attribute Ranking, and Life-cycle Analysis.

5.4.1. Seasonality of Income, Expenditure, Savings and Credit
Purpose

To obtain information on seasonal flows of income and expenditure, and the
demand for credit and savings.
This then allows discussion of key issues relating to these four factors.
Provides insights into risks and pressures faced by clients and how they use
MFIs‟ financial services to respond to these.

Procedure

Best done in closed area with 6-10 people familiar with the community‟s
patterns of income, expenditure, savings and credit.
Seasonality chart drawn on Manila paper. Participants then place small stones
or beans/seeds/bottle-tops (min. 0, max.10 per month) to indicate the relative
magnitude of the four key variables.
The discussions between the participants as they prepare the chart are as
important as the final chart itself
Note the importance of probing during the construction of the seasonality
calendar

Group Discussion

People often wouldn‟t recognize months so they should be avoided as much as
possible and seasonal examples such as flowering be used.

Concern

What if people in the group disagree about the number of stars to be put in the
boxes?

Answer

This data is much more preferable at the individual/household level rather than
the group level.

5.4.2. Product Attribute Ranking

Product Attribute Ranking is a method for finding out what participants view as
the key elements/criteria in financial services and how relatively important
each is.

Purpose

This ranking allows us to see how clients and potential clients perceive the
components of financial services, and to know which of those
elements/attributes are more important for them.
It also helps challenge pre-conceived notions about poor people‟s attitudes
towards financial services, what matters to them, and why they have those
preferences.
Procedure
Participants to describe in their own terms what is good or bad about the
financial service/mechanism or problem etc. that is being ranked.
Probe for further criteria/components and,
Follow up with points of interest and encourage participation by different people.
Participants should list all the criteria/components generated in this way.
Remember to make negative ones positive/neutral (e.g. "high minimum
deposits" becomes "minimum deposit" or “low possibility of getting credit”
becomes “possibility of getting credit”).
Put all the criteria/components on cards – one for each criterion/component
(e.g. distance/proximity, interest on deposits etc.)
Ask participants to rank the cards arranging them with the most important
criterion/component at the top, going down to the least important at the bottom.
Listen and learn from the participants – particularly as they discuss the merits of
each criteria.
You should then finish with a table that looks like the one illustrated on the next
slide.

Comments from group discussion

   It is important to have a local understanding of seasons, terms, and socio-
    economic categories rather than the ones used by other people so that the
    stuff being said can be contextualized.
   When raking wealth it is difficult to define terms that can be applied to all
    areas. Therefore, it is not always a good idea to have a fixed number of
    categories for determining wealth-level.
   We should always be aware of the reason behind why we are doing our
    exercises and ask questions accordingly. The information you are looking for
    will determine the kinds of questions we ask.
   Since people‟s definition of poverty changes with time, it is difficult to define
    terms such as rich and poor for a long period of time.

5.4.3. Financial Services Matrix

Purpose

   Useful to determine financial services used by each socio-economic or socio-
    cultural strata of society, and thus the potential for designing or refining
    appropriate financial products.
   Note: The listing of financial services here will require modification based on
    the participants‟ knowledge and awareness of the various products.

Procedure

   Exercise best done in a closed area with a small working group of 6-10
    individuals from the same socio-economic stratum.
   When seeking to understand the needs/ realities of the poor, focus on
    carefully selecting poor participants for this exercise.
   If necessary the exercise can be repeated with other groups from other
    strata.
   Develop an initial listing of financial services (formal and informal sectors)
    available and used in the community.
   Participants review, edit and develop/ expand the list of financial services
    available /used in the area.
   This part of the exercise should be undertaken with care and as much
    probing as possible (particularly moneylenders).
   Once list developed, turn it into a matrix by making columns for “rich”, “not-
    so-poor”, “poor” and “very poor”.
   Check participants understand these classifications.
   Ask participants to place counters to indicate the level of use of each of the
    types of financial service by each of the socio-economic strata.
   Listen and learn from the participants – particularly as they discuss how and
    why each of financial services are used by each of the socio-economic strata.

5.4.4. Life-cycle Profile.

Purpose

   It seeks to identify the phases of a typical individual‟s life-cycle and the key
    milestones in it,
   Determine which of the events require a cash lump-sum,
   Then examine the implications of these for household income/expenditure
   Establish the current coping mechanisms/financial systems used and their
    associated constraints,
   And finally to discuss how access to MFI/Bank's financial services can help
    the household respond to these.

Use of Information Gathered

   Information gathered can be useful in terms of designing financial product
    that match the various needs expressed at different milestones during a
    person‟s life cycle.

Procedure

   The exercise is best done in a closed area with the assistance of a small
    working group of 6 -10 people.
   Get the participants to discuss and agree on the key household milestones
    requiring significant expenditure from the moment that the household is
    established (generally when one starts up his own house) to death.
   Write each suggestion on a card.
   Once around six suggestions have been given, work with the participants to
    lay the cards out in “life-cycle order”
   Screen out recurrent expenditures (e.g. beer or cigarettes or “poverty” !) and
   Generate additional examples of events requiring sums of money larger than
    those that the participants usually have around the house.
   Draw a life-cycle chart on the paper and layout the cards along the horizontal
    axis of the life-cycle chart.
   When the key milestones/events have been agreed get the participants to
    place a 5-10 seeds/pebbles/bottle-tops to show the level of need for lump
    sums associated with each key milestone/event of the life-cycle chart.
   As they discuss the need for lump sums of money, probe how they amassed
    these lump sums
   through savings (which mechanism, how, where, with whom, how long did it
    take? etc.)
   through borrowing (from whom, how much, at what rate of interest,
    repayable over what time period ? etc.)
   Discuss what constraints each method presents.
   Remember that what you learn from the discussion between participants as
    they prepare the chart is usually more important than the chart itself – these
    should be noted carefully.
   Thus the final chart (which the facilitator should copy into her notebook) will
    appear something like:

Comments and questions from group discussion:

Comment

- CMF used the life-cycle profile slightly differently in order to determine the life
cycle cash needs of its clients. In the exercise, CMF wanted to find out when
financial crunches occur in families. That helped to develop financial products
such livestock insurance, savings product and insurance.

Question

Since most of the people in the field will not be highly literate, is it possible to
make them understand tables, charts, etc.?

Answer

If the exercise is well-facilitated and sufficient time spent on describing the
tools, people will understand them.

5.5. Development Audit:

DA is a tool to improve development planning and organisational accountability.
There are a number of different organisations today working for development in
various ways. They vary in style, scale and objectives – from government
departments and corporates, to co-operatives, NGOs and panchayati raj
institutions. Yet they have at least one feature in common.

They all have a number of different stakeholders who should know if the
organisation is meeting intended – or even unintended – objectives. DA provides
a framework for answering these questions. It enables an organisation to assess
its objectives, activities and impact in relation to the resources invested, and
from the perspective of different stakeholders.

DA is based on a set of recognized audit and development principles including:
transparency, comparability, and verification. It involves a series of steps:
introducing the concept to the stakeholders, identifying stakeholder issues,
assessment of programme results and costs, communication and planning.

The DA process facilitates participation of key stakeholders in development
assessment and planning. The approach contributes to transparency and
organisational accountability and helps an organisation to:
   Clarify and review its objectives
   Highlight the priorities of its different stakeholders
   Assess impact in relation to different objectives and along various dimensions
    (economic, social ,institutional, environmental)
   Set and review realistic targets and benchmarks
   Assess and monitor financial and other resources involved in its programme.
   Document clearly inputs and effects
   Involve key stakeholders in carrying out the audit and in using the results for
    planning, monitoring and evaluation

The stakeholder based approach is similar to a Social Audit, with an emphasis
on:

   Transparency and communication
   Listening to weak as well as stronger stakeholders
   Open discussion of budgets and expenses
   Assessing results by looking at long term impact and achievement

DA in practice

NGOs in India have used DA to strengthen village level accountability and
transparency in the review and planning of village development programmes.

Comments and questions from group discussion

Question

What if objectives of stakeholders change over time?

Answer

It is obvious that objectives will change and get refined over time. Thus, it
becomes all the more important for those doing DA to adjust their tools
accordingly.

Question

Can there be a consensus on indicators between stakeholders?

Answer

Not necessarily. Therefore, building a menu of indicators that is acceptable by
everyone is important.

Further Resources

Manual and case studies available from Frances Sinha, Nand Kishor Agrawal,
Radhika Agashe, EDA Rural Systems Pvt. Ltd., 107 Qutab Plaza, DLF Qutqab
Enclave-1, Gurgaon-122002, India. Tel: +91 124 6350835/6356692 Fax:+91
124 6352489 E-mail: edarural@nda.vsnl.net.in
5.6. Internal Learning System:

Despite the experiences of more than two decades of microfinance program
activity in South Asia, a vast majority of MFIs still lack an on-going system to
track impact on participants. These organisations have relied on case studies of
a few successful participants to demonstrate the positive impact of their
operations. The critique of this approach has been that these participants may
be exceptional and would have achieved success on their own without the
program intervention. Rarely are       case studies of unsuccessful borrowers
undertaken so that the problems and limitations of credit as a development
program intervention can be better understood.

Both external quantitative study and in-house qualitative case-study share the
same disadvantage of being one-off or occasional undertakings. Program
elements cannot be adjusted to increase impact and make operations more
efficient without a system in place to detect lagging performance and
deficiencies as they occur. In addition, despite the rhetoric of participation and
institutional ownership of projects by local people, finding methods to build the
capacities of community participants to cooperatively manage their own
development programs remains a challenge for practitioners. While new
methods, known as participatory rapid appraisal or PRA, do encourage
community members to collectively assess their situation and think about initial
program options, they are rarely utilized on an on-going basis.

ILS aims to overcome many of these shortcomings. It is designed to assist poor
women borrowers and NGO staff to think critically about micro-finance program
impact and processes and make strategic changes in a decentralised,
participatory manner. Participants use the system at each organisational level in
a development program, from the poor participant, to participant groups, field
officers, and program managers. ILS users at each level, especially poor women
borrowers, are the first to learn about program impact and performance, and
alter plans as a result. They are not only data gatherers but they are also the
data analysts, planners, documentors, and trainers.

The medium of the system is the ILS pictorial diary. It is nothing more than a
survey questionnaire in pictorial format, which participants fill out on their own.

Structure and function of ILS

The structure of ILS is patterned on the structure of a typical solidarity or self-
help group micro-lending program that has a multi-tiered system of establishing
and strengthening village-based loan groups. The five levels include:

1.   Individual members of self-help credit groups(women borrowers)
2.   village group or centre (SHG group)
3.   area organisers of clusters of groups (staff)
4.   NGO credit program manager (manager)
5.   monitoring and evaluation staff of intermediary funding organization (funder)

The structure is flexible so that large minimalist operations may have additional
levels participating in ILS and smaller programs may opt to skip area organiser
or cluster level.
ILS was designed to meet the needs of the internal stakeholders by answering
some key impact and process questions of users at each program level.

ILS field result:

ILS has been successfully field tested among five different NGO organization
operating in Northern and Southern states of India. Participants, including very
poor and illiterate women, are able to keep an impact diary, recording changes
in their situation over time on program impact and process. The use of ILS
diaries by very poor and illiterate members, in particular, seemed to have a
strong catalytic effect of increasing women‟s confidence and motivation to
improve their situation. Women have also used the diary to analyse their
situation, set priorities and make future plans. NGO staff have also benefited
from learning about their members‟ lives.

Cautions in using ILS

If the NGO doing impact assessment is not interested in “improving” – finding an
fixing poor performance – or in building client understanding and problem-
solving skills, then doing ILS is a waste of time, resources and staff effort. A
one-time, external impact study is a more appropriate option if the goal of the
evaluation is instead to “prove”.

The NGO must also have a culture of “organisational learning” unafraid of
examining problems and poor performance in a constructive manner and finding
solutions. ILS is wasted if the NGO managers are intolerant of mistakes and
lagging results.

Attention should be given when designing the questionnaire. Questions can fail if
they are too vague or too general, are misunderstood by respondents, are asked
in an insensitive manner, or lack follow-up detail or probing, etc.

Analysis from data collected in questionnaires can fail if key contextual questions
are missed in the questionnaire design stage, such as age, education, household
size.

The ILS Resource Kit is being designed that will guide users around these pitfalls
guiding NGOs through the key steps in pictorial diary construction from an
analysis, as well as a design perspective.

Comments and questions from group discussion

Comment

One has to be really careful about selecting the appropriate pictures since the
kinds of questions to be asked and the kinds of pictures that they are presented
through might not always go together. Also, we should be careful about what
the local people will make of the symbols that are used in the pictures.

Question
What kinds of costs are involved in the implementation of ILS?

Answer

Most of the expenditure will be on staff time because clients will have to be well-
trained to use the diaries properly. However, the printing costs are negligible
(Indian Rs. 14 per copy) and since the pictures will just have to be drawn once,
artist costs will get smaller over time.

Question

How do you consolidate the results from the diaries so that they can be
ultimately analyzed?

Answer

All the information from the diaries is collected and entered into a spreadsheet,
which can then be used for all sorts of quantitative analysis.

Question

How often do you need to revise the diaries?

Answer

Since indicators change quickly over time, the booklet also has to be updated
often. The diary has to be adapted to the local area, so that the pictures reflect
local symbols.

Question

Do people have enough time to fill out the books? Isn‟t asking so much time
from clients transferring the costs of the NGO into the clients?

Answer

It may seem like that is the case, but clients have benefited from the diaries as
much as the NGOs. The diaries have been a symbol of empowerment for so
many women who use them. Thus, both the participants and the NGOs benefit
from the tool.

Further Resources

Naponen, Helzi, “Participatory Monitoring and Evaluation – A Prototype Internal
Learning System for Livelihood and Microcredit Programs”, available through the
CGAP Gateway, www.cgap.org

5.6. Impact Monitoring

The difference between IA and IM is that while IM tries to select a few key
indicators and see how they in terms of impact by trying to look beyond
numbers, whereas IA is incorporates more indicators and is more comprehensive
than IM. IM tries to capture an overall understanding

Data collected on an on-going basis as part of normal operations. Usually
integrated into existing loan-officer work patterns and management information.
May be a combination of quantitative and qualitative data, but generally a very
limited number of simple indicators collected

Notes

Impact Monitoring: part of something bigger and not on its own

   Monitoring what you are doing to improve the programs you are doing. (to
    learn and improve your programs).
   Understand the process underlying the data that are received from the
    indicators. (data + process)
   A small no. of indicators and regularizing them in the org.
   How do things fall in the model of development? Indicators by themselves
    don‟t help you understand and change the way you work.
   Gives a sense of variation, you are not restricted to a place

6. Comments on field trip

All the participants were divided into three groups which worked with different
groups in the field. Each group decided its own goals of the trip and used the
tools that it thought were appropriate to achieve its goals. The following are the
comments on the field trip:


Group I

It decided to test IA at the individual, household enterprise, and SHG
organization level. In order to do that, it used time line and focus group
discussion. At the individual and household enterprise level, it looked for self-
confidence, income, asset-base, control of resources, education of children, and
specifically on women, girls, and boys. At the SHG-level, it looked at the group
dynamics, peer-relations, solidarity, support, leadership, community and
resource management under the leadership, and innovative and initiative ways
of conflict resolution. The objective was to look into their perception of benefits
and non-benefits and their perception of how, why, and what has changed in
their lives. The tools it selected were the empowerment tool, Loans and Savings
tool, timeline, and focus group discussion.

Some of the constraints that they felt about the timeline tool were time and lack
of appropriate field-skills. When they were told to disguise, they came out
briefly. That may not have been the representative of what they had done in
some cases. Some more time was needed to just go through the process and to
think about the past. When they were asked to describe their conditions in the
past, they could not remember details regarding assets and liabilities. Most of
the villagers expressed change mostly in economic terms only.
The results from the empowerment tools were very satisfying. The villagers
looked for being educated, being able to teach other people, and give advice to
them as evidence of empowerment. At the same time, they also said
empowerment of only women is not development in the real sense. It has to be
a combination of whole family working towards development.

It also observed a weakness of questionnaires while using the structured
questionnaire of savings and loans tool. Although one gets direct answers for
direct questions, one does not get any opportunity to divert into problems or talk
about the underlying processes, reasons behind the responses.

Another thing that was noticed was that asking questions about the socio-
demographics of the area facilitates the whole process by creating a suitable
environment for interaction between the field-worker and the villagers.

This group wanted to find out the income, expenditure, and repayment abilities
of individuals. For that purpose, it used the matrix, which is one of the
participatory tools discussed in the seminar. The group realised that although
explaining the matrix to the villagers was initially difficult, the tool worked well
after some repeated and detailed explanation.

Group II

The first sub-group wanted to find out if participation in the microfinance activity
had improved the cohesiveness and problem-solving capacity of the group of
which the villagers were a part. In order to do that, the group decided to use
two different tools to see if they would receive different results. The first tool
was a participatory tool that had pictures of faces ranging from happy to sad.
The women had to vote below the face for how they thought about the
cohesiveness of their group. The second technique was another participatory
tool called the “ladder technique” in which the group climbed up the ladder as
they thought they had fulfilled the criteria for each step in the ladder. While
most of the participants voted for the “neutral” face in the first technique, most
of them thought that they had crossed the seventh of the ten steps in the
ladder.

The result of this group suggests that when using participatory tool, trusting
only one tool might not give the most accurate results. Also, the group found it
difficult to explain the meanings of faces to the villagers. Perhaps trying other
method to vote would have been easier to explain.

The second sub-group wanted to find out the changes in the lives of the clients
between five years earlier and now by drawing pictures. Although CARD,
Philippines had mentioned earlier in the seminar that it is a very effective tool,
this group came across several barriers. Language was the first and the biggest
barrier. Since none of the group spoke or understood the local language well,
they had a difficult time explaining the method to the clients of the program.
Secondly, since some of the clients were not used to writing, they were reluctant
to use a pen and draw.


Group III
One of the sub-groups of this group decided to measure household incomes and
assets by using three techniques: social mapping, focus group discussions, and
transect walk. Transact analysis was very useful from a serendipity point of view
but was time-consuming and somewhat unproductive since it could only cover a
small portion of the village. Social mapping was more effective since it was more
participatory and visible. The focus group discussion allowed a close interaction
with the clients and ask detailed questions about the clients‟ past in a friendly
environment. The group realized that the three tools were complementary to
each other and were a wonderful way of getting a very real feel of the socio-
economic fabric of rural life on the very first day of interaction with the clients.

The other sub-group used the Savings and Loan Use tool of the AIMS tools. They
came across a problem that was previously mentioned while talking about the
AIMS tools. They noticed that only ten of the thirty-eight questions were
relevant to their specific. The lesson to be learnt from the issue is that most of
the tools have to be customized for local use. This is also a problem with many
donor-created or “pre-packaged” IA tools: they do not take into account local
knowledge and situations.

7. BRAC’S Presentation

BRAC, formerly known as Bangladesh Rural Advancement Committee, was
established as a relief and rehabilitation organization in 1972 after Bangladesh
Liberation war by Mr. Fazle Hasan Abed. With the twin objectives of “poverty
alleviation” and “empowerment of the poor”, BRAC has gradually evolved into a
large and multifaceted development organization. BRAC mainly focuses on
women and has developed it‟s own strategy to take resources and services to
the grassroots level. Following a holistic approach, BRAC has initiated a number
of multi-dimensional programmes for economic, social and human development.
It works in all 64 districts and covers 3.8 million households. Thus, BRAC has
national level outreach and offers different programmes and support services.

Impact Assessment Survey - IAS- I (1993-94):

Objectives

The two basic objectives of IAS-I are to gain more understanding of the socio-
economic impact of RDP in both quantitative and qualitative terms and to assist
BRAC in developing its capacity to identify an appropriate methodology to assess
different aspects of BRAC‟s impact.

Considerations

Undertaking IAS-I considers both organizational learning and donor demands.
Since the objectives of BRAC in poverty alleviation and empowerment of the
poor are also national priority issues, both institutional and national context are
influenced by IAS-I.

Who were involved
A large number of people were involved in doing IA. The Research and
Evaluation Division researchers and external consultants were involved in
designing, conducting and finalizing study report. Enumerators were recruited
for data collection. Programme managers, field staff and external experts were
involved for designing and studying draft report.

Methodology

Integrated methodology was used for conducting IA. Both quantitative and
qualitative data were collected through Household Survey, Village Profile and
PRA/ Case Studies.

Indicators used

Four broad set of indicators were used in the process:
 Material well-being
 Seasonal vulnerability and economic security
 Changes in women‟s lives
 Development of VOs (institutional building)




IAS – II (1996-97)

Objectives

The broad objectives of IAS-I were retained in IAS-II. The added objective was
to help evolve an impact assessment system for BRAC / RDP.

Considerations

The considerations for IAS-II were to develop an impact assessment system and
to create a benchmark database for RDP phase IV. It also included measurement
of poverty, which was a national level concern.

Who were involved

BRAC researchers conducted the study. One external consultant helped in
designing the study. BRAC field and management staff helped in selecting the
indicators. Trained enumerators were recruited for field study.

Methodology

Some changes were incorporated in IAS-II. More focus was added on poverty
reduction impact by measuring poverty and correlates. Cost of basic need (CBN)
method was also used. For empowerment, Chen and Mahmud‟s conceptual,
framework of pathways matrix was used. The new feature of IAS-II was the
triangulation of data or results from survey and PRA method.

Tools
Broadly IAS-I tools were used. PRA tools were also included such as Group
Discussions, Physical Mapping, Wealth Ranking, Time Lines etc.

IAS-III (2000-2001):

Objective

The objectives of IAS-III were to measure the impact on poverty level including
socio-economic well-being, to measure the sustainability of VOs and institutional
building, to measure village level impact of RDP / NGO programmes. Besides
these it also had the objectives of covering wider range of programme areas,
cost and time reduction and develop an understanding of the needs of the
extreme poor.

Who were involved

RED researchers, BRAC management or field stagff and programme participants
were involved for doing IAS-III.

Methodology

A new type of integrated approach was used in IAS-III, which consisted of the
Household Surveys, Village Profile. Other studies were also done covering
selected indicators. The methodology incorporated synthesizing results of
recently completed studies and integrating all results into the IAS.

Lessons learned

As various methods are practiced, some lessons are learned from the
shortcomings that were encountered. With the practice of different methods they
are moving from material and social well-being measurement to poverty
measurement, from consultant guidance to organization staff doing their own
study and from impact on participants to village level impact. As a blend of
different methods were used, the scope of participatory impact assessment
gradually widened. The impact study moved to become the impact system.
There has been an increase in time and cost consciousness and coverage of
impact areas also widened. A trend towards greater recognition of national
context was found. Each IAS led to some new areas of enquiry.

a) There are strengths and weaknesses of both survey method and participatory
   method.
b) No uniform or rigid method may be followed.
c) Learning from experience may refine the methods.
d) IAS allows program to amend policies and start follow-up for greater
   sustainability and more effective impact.
e) Cost / time considerations depend on the capacity of the organization.
f) Very often both the quantitative and qualitative methods need to be
   integrated.
g) Survey method ensures objectivity. It yields quantitative data amenable to
   statistical analysis especially where it is necessary to isolate the effect of a
   single variable from the composite effects of multiple variables.
h) The participatory method is used to deal with subjective issues such as
   perceptions, attitudes and motivations.
i) It is often necessary to provide explanation of facts derived from quantitative
   method for which the qualitative method needs to be used.
j) Thus, both methods may complement each other in serving the purpose of
   an impact assessment study.

8. Quantitative Issues in surveying:

Different stages in quantitative surveying:

   sample design
   instrument design
   data collection
   quality control
   data entry
   checking
   analysis

The starting point should be analysis! A major problem in quantitative research
is that lot of the data collected is not used at all. Thus, when designing and
conducting research, one has be parsimonious in the use of resources by putting
thought into why the data is being collected.

   Difference between quantitative and qualitative research: There is a tendency
    to think that participatory methods are qualitative and survey methods are
    quantitative. Many of participatory methods provide quantitative results as
    well. The real distinction should be survey and participatory methods, rather
    than quantitative and qualitative methods.

   Choice of method: Partly because of the enthusiasm behind participatory
    methods, there is an unnecessary hostility towards survey methods. Different
    methods can blend to give a holistic picture, in fact they can be
    complimentary. The purpose of the study and the audience of the analysis
    also determines how precise your data has to be and thus defines the
    appropriate choice of method.

   The real distinction between quantitative and qualitative methods is
    objectivity vs. subjectivity. Researchers who want their research to be
    quantifiable, unbiased and easily replicable by others to receive the same
    results prefer quantitative methods. However, if you are asking an individual
    for his/her perceptions of impact, then the response will be localized,
    depending on your relationship with the individual and may differ from the
    response another interviewer might receive at a different point in time. One
    should decide between the two approaches depending upon whether you
    want to understand how interviewees react to/feel about impact or you want
    to come up with numerical estimates of impact in order to attribute changes
    to the program.

   Origin of participatory methods: Antagonism towards old development
    studies that were marred by tremendous amount of energy spent into
    collecting data without any results coming out of the studies even years later.
   Participatory methods initiated by Robert Chambers emphasized rapid (vs.
   old time consuming) analysis of results.

Control Group

This is a complicated issue that came up several times during the seminar and
initiated lots of dialogue. Questions such as “do we need a control group?”, if so,
“How do we get one?” were raised often by participants. If you are concerned
about the issue of attribution, about the impact of YOUR program versus other
simultaneous factors, then you cannot avoid the issue of control group.
However, if you want to know how a SHG has changed over time or to
understand the broad changes going on in an area due to development efforts,
then a control group is not necessary. Control group involves observing changes
in two groups, one with which the program is working and another with which it
is not. Only if the control group is genuine(similar in all aspects), do you have a
basis of attribution. However, achieving the goal of genuine control group is
virtually impossible.

Selection bias often mars the selection of control group. NGOs often tend to
work in areas that are easier to work in. However, if the control group happens
to be an area that NGOs find unfavorable to work with, the comparison will not
be fair. Similarly, participants that are more likely to succeed in life always self-
select into the treatment group. Although traits such as drive to succeed and
initiative cannot be measured, they might be the reason behind the different
changes in the two groups. You cannot control for many things but not for
everything.

One of the participants also mentioned that a control group is not absolutely
necessary to establish attribution. Maybe well-done qualitative studies, with
techniques such as triangulation, can provide enough evidence to attribute
change to specific programs.

Another participant raised the issue of whether it is ethically acceptable, and if
so, for how long, to deprive members of the control group (which will most likely
be from a nearby village) of access to financial services. Perhaps a Comparison
Group (of people who are enrolled in the SHG but are waiting to receive loans,
rather than a Control Group which is completely outside the treatment group)
can be used as a proxy for a Control Group since the conditions for a
Comparison Group are more relaxed than a Control Group which requires
laboratory conditions.

Two other techniques were suggested as possible options to get around the idea
of control group:

a) Conducting a baseline survey at the beginning of the program and observing
   changes in the group through time is an alternative to control group although
   the problem of attribution still persists.

b) Panel data: Making repeated visits to households to monitor their
   socioeconomic conditions(by measuring program variables) so that you
   measure changes in individual households can get around the issue of control
   group. This can be especially useful in convincing economists! However, you
   must have an understanding of how your program could have caused the
   changes that you observed.

A good research will involve three stages:

1. Qualitative work

   The first stage of the research should be aimed at developing an
   understanding of the socio-economic conditions of the area through
   qualitative techniques such as diagrams, matrices, wealth ranking, social
   mapping. The goal of this process is also to develop relationships of trust
   with the villagers and familiarizing on both ends.

2. Quantitative work

   This is the core stage of the research project and involves selecting and
   adapting appropriate tools for the research and conducting the survey to get
   the data. This stage also involves doing the quantitative analysis of the
   collected data.

3. Comprehension via case studies

   Complicated issues and processes might underlie the changes that are
   observed through quantitative results. Therefore, qualitative methods such as
   FGDs, case studies, semi-structured interviews, and mini surveys can be
   used to investigate specific issues. Thus, this stage will explore specific issues
   for richer, more informed results.

Comments and Questions Group Discussion

The distinction of qualitative methods as subjective and quantitative methods as
subjective is not necessarily true. Although surveys can be objective in their
implementation, they can also be subjective in design: in the kinds questions
you ask/don‟t ask, in how you ask questions, etc.

There are also other places where subjectivity can creep into “objective”,
quantitative methods: how good the interviewers are, interpretation of results
(disciplinary bias of the researcher).

Is absolute subjectivity attainable? What is the virtue of completely objective
results?

One participant noted that there is no universal “best practice” package for
assessing impact. What has to be done is to find the right combination of tools
and to adapt them to specific circumstances and contexts. Ideally, a series of IA
would be ideal so that every organization can learn from its own impact
assessment and IA goes beyond the three year Ford Foundation project.

Another participant added that packages such as AIMS and MSA tools can easily
be substitutes for thinking; some of them are general and others work in one
context but do not in others. Thus it is crucial to develop an understanding of
your specific circumstances in order to develop the appropriate tools.
Another suggested that in trying to understanding the context of the area being
studied, one should focus not only on local circumstances, but also what is
happening in nearby villages, macro policies of the country, etc.

Another warned that it is too easy to be blinded by techniques and to forget
what goals you are trying to achieve through IA: to prove or to improve. Only
after the organization is clear about its goals, should it think about the
appropriate tools.

The final comment was made on the inappropriateness of donor-led IA since
they often neglect local contexts and do not take into account the opinions of
interests of local stakeholders. Since the practitioners have a local
understanding of the program, they should use IA to figure out what is
working/not working in their own groups.

9. Qualitative issues

This presentation was on gender, empowerment, and qualitative methods in IA.
Both empowerment and impact presume change. While impact can be both
positive and negative forms of change, empowerment entails only “positive”
forms of change. When designing impact instruments, the questions one asks
and data one collects are informed by some implicit or explicit model of change.
Thus, it is important to be clear about what model of change one is working
with. Only then can we develop concepts of empowerment and finally indicators
of empowerment.

The model of empowerment that was presented in the seminar was developed in
order to evaluate the measures of empowerment. But before reflecting on the
measures of empowerment, one needs to understand what empowerment
means. Empowerment can be defined as the ability to make choices. But there
are several qualifications to that definition. Firstly, empowerment requires a
change in the ability to make choices. Secondly, since everyone makes choices,
empowerment should involve life choices about strategic aspects of life. Thirdly,
empowerment should also not infringe on the rights of others to make choices.
Fourthly, for the choice to be real, a significant alternative should be an option.

Empowerment can be thought of as a three-dimensional model:

                                 Resources
                                    |         \
                                    |        \
                                    | Achievements
                                    |       /
                                    |           /
                                     Agency

From the above model, it is obvious three kinds of intervention of choices are
possible in life:

a) Resources
Resources can be individual or collective. The absence of resources constitutes a
constraint. Three are three kinds of resources: human resources such as health,
physical well-being, education, knowledge, cognitive skills, leisure, sleep;
material resources such as income, assets, savings, equipment, livestock,
money, new livelihood opportunities; and social resources such as family
relations, extra-family relationships, social standing in community, networks,
access to institutions, friendships, group formation, group federation.

b) Agency

Agency is the ability to define one‟s goals and to act upon them. Agncy can be
individual or collective; hidden or open; negative or positive. The absence or
negative exercise of agency constitutes a constraint. Agency can involve factors
such as decision-making, mobility, economic participation, social mobilization,
and voting/campaigning.

c) Achievements:

Achievements or outcomes can be individual or collective; they can relate to
changes in resources or changes in agency; they can be changes in absolute or
relative measures such as improved levels and changes in gender gaps in
nutrition/ education/ health status, reduction in levels of violence within home,
reductions in levels of violence outside the home, and improvements in mobility.

Levels of change

Conceptually, there are three levels of change:

   individual (which involves dimensions such as resources, agency, and
    achievements),
   intermediate (which involves dimensions such as institutional roles, norms
    and practices),
   a third “deeper” level (which includes structures of constraint such as class
    and gender).

In terms of indicators, the following are possible clusters of indicators:

   Individual
   Family/household
   Enterprise
   Group
   Wider impacts: local level (markets, community and government)
   Wider impacts: national level(markets, society and state)

Another way to think about clusters of indicators is through domains of change.
The following are possible domains of change:

   Social
   Economic
   Political
The onus lies on individual organizations to work out the most likely kinds of
change, the appropriate level of change and the appropriate indicators to be
measured.

Mapping Causality

Causality is attribution by another name. It is about how you explain change.
Can you attribute things to change that are happening anyway or can you
attribute the change to the intervention of a particular organization. This is also
an area where quantitative methodologies often tend to fall down.

Economists believe that individual actions are motivated only by self-interest to
maximize utility or profit. In such a world-view, any change is attributed to
either change in prices or a change in the constraints that prevent people from
acting as self-interested individuals. Any instances of altruism are said to belong
to the family where market relations do not prevail. If an economist does an
econometric model of impact evaluation and finds out that giving loans to
women and men results in very different decision-making outcomes, they will
assume that it means empowered women. Although men and women in a
household have different preferences, since women do not have any power,
men‟s preferences will prevail and will shape the decision. If you now have
evidence that giving loans to women results in a different pattern of decision
making, it will mean that women are empowered.

This kind of analysis is not sufficient to establish empowerment since it does not
find but only infers the meaning of empowerment. What we need in such
situations is additional information to explain the indicator. Such additional
information is provided by qualitative exploration.

There are three links in the chain of causality: observed change, intermediate
causality, and the underlying causality. Examples of chains of causality:

1. Immediate change: reduction in domestic violence
   Intermediate causality: Husband‟s fear of public exposure
   Underlying causality: Women‟s access to extra-household social networks

2. Immediate change: reduction in domestic violence
   Intermediate causality: Reduced anxieties about daily necessities

3. Underlying causality: Women‟s increased economic contribution resulting
   from access to loans.


Methodological Considerations

The advantages of quantitative methods are that they identify empirical
regularities, measures correlation, infers causality, can be generalized,
document incidence and magnitude of occurrence, so distinguishes between
“rules” and “exceptions”.

Qualitative indicators, on the other hand, can interpret quantitative indicators by
analyzing the causalities behind the regularities seen through quantitative work.
Correlations by themselves do not tell you about causality. Qualitative work is
one way of exploring the direction, process of causality or whether there was
any causality. It helps to inform quantitative information by exploring meanings
and subjective change.

   Kabeer, Naila, “Money Can‟t Buy Me Love? Re-evaluating Gender, Credit and
    Empowerment in Rural Bangladesh”, IDS Discussion Paper no. 363 (1998),
    available on web site www.cgap.org.
   Mayoux, Linda, “Micro-finance and the empowerment of women - a review of
    the      key      issues”;      ILO,    available     on     web      site
    http://www.ilo.org/public/english/employment/finance/papers/mayoux.htm

10. Future plans of partner organizations:

10.1. PRADHAN:

PRADHAN plans to use the following methodologies for the remaining stages of
the project:

1. Household-level structured questionnaire to obtain initial data that is
   quantitative and ranges across ages and social groups.
2. FGDs to reinforce the results that are received from the household
   questionnaires and the quantitative results.
3. PRA, FGD, case studies to reinforce the results obtained from the above
   proceses.
4. ILS diary at the group, staff and organizational level to incorporate the
   internal learning system into the program.

PRADHAN wants to measure impact all the following levels: individual, group,
institution the society at large. At the individual level, it wants to measure the
following indicators:

   Census
   Livelihoods
   Enterprise
   Assets
   Credit
   Savings
   Coping strategies
   Diet/food
   Housing/sanitation
   Income – expenditure
   Access to services
   Decision-making
   Participation/ access to public areas

Control group

As the Control group for their project, PRADHAN plans to use members that are
a part of their of their programs but have not yet received loans. Their only
concern with this technique was that the control group will get smaller through
the process of the project and might not even exist after a year or two.
Question and comments from discussion

Question

One participant expressed a concern that since structured questionnaire is the
central part of the PRADHAN‟s methodology, such a methodology might
contradict PRADHAN‟s philosophy of making its programs more participatory.
Hence, PRADHAN should perhaps try to think about how it can capture the
answers through participatory tools for the same questions as in the structured
questionnaire. Although the format of IA may be different and will be more time-
consuming, PRA tools can obtain the same answers as questionnaires.

Response

PRADHAN responded that it was aware of the concern mentioned above. Its goal
of the structured questionnaire was not only to help build the ILS, but also to
substantiate and complement the ILS tools that they are planning to implement.


10.2. CMF

As an institution that is concerned with capacity-building and strengthening of
the microfinance sector in Nepal through action and research activities, CMF has
done the following activities in between the Kathmandu and Bhuwaneshwor
workshops:

   Different organizations approached CMF to do certain surveys and studies.

a) Assessing the poverty level of the clients of one of the leading MFI of Nepal –
   NIRDHAN by CGAP using the tool called poverty assessment of MFI clients.
b) Research on risks and coping strategy of rural women of Nepal and
   specifically SACCO women using AIMS tool.
c) Assess the poverty level of the ex-bonded laborers in Nepal by ILO again
   using the CGAP tool, Focus Group Discussions, PRA.
d) Field work for the evaluation of Women Empowerment project (WEP) run by
   PACT by using the revised Impact Survey tool.

   Learned the strengths and weaknesses of activities in the process.
   Used PRA, FGD, Social Mapping, Time Line and Seasonality of income.
   Also used the Institutional Audit tool.
   CMF itself developed a tool called PAVE (participatory assessment and
    visioning exercise)

Since the last workshop in Kathmandu, it has done the following activities
relating to the IA project:

1. Meeting with different SACCOs and their promoters. Identification of
indicators with them. Prepare, revise and finalise proposal with feedback from
the SACCOs and the UK team team. Proposal approved and signed.
2. Two consultation meetings with SACCOs
3. Conducting of different researches using the following tools
  CGAP tool on poverty assessment
  AIMS tools on Saving and loan use
  Aims tools on Impact survey and Empowerment
  A combination of CGAP poverty assessment and FGD to identify the
indebtedness of the ex bonded labourers for ILO
 PRA – social mapping, FGD/time line/ seasonability
 Wealth ranking

All these different methodologies have been field tested for different studies.
Which gives CMF in-depth knowledge of their strengths and weakness.

4. Selection of four SACCOs
5. Institutional Audit has commenced
6. Regional workshop on Methodology: revision of the different tools used and
feed back from Naila and Helzi.

CMF‟s TA needs are as follows:

   Electronic feedback from the UK team and our Asian partners.
   More extensive PRA training to one person each from SACCO.
   TA to finalize the whole questionnaire, the methodology package
   TA to develop the data directory to place more emphasis on the analysis part
    of the data.
   Practitioner level resource at the field to cross check.
   Academic feedback.

Questions and comments from participants

Question

Very strong emphasis on developing capacity of SACCOs to analyze their own
progress. Both the questionnaire and the qualitative research are external
research. So, how do SACCOs evaluate their own progress?

Answer

Institutional audit called PAVE, will be implemented by SACCOs and their
members. Each SACCO has their own MIS system and CMF will help them to
develop these systems and this will be included in the research.

Question

Institutional audit help in the assessment of the strength of the SACCOs. Does
that include the SACCO members evaluating the impact of the SACCO on
poverty and empowerment?

Answer

There are certain tools used. Some of them are to assess the capacity of the
SACCOs. Some other are only for its members to assess the service they are
receiving. Some tools focus on the assets and property of the very poor. Even
the questionnaires have certain aspects, which can be strengthened and
changed. The methods or exercises involve all the SACCO members as the part
of the process. They are exposed to all the tools and their use and they can
choose the valuable tool from them. Part of the capacity building is to use the
tools and part is to work with the SACCOs.

10.3. CYSD

The basic purpose with which the study will be launched are as follows:

   To incorporate an impact monitoring system within CYSD.
   Assessing its impact on women, i.e. to see the empowerment of women from
    the perspective of the empowerment model.

Strategies

   Flexible as per the requirement of time.
   Choose four geographic locations representing the urban, rural as well as
    tribal location and maximum disaster affected area also, where SHG as an
    intervention can have some impact.
   Use stratified random sample size, and cover 15% to 20% of the SHGs
   Membership age to be another criteria for choosing the groups.
   Collect information at the individual, household, SHG and community level.
   Kind of information to be collected: income level changes, skill level changes,
    loan use, resource base and assets.
   Also intangible information such as the self-esteem, confidence and abilities
    to manage their own things by themselves.

Methodology

   Both qualitative and quantitative
   Look at the Internal Learning System (ILS) more closely for the
    institutionalization of the impact monitoring system and develop the whole
    system.
   Look at the tools and adopt some of those which fits into the context.
   Also do some of the PRAs like focus groups, case studies.
   Reconstruct the memories and have a control book whenever possible to
    make a comparison of the data.
   If not possible, reconstruct the baseline information.
   Longitudinal also in the sense of new groups.

Process

   Start reviewing of baseline information and project location by August.
   Focus group discussion with the elected SHG members from all location to
    have an idea of the changes and also brainstorming within the staff members
   Design the tools, questionnaires by taking clues from the FGD by November.
   Start first information collection and send a draft for feedback by January.
   Launch the whole process.

Technical Support needed

   Help for information compilation and data analysis.
Questions and comments from participants:

Comment

CYSD should perhaps use new/old members instead of the control groups for
comparison because they cannot have control groups most of the times. This is
problematic in urban areas. Addition of control groups implies addition of more
data. CYSD and CARD should dialogue a lot since CARD is really interested in
empowerment.

Different kinds of people involved and different possibilities that are there and
also how we perceive things create a number of tensions in the whole program.
But people can achieve a lot of things from the program:

   Working with the existing tools
   Using it to build up participatory methodologies
   Using all the different tools to do something you don‟t usually get the
    resources to do – Impact Assessment.

Different methodologies enhance this process. Etic and Emic are anthropological
methodologies. Emic explains about the values within the group and Etic is the
outsider‟s standpoint perspective. But however, both these methods provide
different perspectives on the same realities. Lets find out what we can actually
learn and achieve.

Comment

Every institution goes through certain process and at the end of it all comes up
with the fruits of the research like the data. Each one is very concerned about
how to make the research as much participatory with the clients since it is not
„donor led‟. At the same time, institutional strengthening should also be given
high priority.

Comment

This is an opportunity for CYSD to learn, understand impact and think about
what they are doing. Another crucial thing that they need to do is to develop
their process and plans and also to document the whole process. The process of
IA is not just doing one-thing and finding results but a process, which should
integrate a whole lot of other things.

10.4. CARD

CARD intends to conduct an impact assessment of the market research activities
of microfinance institutions in the Philippines. For that process, it will first
implement a pilot project on management information system after extensive
consultative dialogues with local microfinance institutions on recent trends in
Filipino microfinance institutions. The issues it is expecting to encounter are as
follows:

   Cost of impact assessment
   Capacity of MFIs to conduct IA
   Existence of baseline data
   Selection of appropriate indicators for IA
   Integrating IA into the project rather than letting them be controlled
    externally

For this pilot project CARD is planning to work with five MFIs in different parts of
the Philippines. In order to make the process as participatory as possible, CARD
will extensively involve MFIs in the design and implementation of the project.

Another goal of CARD is to empower Filipino women. CARD will conduct another
IA in order to examine if its activities are achieving this goal. It will need
technical assistance for the following activities:

   to determine the model of an empowered landless Filipina woman
   then discuss indicators
   and questionnaires that can be used to measure them.

Since IA is a relatively new thing in the Philippines, CARD is hoping that its two
projects will help institutionalize IA in the Filipino microfinance institutions.

11. Regional planning for what next:

Wider Impact

Paul Mosely/Anup – Bangladesh January

Global Steering Committee

Namrata Sharma was nominated to represent Asia in the global steering
committee of Imp-Act. When she asked if anyone else wanted to represent Asia.
However, Namrata Sharma was adopted unanimously to represent Asia.

Global meetings

New Asia Partners

The Ford Foundation has suggested that Bangladesh and China be included in
the program. BRAC and BIDS from Bangladesh have been selected as partners
from Bangladesh and work is on the way to include one MFI from China.

What next

With similar tools being used by most of the participating organizations, the
participants expressed the need for regional sharing of experience at the end of
the seminar. One way to do that would be study tours inside each region so that
organization can work together to learn from each other. When question was
raised about whether funding exists for such programs, the organizers
mentioned that funding for networking activity is within the budget of the
program and can be utilized when appropriate. The participants also suggested
sharing of individual work-plans so that everyone knows what others are doing.
Sharing would be especially useful with questionnaire since the same tools are
being used by most organizations. The list-serv was suggested as an ideal
platform for partner organizations to inform others of not only their needs and
plans but also preliminary findings from their study. The participants also
suggested that the absence of some experts such as the AIMS and MSA tools
was a disadvantage, although the presenters of those tools had sufficient
experience using them.

12. Final comments on the seminar

At the end of the workshop, the activities were evaluated and
comments were given by the participants individually.

1.    Ratnakar Panigrahi (CYSD):
      I came here with an empty bag but while leaving it is overflowing with
      ideas, concepts and things to do. The workshop has provided me with an
      insight between the options that are there regarding the tools and
      ideologies. All in all, the workshop was able to raise quite a few concerns
      in my side. This will definitely go a long way in improving our program
      and contributing to the global process. I hope I do have more things to
      share the next time we meet.

2.    D. Narendranath (PRADHAN):
      This workshop has given me and hopefully my colleagues here, a clear-cut
      sense of direction as to how to go ahead. The workshop could finally meet
      the long felt need of an effective workshop on methodologies that had
      been expressed in many forums. The workshop provided an exposure to a
      variety of tools though not expertise on any of them. Now, the issues are
      to collate all these things together so that an in-depth understanding of
      participatory methodologies could be developed which was just touched
      upon. The way we are planning to go ahead includes installing a well-
      designed and systematic process in our organization.

3.    Sukanta Sarkar (PRADHAN):
      For PRADHAN, the workshop has given some insights about participatory
      methods that were practiced by us earlier in bits and pieces. I think the
      challenge lies on how to make an inbuilt system into the whole process
      and to practice participatory methodologies in a more methodical and
      systematic way.

4.    Martin Greeley (IDS):
      Firstly, I wish we could organize workshops the way Namrata runs them. I
      think we have all benefited from that a lot. Secondly, I had some slight
      hesitation about how we planned the program, whether we had put too
      much time on talking tools and not enough time on talking the program or
      what individual institutions are going to do over the months/years to
      come. That was my own worry regarding the program plan.

5.    Anup Dash (CYSD):
      This was my expectation: From a very sad, discontented person to
      someone who is happy and contented. And I think my expectation has
      been fulfilled to a large extent. The formal intellectual debates were very
      productive and the fact that people, despite of their busy schedules, saved
      time to come together and share ideas should be appreciated. From the
      perspective of organizational processes and logistics, we try to do our best
      with modest means. At the same time the informal social gatherings
      including the evenings, jokes, talks could bind all of us into an IMPACT
      family of which the UK team is a prominent member. We look forward to
      increased solidarity in this family.

6.    Ani Rudra Silwal (CMF):
      I came into this workshop with very little experience on microfinance and
      impact assessment. So, I have rather been an observer than a
      participant. I always had somewhat negative opinion of most NGOs
      working in microfinance. My impression had been that most of the time
      they are donor activity oriented. The fact that the seminar tries to
      integrate IA as a part of organizational activities and methodologies
      impressed me a lot. Another thing that I liked about the workshop is that
      rather than working on the tools themselves, there was more emphasis on
      why should we use such tools? What are we trying to gain through these
      tools? I am glad that was done.

7.    Navraj Simkhada (CMF):
      IA workshop has been proved to provide an understanding of appropriate
      tools to be used by organizations. I think, selection of tools and proper
      use of tools are two important aspects of IA. However, some problems
      may arise over the course of time regarding selection and use of tools in
      various places or in fields. We hope that the partner and UK team will be
      here in the whole process to clarify issues that keep coming up.

8.    Namrata Sharma (CMF):
      As far as CMF‟s need from the workshop is concerned, I would say that we
      have achieved what we came here for. At the same time, we have also
      been alerted to some of the points we have as part of our organization
      that has to be integrated in the whole process. The best thing we‟ve
      achieved from the workshop is the strong, cohesive team building spirit
      that would be the strength to go along for three years. A mutual help
      from each other and not only from the UK team is required in the whole
      process for effective work.

9. Anton Simanowitz (IDS):
     When I arrived here I was really apprehensive and scared about this
     workshop because all the planning had just taken place by emails. We had
     some consultation but to have a process that works, actually meets the
     expectations is a real achievement and it has been achievement for
     everybody here. It was a success because the logistics, the hotels, the
     food the fields trip, everything worked. What I don‟t like is the financial
     part i.e. the budget that is held in UK. However the workshop was not UK
     based program, one where we are not trying to control and tell people
     what to do. I think the social aspect and the sense team spirit helped
     everybody a lot.

10.   Aniceta Alip (CARD):
      Well, I came here with the idea that we have the experience already and
      we have a lot to share. I came to realize that we are just starting as
      compared to you because when we go back we have to start with our
      indicators. So, this workshop is crystallized for CARD. The wealth of
      experience and expertise that I heard over in the last 5/6 days is
      enormous and I think it will take me a whole week to sort things out.
      Also, I really appreciate and applaud how Namrata was able to
      exceptionally moderate all of us. I also thank the CYSD team, hovering in
      the back and anticipating our needs.

11.   Mary Florence (SHARE):
      Personally for me this workshop has been a great learning and very
      fruitful experience. I came here with very little knowledge on IA but yes I
      am going back proudly, with a spirit of doing something great on IA. I
      would like to congratulate the CYSD team who has exhibited a great team
      spirit. On behalf of SHARE, I take this opportunity to thank the IDS team
      and the regional coordinator for giving us the proud advantage of being
      here.

12.   S. Somayajulu (SHARE):
      Congratulations to madam Namrata for being the regional coordinator.
      Already we have some experience of IA in SHARE. That time we used the
      AIMS tool which we thought was the only way to measure the impact in
      an organization but after participating in this workshop, I learned about
      many tools to measure the impact of an organization. I am thankful to the
      CMF team for organizing such a fruitful workshop.

13.   Lalaine Joyas (MFC):
      In terms of IA in general, this workshop has been very enlightening in
      terms of a number of issues and sharing of experience among institutions
      and partners. This makes us reflect on the challenges ahead. In terms of
      moving on, I think the group work was very helpful as it gave direction
      and a clear idea of how to move on. We thank the UK team and the
      others, of course, in this respect. As far as the microfinance council of the
      Philippines is concerned, I guess this was a very good opportunity to
      establish relationships and collaborations in South Asia. I look forward to
      more collaborative work with you.

14.   Parashuram Nayak (CYSD):
      Though I am a hard-core banker, this workshop provided me the
      opportunity to meet the participants individually and so got a lot of insight
      on microfinance itself, which was not much talked about. Unfortunately
      the workshop lacked group discussions where all the participants could
      have easily participated. Otherwise this has been an excellent experience .

15.   Binaya Kumar Rout (CYSD):
      I am happy with the workshop. It has helped me fulfill my expectations
      and get some clarification on methodological issues.

16.   Geetanjali Jena (CYSD):
      Personally, this has been a great learning experience for me particularly
      the exposure to the tools. Organizationally also, this has created a sense
      that we have to move. So it is a beginning and there‟s definitely a
      direction where we are going to move. In fact, we had very little time for
      really using this kind of opportunity where there are many resource
      person available. Some homework prior to such group interaction would
      be very helpful in my opinion.

17.   Namrata Sharma (CMF):
      It‟s been wonderful coming here together and all the participation going
      on. My heartfelt thanks to all the participants, partners and also to my
      colleagues back home in CMF. The biggest thanks go to CYSD.

18.   18. P.K. Sahoo (CYSD):
      Sahoo gave a vote of thanks to everyone who participated in the seminar
      and helped organize the seminar and formally declared the end of the
      workshop.
Annex:

Agenda of the Workshop:

July 4, 2001

9:00-9:30      Inauguration
10:00-13:00    AIMS tools (CARD/CMF)
14:00-16:00    AIMS tools
16:30-19:00    MSA tools (CARD)


July 5, 2001

9:00-11:00     MSA tools
11:30-13:00    Internal Learning Systems by Helzi Naponen
14:00-16:00    Impact Monitoring/ Participatory tools (Anton/CYSD)
16:30-19:00    Preparations for field trip-CYSD


July 6,2001

8:00-evening         Field trip and feedback/review of field trip.

July 7, 2001

9:00-11:00     BRAC Presentation
11:30-13:00    Quantative Methods/Surveys (Martin Greeley)
14:00-16:00    Gender, Empowerment, and Qualitative Issues           (Naila
Kabeer)
16:30-17:30    Approaches/tools review
17:30-19:00    Group work


July 8, 2001

9:00-11:00     Group work
2:00-6:00      Presentations and feedback


July 9,2001

9:00-11:00     Way forward-commitments to trying out tools from the
workshop
11:30-13:00    Way forward in the region.
    List of participants

   Name of the participants   Organization Address                 Tel., Fax, Email
  1Dr. Martin Greeley         IDS          University of Sussex    Tel: +44 1273 678282, 691189 ®
                                           Brighton                Fax: +44 1273 621202
                                           BN1 9RE, United
                                           Kingdom                 Email: marting@ids.ac.uk
                                                                   Website: http://www.ids.ac.uk

  2Dr. Naila Kabeer           IDS          University of Sussex    Tel: +44 1273 606261
                                           Brighton                Fax: +44 1273 621202
                                           BN1 9RE, United
                                           Kingdom                 Email: n.kabeer@ids.ac.uk
                                                                   Website: http://www.ids.ac.uk

  3Mr. Anton Simanowitz       IDS          University of Sussex    Tel: +44 1273 678731, 728191®
                                           Brighton                Fax: +44 1273 621202
                                           BN1 9RE, United
                                           Kingdom                 Email: a.simanowitz@ids.ac.uk
                                                                   Website: http://www.imp-act.org

  4Dr. A.M.Muazzam Hussain BRAC            75, Mohakali            Tel: +880 2 9881265, 8824180 - 7
                                           Dhaka, Bangladesh, 1212 8015345®
                                                                   Fax: +880 2 8823542, 8823614
                                                                   Email: bracamr@bdmail.net
                                                                   Website: http://www.brac.net

  5Ms. Frances Sinha          EDA          107, Qutab Plaza         Tel: +91 124 6350835, 6351064®
                                           DLF - City 1             Fax: +91 124 6352489
                                           Gurgaon, Haryana, India,
                                           122 002                  Email: edarural@nda.vsnl.net.in
                                                                    Website: N/A

                                           2A, Alli Street, 10th
  6Dr. Helzi Noponen          ASA          CROSS                   Tel: +91 0 9843126845 (cell phone)
                                           Annamalai Nagar,
                                           Tiruchirappalli             +91 431 763980 (ASA office)
                                           Tamilnadu, India, 620
                                           018                     Fax: +91 431 763356
                                                                   Email: hnoponen@aol.com
                                                                   Website: N/A

participants:

  1Ms. Namrata Sharma         CMF          P.O. Box 20933          Tel: +977 1 434041, 432947
                                           Kathmandu, Nepal        Fax: +977 1 432947
                                                                   Email: namratas@cmf.org.np
                                                                   Website: http://www.cmfnepal.org

  2Mr. Navraj Simkhada        CMF          P.O. Box 20933          Tel: +977 1 434041, 432947
                                           Kathmandu, Nepal        Fax: +977 1 432947
                                                                   Email: navrajs@cmf.org.np
                                                                   Website: http://www.cmfnepal.org

  3Mr. Paul Bernard G. Lobo   PLAN         6/F, N&M Building       Tel: +63 2 8972745, 8971656
                                           1184, Chino Roces            +63 49 5367280®
                                     Avenue
                                     Makati City, Philippines   Fax: +63 2 8978358
                                                                Email: lobop@plan.geis.com
                                                                Website: N/A

                                     20 Quezon Street, City
 4Ms. Aniceta R. Alip      CARD      Subud                      Tel: +63 49 5624309
                                     San Pablo City,
                                     Philippines                     5621721® , 8003010®
                                                                Fax: +63 49 5620009
                                                                Email: card@msc.net.ph
                                                                        aralip@yahoo.com
                                                                        aralip@digitelone.com
                                                                Website: N/A

 5Ms. Lalaine M. Joyas     MFC       Suite B, Padilla Building Tel: +632 6312783, 7160332®
                                     Emerald Avenue, Ortigas
                                     Centre                    Fax: +632 6335904
                                     Pasig City, Metro Manila,
                                     Philippines               Email: microfi@pworld.net.ph
                                                               Website: N/A

                                     3, Community Shopping
 6Mr. D. Narendranath      PRADHAN   Centre                Tel: +91 11 6514682
                                     Nitibagh, New Delhi,
                                     India, 110 049             +91 141 232953, 338327
                                                                 +91 141 338327®
                                                           Fax: +91 11 6518619
                                                           Email: pradhano@ndb.vsnl.net.in
                                                                  pradhanjp@jp1.dot.net.in
                                                                  narendranath_d@yahoo.com
                                                           Website:
                                                           http://www.icicicommunities.org

 7Mr. Sukanta Sarkar       PRADHAN   Dhanbad Road               Tel: +91 6543 66284
                                     Baehi, Hazaroibagh         Fax: N/A
                                     Jharkhand, India, 825
                                     405                        Email: N/A
                                                                Website:
                                                                http://www.icicicommunities.org

 8Mr. S. Somayajulu        SHATRE    1 - 224/58, Rajeev Nagar Tel: +91 40 7158387, 7153603®
                                     Nacharam, Hyderabad      Fax: +91 40 7173558
                                     Andra Pradesh, India,
                                     500 076                  Email: sharemicrofin@india.com
                                                              Website: N/A

                                                                Tel: +91 674 300983, 301725,
 9Mr. Praful Kumar Sahoo   CYSD      E-1, Institutional Area    500729®
                                     Bhubaneshwar, Orissa,
                                     India, 751 013             Fax: +91 674 301226
                                                                Email: cysdbbsr@sancharnet.in
                                                                Website: http://www.cysd.org

                                                                Tel: +91 674 300774, 301725,
10Ms. Gitanjali Jena       CYSD      E-1, Institutional Area    441738®
                                     Bhubaneshwar, Orissa,
                                     India, 751 013             Fax: +91 674 301226
                                                                    Email: cysdbbsr@sancharnet.in
                                                                    Website: http://www.cysd.org

                                                                    Tel: +91 674 300774, 301725,
 11Mr. Ratnakar Panigrahi    CYSD         E-1, Institutional Area   522647®
                                          Bhubaneshwar, Orissa,
                                          India, 751 013            Fax: +91 674 301226
                                                                    Email: cysdbbsr@sancharnet.in
                                                                    Website: http://www.cysd.org

                                                                    Tel: +91 674 300774, 301725,
 12Mr. Binaya Kumar Raut     CYSD         E-1, Institutional Area   400693®
                                          Bhubaneshwar, Orissa,
                                          India, 751 013            Fax: +91 674 301226
                                                                    Email: cysdbbsr@sancharnet.in
                                                                    Website: http://www.cysd.org

                                                                    Tel: +91 674 300774, 301725,
 13Dr. Anup Kumar Dash       CYSD         E-1, Institutional Area   586409®
                                          Bhubaneshwar, Orissa,
                                          India, 751 013            Fax: +91 674 301226
                                                                    Email: cysdbbsr@sancharnet.in
                                                                    Website: http://www.cysd.org

Observers:

  1Mr. Ani Rudra Silwal      CMF          P.O. Box 2900             Tel: +977 1 434041, 432947
                                          Kathmandu, Nepal          Fax: +977 1 432947
                                                                    Email: silwal@sccs.swarthmore.edu
                                                                    Website: http://www.cmfnepal.org

  2Mr. Nand Kishor Agrawal   EDA          107, Qutab Plaza         Tel: +91 124 6350835, 6388676®
                                          DLF - City 1             Fax: +91 124 6352489
                                          Gurgaon, Haryana, India,
                                          122 002                  Email: edarural@nda.vsnl.net.in
                                                                   Website: N/A

  3Mr. Langsun T. Mate       SADHAN       B-4/3133, Vasant Kunj     Tel: +91 11 6132629, 5894564®
                                          New Delhi, India, 110
                                          070                       Fax: +91 11 6132629
                                                                    Email: sa_dhan@mantraonline.com
                                                                    Website: N/A

                                          2/1, Nayapalli Civic
  4Dr. Gyanendra Mani        NABARD       Centre, P.O.Box:179       Tel: +91 674 554034, 302449®
                                          ANKUR', Nayapalli,
                                          Bhubaneshwar, Orissa      Fax: +91 674 552019
                                          India, 751 007            Email: nab_bhu@coop.nic.in
                                                                           mani_gyan@sify.com
                                                                    Website: N/A

                                                                    Tel: +91 674 507176, 514465,
  5Mr. Srimanta Kumar Patra CARE - INDIA 372, Sahidnagar            431902®
                                         Bhubaneshwar               Fax: +91 674 507176
                                         Orissa, India, 751 007     Email: spatra@careindia.org
                                                                    Website: http://www.careindia.org
 6Mr. Srikant Das        SIDBI   OCHC Building, 4th Floor Tel: +91 674404854, 404298®
                                 Janpath, Bhubaneshwar Fax: +91 674 404476
                                 Orissa, India, 751 001   Email: srikantdas_bbsr@rediff.com
                                                          Website: http://www.sidbi.com

                                                           Tel: +91 674 301339, 301725,
 7Mr. Parashuram Nayak   CYSD    E-1, Institutional Area   436005®
                                 Bhubaneshwar, Orissa,
                                 India, 751 013            Fax: +91 674 301226
                                                           Email: cysdbbsr@sancharnet.in
                                                           Website: http://www.cysd.org

                                                           Tel: +91 674 301339, 300774,
 8Ms. Ambilaya Patnik    CYSD    E-1, Institutional Area   320334®
                                 Bhubaneshwar, Orissa,
                                 India, 751 013            Fax: +91 674 301226
                                                           Email: cysdbbsr@sancharnet.in
                                                           Website: http://www.cysd.org

                                                           Tel: +91 674 550490, 556069,
 9Ms. Chapala Patnik     CYSD    N-1, A/29, Nayapalli      533051®
                                 Bhubaneshwar, Orissa,
                                 India, 751 013            Fax: +91 674 551087
                                                           Email: cysdbbsr@sancharnet.in
                                                           Website: http://www.cysd.org

10Mr. Hari Kumar Das     CYSD    N-1, A/29, Nayapalli      Tel: +91 674 550490, 556069
                                 Bhubaneshwar, Orissa,
                                 India, 751 013            Fax: +91 674 551087
                                                           Email: cysdbbsr@sancharnet.in
                                                           Website: http://www.cysd.org

								
To top