Group 3: Evaluating Research
January 24- 2001
Holds the researchers accountable to the donor.
Encourages researchers and funders are accountable
“downward” to communities;
Improves the quality and impact potential of future
Provides justification for further funding
Identifies “unexpected added value” (that researchers
Provides donors with justification to their higher-ups on
the value of the research
Why NOT to evaluate?
Opportunity costs; alternative use of funds
Some consider research a “waste of money”;
what then of funds used to evaluate research?
Types of RH Research
Basic research for improving knowledge:
--Large scale: testing systems
--Smaller scale: testing elements within a
Alternative methodologies for
1. Case study: good for learning about good practices
and generating lessons learned
2. Systematic review of a portfolio of projects on a set
3. Continuous reporting of results (quarterly reports);
4. Assessment by external evaluation team
5. Biblio-metric assessment: number publications, type
of publication, and citations.
6. Audit: to evaluate impact over time
Recommended principles for
evaluating research (…a start)
1. Clarify the expectations for evaluation at the onset of
2. Make evaluation proportional to the cost of the
3. Make evaluation constructive, not punitive.
4. Recognize that there is an element of irrationality;
need to remain flexible
5. Evaluate different types of research on different criteria
Is this a concern of the “rich countries” only? Do “poor”
countries have the luxury to spend additional funds on
There seems to be a dichotomy of “useful and
subjective” versus “more mechanical but more
The impact of the research may not occur until years
later; if too much time elapses, it becomes more
difficult to attribute change to the research.
-- Alternatively, one can ask: has the research been
USEFUL-- used in further research, cited, etc
Possible problems with inter-project peer review:
(1) groups collude to evaluate each other
(2) “peers” know that they will be competing in
the future, so they are hesitant to reveal
information about the “inner workings” or
weaknesses of the program.
What are the differences if one has a program
versus a project?
What are the pathways that are likely to lead to
the desired outcome for different types of
research in different environments?
What do we mean by “monitoring, evaluation,