SII Women's Empowerment Guidance 12 Steps to a Research

Document Sample
SII Women's Empowerment Guidance 12 Steps to a Research Powered By Docstoc
					                             12-Steps to a Research Design
                    Identify the resources available for the Study: time, money, expertise.

    We have $______ for this study. We have ____ staff days. We have ____ partner days. We have
                                      _____ participant’s days
                      Key Decision                                    CASE EXAMPLE1
STEP ONE: Determine the key research questions or hypotheses.                  “Women’s participation in village savings and
What do you need to know? What relationships are you interested                loans will have a positive impact on women’s
in investigating?                                                              empowerment”
STEP TWO: Determine very clearly what your dependent (the
“effect” or impact you are interested in) and independent (the                 Dependent variables: women’s empowerment
proposed cause of the effect) variables (are). It always easier and            Independent variable: Participation in a village
less costly to investigate a one-to-one relationship. However, it is           savings and loan program.
often the case that we want to know either how multiple causes lead
to a single effect, or to multiple effects. In this study, for example,
we are looking at two dependent variables and one independent
variable:
HISSA/MJT groups  economic security
HISSA/MJT groups  women’s empowerment
If you wish to investigate additional causal relationships, be
forewarned that each additional independent or dependent
variable will increase the load of “burden of rigorous proof”
exponentially. How does this translate in terms of your plans: if
you have the resources to actually assess the causal relationships
between HISSA/MJT groups on two different dependent variables
(effects, or impacts) among, say, 75,000 people…the addition of
one dependent or independent variable will mean that you probably
can only demonstrate such a relationship in around 20,000 people.
The addition of a second variable would take it down to around
5,000 people (for the same resources you have available to you).
STEP THREE: Identify crucial intervening or confounding                        Urban vs. rural setting
variables. These are variables that may intrude themselves between             Length of time a VSL has been in existence
your purported dependent and independent variables. Unless you                 Women’s marital status
attempt to control for them – qualitatively or quantitatively – you            Ethnicity
risk identifying false relationships.
STEP FOUR: Define and identify specific and measurable (can be                 Empowerment is defined broadly as “the
qualitative or quantitative) indicators for the dependent variable(s).         expansion of assets and capabilities of poor
                                                                               people to participate in, negotiate with,
                                                                               influence, control, and hold accountable the
                                                                               institutions that affect their lives”2 and “the
                                                                               expansion in [women’s] ability to make
                                                                               strategic life choices in a context where this
                                                                               ability was previously denied to them.”3 (See
                                                                               FY06 research framework for more specific
                                                                               evidence categories and core indicators for
                                                                               empowerment).
STEP FIVE: Determine data sources                                                        Participating Women
                                                                                         Husbands or male relatives of
                                                                                         participating women
                                                                                         Village headmen
                                                                                         CARE staff
                                                                                         Secondary studies


1
  This column shows a hypothetical example. It is intended to give a very concrete picture of the products from each step. However,
it is important to keep in mind that the example is one in which mixed methods play an important research role, and in which
quantitative data from a survey are needed to answer the research question. Purely qualitative research designs may involve a number
of questions and design considerations that do not get covered in this example.
2
  Deepa Narayan ed, Empowerment and Poverty Reduction: A Sourcebook. Washington, DC: World Bank, 2001.
3
  Naila Kabeer, “The Conditions and Consequences of Choice: Reflections on the Measurement of Women’s Empowerment.”
UNRISD Discussion Paper 108. United Nations Research Institute for Social Development, Geneva, 1999.
                                                                                         Government statistical records
                                                                                         (judiciary, participation in elections,
                                                                                         pro-women policies)
                                                                                         Local gender experts
                                                                                         Nonparticipating men/women
                                                                                         (comparison group)
STEP SIX: Determine the methods that you need in order to gather                    1.    Quantitative survey that allows us to
the information and data required and meet levels of rigor that will                      make statistically valid comparisons
satisfy the intended audience(s) of the research. PRINCIPLE:                              between participants and non-
adopt methods that are as complex as needed but simple as possible.                       participants on the dependent variable
                                                                                          (women’s empowerment).
                                                                                    2.    Secondary data review (results of
                                                                                          similar studies in similar contexts)
                                                                                    3.    Key informant interviews (on
                                                                                          sociocultural and gender context)
                                                                                    4.    Semi-structured interviews with
                                                                                          women and men (qualitative and
                                                                                          participatory numbers4)
                                                                                    5.    Focus group discussions (qualitative
                                                                                          and participatory numbers)
    Think about your resources: Do you sense that they allow you to take on all four of these methods?
STEP SEVEN: Determine the overall research design strategy.                    Cross Sectional
Longitudinal (data will be collected at least twice over some period
of time)? Cross-sectional (a single point in time)? To a very large
extent, this decision is determined by your actual research questions
from step one. It also can be influenced by the resources you have
available. It can also be influenced by a longer term evaluation
strategy that you have in mind.
STEP EIGHT: Determine the appropriate sampling population.                     9057 women and their households in 360
Who or what is the largest population that you wish to be able to              CARE village and savings and loans in one
describe and/or account for in relation to your hypothesis?                    project in North Region.
Frequently, this decision is made not by the actual limits or
extent of a particular program but by the resources you have
available. The choice of the largest population that you wish to be
able to speak authoritatively about vis-à-vis impact is frequently
strategic and purposive. The key is to be very clear with ourselves
and our stakeholders about who or what we’re leaving out, and
why…and what population our research actually represents in terms
of its findings.
STEP NINE: Identify the critical sub-levels of analysis, from                  Largest: CARE project
largest to smallest.                                                           Next: 360 VSLs
                                                                               Next: 9057 Households
                                                                               Smallest Analytical Unit: Individual women
      Think about your resources: Do you sense that they allow you to take on all of this? Example:
    imagine, right now, that you know you have a 2-hour long household questionnaire that you hoped
    could be used in a random sample of the households. For it to be statistically valid, you would have
      to randomly sample around 500 of those households. Is that even possible, given your resource
                                          constraints? If not…
STEP TEN: Select a sampling strategy for every level                 1. LARGEST: One CARE Project
identified in step nine. There are basically two broad               2.        Next Largest: 360 VSLs. Our resources do not
kinds of sample: probability samples and non-probability                allow us to do rigorous impact research in all of them.
samples. Probability samples, also known as random                      We’ll do a stratified random sample based on the
samples, allow every analytical unit to have an equal                   intervening variables of a) length of time in program
chance of being selected. They allow you to generalize to               and b) urban/rural setting. We’ll classify all VSLs in
a larger population. They also are best for avoiding                    the total population then randomly choose:
researcher bias. Nonprobability samples, also known as                  -2 VSLs, urban, less than one year in program
purposive samples, cannot, on their own, allow you to                   -2 VSLs, urban, more than three years in program
generalize to a wider group. They are more subject to                   -2 VSLs, rural, less than one year in program
researcher bias although this can be minimized through                  -2 VSLs, rural, more than three years in program

4
    See “Participation and Numbers,” pla notes 47 (August 2003): 6-12.
establishing strict, objective criteria for choosing data           Organize one female and one male (i.e., spouse or
sources.                                                            HH head) focus group discussion in each of the
                                                                    eight.
You can mix probability and nonprobability sampling           3.           Next largest: household. Among 8 VSLs – 240
strategies across methods, it needs to be emphasized.               women – randomly select the largest possible number
What remains very important, with nonprobability                    that we can handle, given our budget and human
sampling (almost always used, for example, for                      resources, for the quantitative survey. The random
identifying informants for in-depth interviews, focus               selection of the women results in random selection,
group discussions, key informant interviews, etc.) is that          too, of their spouses or other male household head
you are clear and transparent about your selection criteria         (if applicable). So…the resource question needs to be
and that you follow those criteria to the letter. Being clear       thought of in light of the pair of interviews. For the
about your sampling strategy for qualitative methods is             sake of this case example, imagine we’ve decided we
one very important form of impact research rigor                    can handle 80 total quantitative surveys. We select,
frequently missing from impact assessments.                         therefore, 5 women (and therefore their spouses or
                                                                    other HH head, if applicable), at random, from each of
                                                                    the eight VSLs.
                                                                 4.        Next largest: individual. Among the 8 VSLs –
                                                                    240 women – Randomly select two per group for in-
                                                                    depth, individual interviews. Avoid selecting the
                                                                    same women who participated in the quantitative
                                                                    survey. This random selection of women results in
                                                                    your random sample of men for in-depth interviews.


STEP ELEVEN: Select a comparison group. Identify at           In one of the urban and one of the rural villages, identify
least one comparison group – sometimes called a control       all HHs that are similar to participating HHs with regard
group – that can represent the “counterfactual”, in other     to the intervening variables of marital status and ethnicity
words, a group that has similar characteristics, contexts,    (purposive sample). Then, within that, randomly select
and cultures as the main analytical units from STEP TEN.      10 nonparticipating households for the quantitative
                                                              questionnaire. Organize one FGD of nonparticipating
NOTE: formal comparison groups are frequently omitted         women in each of the 8 VSL sites. Organize one FGD of
from development impact research and the                      nonparticipating spouses (why these numbers? Resource
“counterfactual” is treated in a more qualitative manner,     constraints. It’s the most we can do in the time that we
triangulating between data sources, using secondary data,     have).
etc. In the SII, we wish to include a comparison group if
financially possible.
NOW: Calculate the actual costs of the research you’ve just described. Do you have the
  resources? Do you need to change any decisions? Frequently, at this stage, you’ll find
 you need to circle back as far as your research questions and make them more focused.
 Or, you might find that while you hoped to do impact research in a given project across
 35 villages, you need to focus on a much smaller subset of those villages. Or, you might
find that you need to reduce the scope of one or another particular method: for example,
   you might still go ahead with a stratified random sample HH questionnaire in all 35
              villages, but only choose two for in-depth, qualitative methods.
                               TOTAL EFFORT FOR ABOVE STUDY
    1.  A 1-hour long quantitative survey with 40 female participants = 40 hours (5 person days)
    2.  A 1 hour long quantitative survey with 40 husbands or elder males in women’s households = 40
        hours (5 person days)
    3. A 1 hour long quantitative survey with 20 comparison group women = 20 hours (2.5 person days)
    4. A 1 hour long quantitative survey with 20 comparison group husbands/sr. males = 20 hours (2.5
        person days)
    5. In-depth interviews with 8 village headmen = three person days
    6. Key informant interviews re., context = 3 person days
    7. 8 focus group discussions with female participants = 3 person days
    8. 8 focus group discussion with spouses of female participants = 3 person days
    9. 2 focus group discussions with non-participating females = 1 person day
    10. 2 focus group discussions with non-participating males = 1 person day
    11. 16 in-depth interviews with female participations = 8 person days
    12. 16 in-depth interviews with spouses of female participants = 8 person days

TOTAL PERSON DAYS NEEDED FOR DATA COLLECTION: 45
BUFFER DAYS (to account for travel, logistical snags, etc.): 9
TOTAL PLANNED PERSON DAYS FOR THIS RESEARCH: 54
Size of research team: 5
# of days of field research needed FOR DATA COLLECTION: 11
Cost/day for data collection: $1200
TOTAL COST FOR DATA COLLECTION: $13,200

STOP: DO YOU HAVE ADEQUATE RESOURCES? IF NOT: YOU’LL NEED TO RETURN TO
STEP ONE AND SCALE BACK YOUR AMBITIONS.
STEP TWELVE: Apply the appropriate methods at 1. Secondary research on gender, power, political-
the appropriate levels from step ten          economic, and social context.
                                              2. 8 VSLs and their villages: sociocultural context,
                                              secondary data, interviews with 8 customary leaders
                                              and eight key informants
                                              3. In 8 VSLs, random sample of 40 women and
                                              their spouses/male HH heads: quantitative
                                              socioeconomic survey; related survey for husbands
                                              (if appropriate) = 80 HH surveys
                                                   4. In 8 VSLs, one focus group of VSL
                                                       participants and one of their spouses = 16.
                                                   5. In 8 VSLs, individual interviews with two
                                                       VSL participants = 16 in-depth interviews
                                                   6. In two of the VSL villages: 10 female and
                                                       10 male survey questionnaires as a
                                                       comparison group = 40 surveys
                                                   7. In 8 VSL villages: one FGDs with
                                                       nonparticipating females = 8 FGDs
                                                   8. In 8 VSL villages: one FGD with
                                                       nonparticipating spouses = 8 FGDs