Heading by pengtt

VIEWS: 4 PAGES: 55

									Monitoring and Evaluation:
FAMILY PLANNING PROGRAMS
Session Objectives
• Be able to apply basic M&E concepts
  (frameworks, indicators, etc.) to family-planning
  programs
• Be able to summarize the main issues in M&E
  of family-planning programs from a post-Cairo
  perspective.
• Be able to summarize the emerging issues for
  M&E of family-planning programs in high HIV
  prevalence countries.
Session Overview

 •   Family-planning frameworks
 •   M&E implications of the Cairo agenda
 •   Contraceptive prevalence and unmet need
 •   Monitoring quality of care
 •   Evaluating the impact of quality
 •   Family planning and HIV
Family Planning Frameworks
    Conceptual Framework for FP
    Demand and Program Impact on
    Fertility
                                                         Other
                                                         intermediate
                                                         variables

Societal &                                                                   Fertility
                    Value &              FP demand
individual          demand for                                               • Wanted
factors                                  • Spacing
                    children                                                 • Unwanted
                                         • Limiting        Contraceptive
                                                           practice

                      Service outputs:
Development
programs              • Access                 Service Utilization         Other health &
   FP supply          • Quality                                            social
                                                                           improvements
   factors            • Acceptability



              Source: Bertrand, Magnani, and Rutenberg, 1996.
  Conceptual framework of family
  planning supply factors
External              FP Organizational          Operations
Development           Structure
Assistance                                       • Management &
                      • Service                  supervision          Service Outputs
                      infrastructure
                                                 • Training           • Access
Political and         • Sectoral integration
Administrative                                   • Commodity          • Quality
System                • Delivery strategies      acquisition &
                                                                      • Acceptability
                                                 distribution
• Political           • Public-private
support               partnerships               • IEC
• Resource                                       • Research &
allocations                                      evaluation
• Legal code /
regulations
                 Larger
                 societal &              Source: Bertrand, Magnani, and
                 political               Rutenberg 1996
                 factors
Applying the frameworks for FP
M&E
• Inputs, e.g.
   – Types and levels of resources
   – Qualified personnel
   – Unit and total costs of program resources
• Outputs – functional areas, e.g.
   – People trained
   – Performance of people trained
   – Cost per person trained
Applying the frameworks for FP
M&E
• Outputs – Service outputs, e.g.,
  – Service delivery points providing FP services
  – Quality of FP services
  – Cost of increasing access/quality of FP services
• Outputs – Service utilization
  – New FP acceptors, Couple Years of Protection
    (CYP)
  – Returning clients
  – Cost of increasing CYP, etc.
Applying the frameworks for FP
M&E

  • Outcome – intermediate outcomes
    – Contraceptive prevalence rate (CPR)
    – Unmet need
    – Costs associated with increased CPR
  • Outcome – long term outcome
    – Fertility rates
    – Unintended pregnancy
    – Costs of changes in fertility, unintended
      pregnancy
Indicators for FP programs


  • See Bertrand and Escudero, 2002,
    Compendium of Indicators for Evaluating
    Reproductive Health Programs, 2 volumes
    – Indicators that crosscut program areas
    – Indicators for specific program areas
What is different about M&E of FP
programs?
• Basic principles are the same as in other
  health programs
• Outcomes relatively well-defined, focused,
  and measurable
• Long history of data collection on FP
  outcomes through WFS, DHS – document
  global trends
• Attempts to link outcomes to program
  outputs - evidence of program effects
Programme of Action adopted at
ICPD, Cairo 1994
Traditional (pre-Cairo) focus of FP
program M&E


  •   Demographic impact
  •   Focus on married women
  •   Availability of services
  •   Contraceptive adoption (new users)
  •   Characteristics of women
  •   Cross-sectional measurement
Cairo: Objectives of FP Programs
• To help couples and individuals meet their
  reproductive goals
• To prevent unwanted and high-risk pregnancies
• To make quality FP services affordable,
  acceptable, and accessible
• To improve the quality of family planning IEC,
  counseling, and services
• To increase the participation and sharing of
  responsibility of men in FP
• To promote breastfeeding to enhance birth
  spacing
Exercise 1

• Discuss the implications of the Cairo programme
  of action for M&E of FP programs. Identify 3 or
  more ways in which the traditional focus of FP
  programs listed on the earlier slide should change
  to respond to the Cairo agenda. What are the
  implications of these changes for M&E?
Contraceptive Prevalence Rate
(CPR)

• Percentage of (married) women of reproductive
  age (15-49) who are currently using a
  contraceptive method.
Unmet Need for Family Planning

• Percentage of fecund women exposed to the risk
  of pregnancy who say they want to wait at least
  two years for another birth (spacing) or do not want
  any more children (limiting), but are not currently
  using a method of contraception.
Related Indicators

• Demand for FP = % (married) women using FP +
  % (married) women with unmet need for FP
• Percentage of demand satisfied = % (married)
  women using FP / % (married) women with
  demand for FP
Unmet Need Exercise
CPR vs Unmet Need
CPR                       Unmet Need
• Relatively simple to    • Relatively complex to
  define                    define
• Uni-dimensional         • Multi-dimensional –
• Consistency over time     demand & use
• Does not capture        • Definition has evolved
  concept of meeting      • Captures concept of
  needs                     meeting need
Monitoring Quality of Care
What is Quality of Care in FP?
• General, loosely-defined concept
• Different people define quality in different
  ways
• Multi-dimensional
• Appropriate standards against which to
  measure quality vary
Bruce-Jain Framework
• Choice of contraceptive methods
• Information given to users
• Provider competence
• client/provider relations
• re-contact and follow-up mechanisms
• appropriate constellation of services
Indicators for QOC


  • No single indicator can capture the
    different components of QOC
  • Indicators need to be adapted to specific
    program context and priorities
  • Shortlist of 24 QOC indicators (see
    Bertrand and Sullivan, Evaluation Bulletin
    No. 1, Table 1 page 2).
Facility Surveys for QOC
Indicators

  • Situation Analysis
  • MEASURE Evaluation Quick Investigation
    of Quality (QIQ)
  • MEASURE DHS+ Service Provision
    Assessments (SPA)
  • DHS service availability modules and
    community surveys (SAM)
Some Data Collection Issues

• Small sample sizes for FP clients,
  especially in low prevalence countries
• Observation in clinics that use a client
  flow approach
• Sampling
• Courtesy bias and hawthorn effects
• Unit of analysis (client, provider, facility)
Case Study: QOC in Turkey
Turkey’s Strategic Framework


                           Strategic Objective
                  Increased utilization of FP/RH services


     Intermediate Result 1                       Intermediate Result 2
  Strengthened sustainability          Expansion of high quality FP/RH services
       of FP/RH program                    in the public and private sectors
The Quality Index


   •   Method availability
   •   Availability of trained personnel
   •   Perceived quality of FP counseling
   •   Adequate infection-prevention measures
   •   Availability of IEC materials
   •   Physical access to FP services
Data Source

• Istanbul Quality Surveys
  – Facility inventory
  – Client exit interviews
• Based on MEASURE Evaluation QIQ
The Quality Index


  • Sum of scores from    Private
                         hospitals
    the 6 components
                           SSK
    (range 0-6)          hospitals


                          Health
                          centers


                          MCH/FP
                          centers


                           MOH
                         hospitals


                                     0   2   4   6
Method Availability


  • Proportion of facilities    Private
                               hospitals

    that distribute or           SSK
    prescribe 3 or more        hospitals


    modern FP methods           Health
                                centers


                                MCH/FP
                                centers


                                 MOH
                               hospitals


                                           0   0.2   0.4   0.6   0.8   1
Perceived Quality of FP
Counseling

• Proportion of clients who      Private
  report                        hospitals


   – they were seated             SSK
                                hospitals
   – had sufficient time with
     the provider                Health
                                 centers
   – clearly understood the
     information provided        MCH/FP
                                 centers


                                  MOH
                                hospitals


                                            0   0.2   0.4   0.6   0.8   1
Adequate Infection Prevention
Measures

  • Proportion of facilities
                                 Private
    that meet the following     hospitals

    standards :                   SSK
     – Plastic bucket for CL    hospitals

       solution                  Health

     – Unused IUD kits kept      centers

       sterile                   MCH/FP
                                 centers
     – Medical waste kept in
       leak-proof containers      MOH
                                hospitals
       with lids
     – Appropriate containers               0   0.2   0.4   0.6   0.8   1

       for sharp objects
Evaluating the impact of quality
of care
    Framework for links between quality of
    family planning services and fertility
Quality of
services              Acceptance
•Choice
Information to                      Contraceptive
users                                 prevalence
Provider
competence
Client-provider      Continuation                               Fertility
relations
Follow-up
                                        Other
Appropriate
                                      proximate
constellation of
                                     determinants
services


  Other factors                                      Known effects

                                                     Hypothesized effects

 Source: Jain, 1989
Outcomes of interest
• Intention to use
• Contraceptive adoption
• Contraceptive discontinuation
  – Failure
  – Switching
  – Stopping
• Current contraceptive use
  – Contraceptive choice
• Unwanted pregnancy
Examples of impact studies

• Peru (Mensch, et al., 1996)
• Morocco (Steele, et al., 1999)
• Bangladesh (Koenig et al., 1997)
Morocco Study Design (1)
• To explore whether the service environment in
  which a woman resides affects adoption and
  continuation of the pill
• Linkage of 1995 Demographic and Health Survey
  calendar with 1992 DHS Service Availability
  Module
• Multi-level hazards models with contraceptive
  adoption and discontinuation as outcomes
• 862 births and 775 episodes of pill use in 107
  clusters
Morocco Study Design (2)
• Explanatory factors - Individual and
  Community
  – age, education, residence, community drinking
    water & toilet facilities, principle economic
    activity
  – Contraceptive intention (discontinuation)
  – Breastfeeding status, last child wanted
    (adoption)
• Explanatory factors – Program
  – Public health center <10km, pharmacy <5km,
    outreach services, 3+ methods available at
    clinic
  – Source of pills (discontinuation)
Predicted percentage of women
adopting a modern contraceptive
method within 12 months of giving
birth by service factors
    70
    60

    50

    40
%
    30

    20

    10
    0
          Yes         No           <3           3+
         Health center <10km   No. methods offered at
                               closest set of facilities
Predicted 12-month pill discontinuation
rate by reason and service factors,
Morocco
35%

30%

25%

20%

15%

10%

5%

0%
      Government   Pharmacy or         Yes           No              Yes         No
                      other         Health center within 10KM      Pharmacy within 5KM
               Source

 Failure    Desired pregnancy    Side effects/ health concerns   Other method-related
Main Findings: Morocco
• Relatively strong service effects on post-
  partum adoption
• Service availability associated with both
  adoption and discontinuation
• Number of methods available only
  associated with adoption
• Users of government sources have lower
  discontinuation
Limitations of Impact Studies
• Measures of quality inadequate (often
  limited to access and method choice)
• Cross-sectional designs (endogenous
  inputs)
• Linking individual and program data
  (geographic boundaries, service
  environment vs. individual service
  experience)
Emerging areas: FP/HIV linkages
and integration
Context

• Considerable progress in preventing unwanted
  pregnancy but unmet need remains substantial
• Rapid increases in HIV in many countries
• Changing funding focus to HIV from FP
• Integrated vs. vertical programs
Synergies between FP and HIV
programs

  • Both are central to reproductive health
  • “ABC” messages in HIV programs also
    relevant to FP programs
  • Youth programs that encourage responsible
    sexual behavior prevent both HIV and teen
    pregnancy
  • Strong RH policies support both HIV and FP
    programs
Dual Protection

• Abstinence
• Monogamous couples using effective
  contraception
• Correct and consistent condom use
FP in high HIV-prevalence
countries

  • Relationship between HIV and fertility desires
  • FP/RH needs differ for:
     –   HIV- concordant monogamous couple
     –   HIV- concordant non-monogamous couples
     –   HIV discordant couples
     –   HIV+ concordant couples
  • HIV counseling in FP services
FP and VCT
• FP counseling opportunity for VCT or general HIV
  counseling and VCT referral
• VCT services could include FP services or FP
  counseling and referral
• Concern over unintended consequences of
  integration
   – Provider burn-out
   – Discourage FP clients
   – Quality of integrated vs. vertical FP & VCT services
FP and PMTCT
• Averts child infections by preventing
  unintended pregnancies among HIV+
  women
• PMTCT programs provide opportunity for
  prenatal FP counseling and post-partum
  contraceptive use
• Reduced breastfeeding by HIV+ mothers will
  lead to shorter birth intervals in the absence
  of FP
FP Counseling of PMTCT clients,
Zambia
  60

  50

  40

% 30

  20

  10

   0
       ANC visit        3 months PP       6 months PP

              pregnant women   HIV+   HIV-
                                 Source: Rutenberg & Baek, 2004
PMTCT-Client FP Use 6 Months Post-
partum, Zambia
    45
    40
    35
    30
    25
  %
    20
    15
    10
     5
     0
         Modern     Condom         Sex active, no
                                      method

                  HIV+   HIV-
                          Source: Rutenberg & Baek, 2004
HIV Counseling in FP Sessions,
Uganda
                    40
                    35
                    30
    % FP sessions




                    25
                    20
                    15
                    10
                     5
                     0
                         HIV needs      HIV risk discussed Dual protection
                          assessed                           proposed

                                     Baseline   Follow-up
                                                    Source: Rutenberg & Baek, 2004
Exercise 3
• Select an area of FP/HIV integration (e.g.
  PMTCT, VCT, HIV counseling in FP etc.).
  – Develop a basic input-output-outcome-impact
    framework for a simple program in this area.
  – Suggest 3-6 indicators to monitor your program.
  – What data sources would you propose to collect
    these indicators?

								
To top