Docstoc

Impact Evaluation of Brazil's School Based Financial Education .._1_

Document Sample
Impact Evaluation of Brazil's School Based Financial Education .._1_ Powered By Docstoc
					Impact Evaluation of Brazil’s
  School Based Financial
   Education Program
                1

        PRELIMINARY RESULTS
             JUNE 6, 2011

         ARIANNA LEGOVINI
          MIRIAM BRUHN
             BILAL ZIA

            WORLD BANK
 Impact Evaluation Objectives (Students)
                                   2


 Measure the impact of a three semester long program
 that teaches high school students financial concepts
    Does the program improve students’ financial knowledge?

 The program also includes exercises and case studies
 related to the students’ daily life (e.g. making a budget)
    Does the program change attitudes about financial decisions, as
     well as actual financial behavior and decision making?
  Impact Evaluation Objectives (Parents)
                                3


 Some exercises are meant to be done at home,
 jointly with the parents
    Is this enough to also improve parents’ financial knowledge
     and to change their attitudes and behavior?




 Can we achieve a greater impact by also providing a
 financial literacy workshop directly to parents?
                           Methodology
                                      4

 Pilot test the financial education program in close to 900
  schools in various states in Brazil

 Schools that were interested in participating in the pilot
  were randomly divided into two groups
     Treatment group receives financial education text books for free
      AND receives teacher training for how to implement the material
     Control group will receive the same two school years later (possibly
      with modified material and training, based on evaluation results)

 Half of the parents in treatment schools will also
  randomly be selected for participation in a parent
  financial education workshop in schools
                       Pilot Sample Size
                               5


 Total of 891 schools in six states (439 treatment and 452
 control schools)
        State                      Number of schools
        São Paulo                         372
        Rio de Janeiro                    270
        Ceará                             122
        Distrito Federal                  64
        Tocantins                         34
        Minas Gerais                      29

 Approximately 26,000 students and parents (one class
 per school)
                         Timeline
                                 6


Time                 Activity

April – May 2010     Complied list of interested schools with help of State
                     Ministries of Education;
                     Randomly divided schools into treatment and
                     control group
May – July 2010      Teacher training
Early August 2010    Baseline survey
Mid August 2010      Teachers started teaching financial education
                     material, continuing until November 2011
Late November 2010   First follow-up survey
Spring 2011          Parent workshops
November 2011        Second follow-up survey
             Results Presented Today
                              7


 Use data from August 2010 baseline survey and
 November 2010 follow-up survey

 Students had received about 1 semester of financial
 education before the November 2010 follow-up survey

 Full program takes 3 semesters – the pilot ends in
 November 2011

 Results presented today are preliminary since they
 only cover the first out of three semesters of financial
 education
          Baseline Characteristics:
Balance across Treatment and Control Group
                    8
         Follow-up Data: Impact Analysis
                                  9


 At baseline, average student and parent
 characteristics, as well as test scores, were the same
 in the control and treatment group
    This is because the treatment was randomly assigned
    Therefore the control group is a valid comparison group for the
     treatment group


 Thus, any differences in averages between treatment
 and control group at follow-up are CAUSED BY the
 financial education program
       Students’ Financial Proficiency
                          10




 To measure students’ financial knowledge, the
 survey firm (CAEd) developed a test tailored to the
 program’s material

 Based on this test, CAEd calculated the level of
 financial proficiency for each student, ranging
 from 0 to 100
                                   Impact on Financial Proficiency
                                                      11

                                65.0
   Average Level of Financial




                                                                  60.4
                                60.0
                                                           56.1
         Proficiency




                                55.0
                                                                         CONTROL
                                       49.8    50.2
                                50.0                                     TREATMENT


                                45.0


                                40.0
                                         Baseline           Follow-up

 In the follow-up survey, the average level of financial
  proficiency was significantly higher in the treatment
  group (60.4) than in the control group (56.1)
                 Statistical Analysis of
             Impact on Financial Proficiency
                              12




 The statistical test of these differences shows that the
  financial education program increased the level of
  proficiency on the financial knowledge test by 3.6 points
  (compared to 56 points in the control group)
 How large is the effect on financial proficiency?
                                        13


 The 3.6 point increase corresponds to a 6.4% increase
  compared to the average in the control group (in one
  semester)
     According to CAEd, other school programs they have worked with in
      Brazil tend to improve test scores by 2.5% per school year
     In Mexico, providing cash transfers to parents conditional on children’s
      school attendance improved test scores by 8-9% for 6-14 year olds
      (measured 6 years after the start of the transfers)

 The effect size (0.25 of a standard deviation) is similar to the
  effect of
     A program in Colombia that provided low-income children with
      vouchers for half the cost of private secondary school (0.2 std. dev. after
      6 years)
     A program in India that provided remedial education to students lagging
      behind in school (0.28 std. dev. after two years, the effect of the same
      program after one year was 0.14 std. dev)
                          Effect Size (est.)
                                         14

           Percentage change in learning achievement
9
8
7
6
5
4
3
2
1
0
    Brazil Financial    Other       CCT in Mexico     Colombia     India remedial
     Literacy (after programs in    (after 6 years) vouchers (after education
       4 months)      Brazil (per                      6 years)    (after 2 years)
                        year)
 Students’ Self-Reported Knowledge Ratings
                                 15


 The surveys also asked students to rate their
 knowledge about different financial concepts
    On a scale from 0 to 10, how would you rate your knowledge
     about
      Interest rates
      Loans/financing
      Insurance
      Income tax
      Minimum payment for credit cards
       Impact on Students’ Self-Reported Knowledge Ratings
                        (Follow-Up Data)
                                                        16

 7.5                                                                                      7.2
 7.0                                  6.8
                                                                                    6.4
 6.5
                  5.9           6.0                    6.0
 6.0                                                                    5.7
 5.5
            5.1
                                                 5.0              5.0                               Control
 5.0
                                                                                                    Treatment
 4.5

 4.0

 3.5

 3.0
       Q76: Interest Rates        Q78:             Q80:        Q82: Income Tax   Q84: Min. Credit
                             Loans/Financing Welfare/Insurance                    Card Payments

 At follow-up, average self-reported knowledge ratings for
  all these concepts were higher in the treatment group
  than in the control group
Two Questions Often Used in the Economics Literature
           to Measure Financial Literacy
                                    18


1. Suppose you are interested in taking a loan of R$50,000. Bank “A” offers
you a loan of R$50,000 payable in one year with an annual interest rate of
15%. Bank “B” offers you a loan of the same amount, R$50,000, but you will
have to pay $60,000 within one year. Which loan would you prefer?
a) Bank A
b) Bank B
c) Don’t know

2. Suppose the interest rate on your savings account is 1% per year and
inflation is 2% per year. After a year, if you had not made any deposits or
withdrawals in this account, would you be able to buy more than, exactly the
same as, or less than today with the money in this account?
a) More than today
b) The same as today
c) Less than today
d) Don‘t know
 Impact on Students’ Correct Answers to Financial Literacy
 Questions Used in Economics Literature (Follow-Up Data)
                                  19

       45%
                      42%
                40%
       40%
                                             36%
       35%                             33%
                                                     Control
       30%
                                                     Treatment
       25%

       20%
             Q127: Prefers R15K    Q128: Inflation
               plus 15% int.          question
 At follow-up, a higher percentage of students in the treatment
  group than in the control group gave the correct answer to the
  financial literacy questions often used in the economics
  literature
            Student Financial Autonomy
                                21

 Autonomy index, designed by CAEd, based on 3
 dimensions of Autonomy
    Reflexive Autonomy e.g. “ I like to think carefully before
     deciding to buy something”
    Emotional Autonomy e.g. “I am prepared to talk about money
     with my parents”
    Functional Autonomy e.g. “I always try to save some money to
     do things I really like”


 These three components were combined into one
 overall index of financial autonomy, where higher
 values indicate more financial autonomy
           Impact on Student Financial Autonomy
                     (Follow-up Data)
                             22


                 Financial Autonomy Index
   51.50
                                          50.91
  51.00
  50.50
  50.00
  49.50
  49.00            48.86
  48.50
  48.00
  47.50
                   Control              Treatment

 At follow-up, students in the treatment group obtained
  higher scores on the autonomy index than students in the
  control group
               Students’ Intention to Save
                                     24


 Intention to save index, designed by CAEd, based on 3
 components
    Attitudes towards behavior e.g. “In my opinion, saving money every
     month extremely beneficial
    Subjective norms and expectations e.g. “My family has the habit of
     saving money every month”
    Perception of capacity of controlling one’s behavior e.g. “I believe I
     can save a little money every month”

 These three components were combined into one overall
 index of intention to save, where higher values indicate
 higher intention to save
         Impact on Students’ Intention to Save
                  (Follow-Up Data)
                          25

                Intention to Save Index
104                                       103
102
100
 98            97
 96
 94
 92
             Control                  Treatment

 At follow-up, students in the treatment group had a
  higher measured intention to save than students in
  the control group
            Students’ Saving Behavior
                               27




 In order to assess whether intention to save is also
 reflected in savings behavior, the survey asked
 students
    Which fraction of their income they save
      About 61% of students have non-zero income (from work or
       from parents)
      About 37% of students work

    Whether they save to be able to finance future projects
              Impact on Students’ Saving Behavior
                       (Follow-Up Data)
                                       28

60%
                        50%
50%            44%
40%

30%                                                                Control
                                                        19%        Treatment
20%                                         17%

10%

 0%
        Q53: Pct of income saved is Q65: I save money for future
                 non-zero                     projects
 At follow-up, a higher percentage of students in the treatment
  group than in the control group
     Saved at least some of their income
     Saved to able to finance projects in the future
             Students’ Spending Behavior
                                    30



 The survey asked a number of questions to assess
 students’ spending behavior
    Do students make a list of expenses every month?
    How do students pay for purchases?
      Do they use a credit card?
      Do they use a booklet/installment plan in the store?
             Impact on Students’ Spending Behavior
                       (Follow-Up Data)
                                           31

20%
                  16%
15%        13%
                                 11%                  10% 9%
                                        9%
10%
5%                                                                   Control
0%                                                                   Treatment
       Q64: I make a list of Q68: I often shop Q71: I often shop
        all expenses every with a credit card        with a
              month                            booklet/installment
                                                plan in the store
  At follow-up, a higher percentage of students in the treatment group
   than in the control group made a list of expenses every month
  A lower percentage of students in the treatment group than in the
   control group
      Often shopped with a credit card
      Often shopped with a booklet/installment plan in the store
      Other Measures of Financial Behavior
                                   33


 Apart from saving and spending behavior, the survey
 asked about several other features of students financial
 behavior
    Did the student learn to manage money in school?
    Does the student negotiate how to pay when making a purchase?
    Does the student search for other brands and model when deciding
     which product to purchase (i.e. do the compare options?)


 These are often thought to be savvy/smart financial
 behaviors
                   Impact on Students’ Financial Savvy
                           (Follow-Up Data)
                                        34

80%
70%                             63% 67%             66% 68%
60%
50%
40%
30%                                                                 Control
20%                 15%
                                                                    Treatment
10%           2%
 0%
          Q86: Learned to  Q95: In general, I  Q96: In general, I
          manage money in negotiate how to pay  search similar
              school                            models/brands
  At follow-up, a higher percentage of students in the treatment
      group than in the control group
         Say they learned to manage money in school
         Negotiate how to pay
         Search similar model/brands when deciding what to buy
                        Parent Outcomes
                                     36



 Parents have not received financial education directly


 But the students’ financial education material includes
  exercises to be completed with parents (e.g. make a
  household budget)

 This can have positive “spillovers” on parents’ financial
  knowledge
     To measure parent’s financial knowledge, the parent survey included
      the same two financial literacy questions often used in the economics
      literature that were also included in the student survey
                       Impact on Parent Outcomes
                           (Follow-Up Data)
                                              37
80%
70%        67%70%
60%
                            48%52%
50%                                           44%45%
40%                                                         33%35%
30%                                                                  Control
20%                                                                  Treatment
10%
 0%
        Q33: Student Q34: Student Q36: Prefers Q37: Inflation
         talks to you helps organize R15K plus 15% question
       about finances HH budget           int.
 At follow-up, a higher percentage of parents in the treatment group than in
  the control group
     Say they talk to the student about finances
     Say the student helps organize the household budget
 But, there are only small differences in the percentage of parents who
  answer the financial literacy questions correctly
    Statistical Analysis of Impact on Parent Outcomes




 The program significantly increased the % of parents who
   Say they talk to the student about finances (by 3.6% compared to 67% in the
    control group)
   Say the student helps organize the household budget (by 4.9% compared to
    48% in the control group)
 So far, we see no statistically significant impact of the program
  on parents’ financial knowledge
                                      38
                             Conclusions
                                      39

 The first semester of the high school financial education
  program increased students’ financial knowledge
 The program also led to improvements in students’
  financial attitudes
 And, the program changed financial behavior – due to the
  program, students are more likely to
     Save and manage their expenses
     Engage in “smart” financial behaviors
     Students are also more likely to talk to their parents about financial
      matters and to help with organizing the household budget
 “Spillover” effects on parents’ financial knowledge are
  small so far  need for the parent workshop
                             Next Steps
                                     40

 Implement the parent workshop
   Currently working on this

   Have received updated student lists from close to 150 schools
    and have randomly allocated parents in these schools into two
    groups
       One will watch financial literacy video
       Other will watch health video

     Still need to receive student lists from the remaining treatment
      schools


 Start planning the November 2011 survey
         World Bank Team & Contact Info
                                  41

 TTL
    Rogelio Marchetti (LCSPF) rmarchetti@worldbank.org

 Researchers
    Miriam Bruhn (DECFP) mbruhn@worldbank.org
    Bilal Zia (DECFP) bzia@worldbank.org


 Field coordinator
    Luciana Leão (LCSPF) lululeao@terra.com.br


 Advisor
    Arianna Legovini (DECOS) alegovini@worldbank.org

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:1/31/2013
language:
pages:34