Get documents about "Household Budget Surveys
Document Sample
Surveys: Collecting Policy
Relevant Data
Rachel Govoni-Smith
Kinnon Scott, DECRG
January 17, 2006
1
Sources-
• The Impact of Economic Policies on Poverty and
Income Distribution: Evaluation Techniques and
Tools, eds. Francois Bourguignon and Luiz Al
Pereira da Silva, World Bank, Washington, D.C.,
2003.
– Scott, Kinnon (2003) “Generating Relevant Household
Level Data: Multi-topic Household Surveys”
• Muñoz, Juan and Kinnon Scott (2005)
“Household Surveys and the Millennium
Development Goals”, report for Paris21 Task
Force on Improved Statistical Support for
Monitoring Development Goals
2
Household Surveys and the
Impact of Economic Policies on
Poverty and Income Distribution
Micro Level
• Estimating Incidence of Indirect Taxes
• Analyzing the Incidence of Public Spending
• Behavioral Incidence Analysis of Public
Spending
• Estimating Geographically Disaggregated
Welfare Levels and Changes
• Assessing the Poverty Impact of an Assigned
Program
• Ex Ante Evaluation of Policy Reforms 3
Household Surveys and the
Impact of Economic Policies on
Poverty and Income Distribution
Macro Level
• The Effect of Aggregate Growth on Poverty
• Linking Macro-consistency Models to
Household Surveys
• Partial Equilibrium; Multi-market Analysis
• The 123PRSP Model
• Social Accounting Matrices
• Poverty and Inequality Analysis and CGE
models 4
Goals and Needs
Goals:
• Measure the poverty impact of economic policy
• Measure the distributional impact of economic
policy
Needs:
• Rely heavily on household survey data
5
Household Surveys
• Single Topic
• In-between
• Multi-topic
6
Household Surveys
• Single Topic
• Labor Force Surveys( LFS) (ILO)
• Housing Surveys
• Census – national, UNFPA, 10 years
• In-between
• Multi-topic
7
Household Surveys
• Single Topic
• In-between
• Agricultural Surveys (FAO)
• Demographic and Health (DHS)
• Household Budget Surveys (HBS)
• Multi-topic
8
Household Surveys
• Single Topic
• In-between
• Multi-topic
• Multiple Indicator Cluster Surveys (MICS,
UNICEF)
• Survey on Income and Living Conditions (SILC,
EU)
• Core Welfare Indicator Surveys (CWIQ, WB)
• Living Standards Measurement Study Surveys
(LSMS) and Integrated Surveys (IS) (WB)
• Family Life Surveys (FLS, RAND)
9
What type of household data?
• Poverty measure: per capita or per adult
equivalent consumption
• Government programs receipt, format, costs
(formal and informal), use level
• Consumption of taxed goods
• Labor market participation (sector, hours,
earnings)
• Income by sources
10
Census
Purpose
• Accurate measure of the population of a
country
• Geographic distribution of the population
• Basic demographic information
11
Census
Sample
• Not a sample
• Universal coverage
• No sampling errors in estimates
• Some corrections for non-response may
be needed
12
Census
Content
• Short
• Trade-off between coverage and content
• Two types of errors: sampling and non-
sampling
13
Sampling vs. non-sampling errors
Total error
Non-sampling error Sampling error
Sample size
Census
Content
• Short
• Trade-off between coverage and content
• Two types of errors: sampling and non-
sampling
• Cost •Time
• Training •Non-response
15
Census
Content
• Demographic information: age, sex,
race/ethnicity, family and household
composition
• Housing information
• Others: basic education, labor, disability
16
Census
Poverty Measurement
•Albania: 2001 (1989)
• Basic needs •BiH 1991 (1981)
•Montenegro 2003 (1991)
– Subjective
– Limited monitoring use
– Limited use if looking at impact of
policies affecting taxes, tariffs or pricing
• Income: Panama example
17
Census
Uses
• Sample frame
• Link with household surveys for small
area estimation
18
Poverty Indicator by Commun,
Albania, 200
19
Labor Force Survey
Purpose
• Direct measurement of unemployment
• General characteristics of the labor force
20
Labor Force Surveys
Sample
• Relatively large samples
Need for precise estimates (change)
Desire to disaggregate to different
geographic areas
• Individuals of working age
21
Labor Force Survey
Content
• Characteristics of the labor force
– Demographics
– Education
• Sectoral distribution of employment
• Degree of formality
• Seasonal
• Income
22
Labor Force Survey
Poverty Measurement
Three problems:
• LFS typically capture partial, not total,
income
– Under-estimate welfare (vs. NA)
– Mis-ranking of households by welfare
level
23
Venezuela: Income and
Expend Survey
24
Venezuela: Social Survey
25
Labor Force Surveys, cont.
Poverty Measurement
Three problems:
• LFS typically capture partial, not total,
income
• Measurement Error
– Labor income measurement error
– At both ends of the distribution
26
LFS in Latin America
Item non-response
Salaried Self- Employer All Indep-
employed endent
Mean 3.9% 10.2% 12.0 10.6%
non-
response
rate
Source: Feres, 1998
27
Labor Force Surveys, cont.
Poverty Measurement
Three problems:
• Partial vs total, income
• Measurement error
• Income vs consumption measure
– Potential vs actual welfare
– Smoothing
– Measurement Error
28
Household Budget Surveys
Purpose
• Inputs to national accounts on consumer
expenditures
• Track changes in expenditures over time
• Track changes in the relative share of
different expenditures
• Weights for the consumer price index
29
Household Budget Surveys
•Non response rates (Eurostat, 2003)
•Bulgaria: 39.7%
Sample
•Estonia, 44%
•Hungary, 58.8% before replacement
• Medium size sample
•Romania, 21.6 %
• Sampling errors high at disaggregated
level
• High non-response rates
• In some parts: only urban (capital city or
group of large cities)
30
Household Budget Surveys
Content
• Total Income
• Total Consumption
• Short Demographics
• In FSU and Central Europe: agriculture
31
Household Budget Surveys
Poverty Measurement
• Possible to construct both total
consumption and total income
• Income may suffer from same
measurement errors as LFS
32
Household Budget Surveys
Poverty Measurement
• Consumption based welfare measure
• Purpose of an HBS survey is NOT to
measure welfare but to precisely
measure mean expenditures on specific
goods and services
• These are conflicting goals
33
Household Budget Surveys
Poverty Measurement
• Shortest possible reference periods
• Minimize number of omitted expenditures
• Good for precise measurement of regional or
national means
• Because of lumpy nature of purchases, not
good for comparisons among households
Need to adjust (lengthen) the
reference periods used in HBS 34
Household Budget Surveys
Poverty Measurement
• Focus on expenditures
– Not all expenditures are consumption
– Only purchases of durable goods and
housing
Durable goods: list of items owned by
household, age of items, current value
Housing: housing characteristics
affecting value 35
Household Budget Surveys
Uses
• Good for taxation issues
• Good for public (and private) transfers
• Sometimes has basic labor
• FSU and Central European countries:
agriculture
• No health, education data
• Limited for other areas
36
Multi-topic Household Surveys
Those with a focus on measuring poverty
• National Socio-Economic Survey of Indonesia,
SUSENAS
• Survey on Income and Living Conditions (SILC)
• Rand Family Life Surveys (FLS)
• Living Standards Measurement Study Surveys
(LSMS)
37
Multi-topic Household Surveys
Purpose
• Analysis of welfare levels and distribution
• Study links between welfare levels and
individual and household characteristics,
economic, human and social capital
• Social exclusion
• Causes of observed social outcomes
• Levels of access to, and use of, social services,
government programs and spending 38
Multi-topic Household Surveys
Sample
• Small sample sizes
• Trade-off issue: Quality and cost
considerations
• Limits ability to assess programs or policies that
affect small groups or small areas (over-
sample)
• Infrequent in many countries (exceptions, inter
alia, Indonesia, Panama, Jamaica, Peru,
Ghana) 39
Multi-topic Household Surveys
Content
Household Demographics* Agricultural Activities*
Housing* Non-farm household businesses*
Education* Food consumption (purchase, produced, gift)*
Health* Non-food consumption and durables*
Labor* Other income (incl. public &private transfers)*
Migration* Social capital
Fertility* Shocks, vulnerability
Privatization Time Use
Credit Subjective measures of welfare
Anthropometrics
Note: Starred modules are those most often used.
40
Multi-topic Household Surveys
Poverty Measurement
• Total consumption
– Longer reference periods
– Able to calculate use value of durables and
housing
• Total income
– Suffers from standard measurement errors
41
Multi-topic Household Surveys
Uses
• Poverty levels and distribution
• Social exclusion
• Public and private transfers
• Incidence analysis
• Tax policy
• Labor markets
• Education, health, social protections
• Changes in relative prices
42
• Monitoring (PRSP, MDGs), impact evaluation
Cross Section or Panel
Surveys?
• Substantive applications
• Methodological issues
43
Panels
1. Why do we need longitudinal data?
2. Designs for surveys across time
3. Advantages and uses of panels
4. Methodological issues
44
Understanding change
Longitudinal data are needed to
understand the process of change,
transitions between states, and the factors
or events that are associated with those
transitions
‘Longitudinal’ data is a catch-all phrase for
a wide range of different types of studies
45
Designs for surveys across time
Repeated cross sectional surveys
(e.g. Household Budget Survey, Labour Force
Survey)
• Common design for large government
surveys
• New sample drawn for each survey
• Carry similar questions each year
• Used for trend analysis at aggregate
46
level
Designs for surveys across time
Cohort Studies
• Sample often based on an age group
• Follow up same sample members at fairly
long intervals
• Developmental data as well as social and
economic data
• Data from parents, teachers associated with
cohort member
47
Designs for surveys across time
Rotating Panel Survey Survey of Income and
Programme Participation, USA (SIPP)
• Respondents stay in the panel for a set period of
time and are rotated out systematically and
replaced by new sample members.
• Used where the interviews are fairly close
together (every 3 to 6 months) and respondent
burden is high.
• Used where the collection of short spells e.g. a
few weeks unemployed or in receipt of a
particular benefit, is critical.
48
Designs for surveys across time
Indefinite Life Panel Surveys
e.g. Panel Study of Income Dynamics, USA – since 1968!
Living in BiH, LSMS Albania, LSMS Serbia
• Draw a sample at one point in time and
follow those sample members indefinitely (or
as long as the funding continues)
• Collect individual level data in household
context
• Repeated measures at fixed intervals (annual
data collection)
49
Panels from conference attendee
countries
• Albania – 4 waves 2002 - 2005
• BiH – 4 waves 2001 - 2004
• Serbia – 2 waves 2002 - 2003
50
Advantages of Panel Data
• Comparison of same individual over time - outcomes
• Track of aspects of social change
• Facilitates study of change and causal inference
• Minimise the problem of inaccurate recall
• Compare a person’s expectations with real change
• Look at how changes in individuals’ behaviour
affects their households
Identifies the co-variates of change and the relative
risks of particular events for different types of people
51
Changes in Employment
Status
A: CROSS-SECTIONAL INFORMATION
Unemployed
Net change -
Employed 0.1% unemployed
2001 2007
52
Changes in Employment
Status
B: PANEL INFORMATION
3.2% continuously
unemployed
Still Unemployed
Unemployed
5.1% unemployed 2001 but
employed 2007
5% employed 2001
but unemployed 2007
Employed Still Employed 86.7% continuously
employed
2001 2007
Net change - 0.1% unemployed Actual change is 10.1
53
Balkan Examples
Albania - 15% of the unemployed in 2002 had
made the transition to formal sector
employment by 2004
BiH - About half who were poor in 2001 remained
poor in 2004. Many individuals moved out of
poverty.
(Cross section headcount 18% for both years)
54
Employment and the labour market
Unemployment duration and exit rates
Do the unemployed find stable employment?
The effect of non-standard employment on
mental health
Temporary jobs: who gets them, what are they
worth, and do they lead anywhere?
Family and Household
Patterns of household formation and dissolution
Breaking up - finances and well-being following
divorce or split
The effect of parents’ employment on children's
educational attainment
55
Panel analysis
Mobility, poverty and well-being among the
informally employed – Peter Sanfey European
Bank for Reconstruction and Development
The origins of self employment, Leora Klapper et
al, WB (soon to use Albania Panel also)
The impact of health shocks on employment,
earnings and household consumption, Kinnon
Scott et al
56
A Sample
• Concept of ‘longitudinal household’
problematic for a panel - households
change in composition over time or
disappear altogether
• Individual level sample
57
Following rules
• All members of households interviewed at
Wave One
• Children born to these original sample
members
• Original members are followed as they
move house, and any new individuals who
join with them are eligible to be
interviewed
• New sample members are followed if they
split from the original member
58
Questionnaire design
• Core content carried every wave
• Rotating core questions
• One-off variable components
– lifetime job history
– marital and fertility history
• Variable questions to respond to new
research and policy agendas
59
Attrition in panel surveys
• Inevitable to some extent but can be
minimised
• Multiple sources of attrition in a panel
– refusal to take part
– respondents move and cannot be traced
– non-contacts
• Worry is potential bias if people who drop
out differ significantly from those who stay
in
60
UK Panel Wave 1 Respondents
Wave-on wave re-interview rates
94.9 94.8 97.5 97.2 97
100 90.3
87.7
90
80 70
70
60
50
40
30
20
10
0
Wave 1 Wave 2 Wave 3 Wave 4 Wave 5 Wave 6 Wave 7 Wave 8
61
Fieldwork
• respondent incentives as a ‘thank-you’
• extended fieldwork period for ‘tail-enders’
• refusal conversion programme
• tracking procedures during fieldwork
• panel maintenance between waves
– Change of Address cards to update addresses
– mailing of Respondent Report
– details of contacts with respondents between waves
62
Post-field checking and cleaning
• Within wave consistency
• Cross wave consistency and longitudinal
integrity
• Sample management
– individuals within households correctly
identified across time
– issuing of sample for each wave
63
The user database
• Longitudinal data is complex
• Provide users with database structure
which enhances usability
• Consistent record structure over time
• Key variables for matching and linking data
cross wave
• Consistent variable naming conventions
64
Added value
• ‘Added value’ to data set
• Extensive set of derived variables
• Production of weights
– household and individual levels
– cross sectional and longitudinal
• Imputation of missing data
• Flags to indicate imputed values
65
Conclusions
• Longitudinal panel data allows us to
answer research questions that cannot be
answered with with cross-sectional data
• Provides a different view of the world - see
process through the life-course not just a
static picture
• Is complex (but so is the real world) - so
needs to be well designed and conducted
with sufficient resources to be successful
66
System of Household Surveys
• GOAL: System able to respond to
evolving needs: not produce data X or
survey Y
– Determine data needs before they are
URGENT
– Identify appropriate instruments,
– Implement them properly, timely fashion,
– Analyze the resulting data 67
Improving the SHS
• Linking Users and Producers
• Providing adequate resources
• Continuous Survey Program
– Not necessarily permanent survey
– Benefits
• Avoid loss of capacity
• Create greater levels of capacity (building on existing)
• Economies of scale
• Policy makers know when data will be available
• Protects NSO from pressures for ad hoc surveys
• Ongoing system actually allows more flexibility and
responsiveness 68
Final points
• Welfare: household surveys- always
missing the homeless, street children,
institutionalized population
• No one survey can meet all needs,
review its purpose, coverage, content
and quality before using
• Need a system of surveys that meets the
needs of data users
69
