REPUBLIC OF SOUTH AFRICA GOVERNMENT-WIDE MONITORING & IMPACT EVALUATION SEMINAR Evaluating Impact: Turning Promises into Evidence Assessing the Impact of Municipal Infrastructure Grants (MIG) June 2006 ORGANIZED BY THE WORLD BANK AFRICA IMPACT EVALUATION INITIATIVE IN COLLABORATION WITH HUMAN DEVELOPMENT NETWORK AND WORLD BANK INSTITUTE Background of MIG • A consolidated grant mechanism, referred to as the Municipal Infrastructure Grant (MIG) approved by Cabinet on 5 March 2003 • Previous grant mechanisms included: - Consolidated Municipal Infrastructure Grant (CMIP) - Water Services Capital Grant - Community Based Public Works Programme - Local Economic Development Fund - Urban Transport Fund • MIG has an overall target of removing backlog with regard to access to basic municipal services over a 10 year period Background of MIG (Cont…) Vision of MIG – “ To provide all South Africans with at least a basic level of service by the year 2013 through the provision of grant finance aimed at covering capital cost of basic infrastructure for the poor”. Allocation of MIG Funds is based on the formula : MIG = B + P + E + N + M B = Basic Residential Infrastructure P = Public Municipal Service Infrastructure E = Social Institutions & Micro-enterprises N = Allocation to Nodal Municipalities M = Negative or positive allocation related to past performance of municipality MIG is a conditional grant (e.g. Conformity with IDPs, Poverty Alleviation through job creation – EPWP) Policy Questions, results framework • Is MIG providing infrastructure for a basic level of service to the poor? e.g. water, sanitation, roads etc. • Are economic conditions of poor communities improved as a result of MIG? • Are MIG funds used efficiently? i.e. greatest possible improvement in access to basic services at the lowest possible cost. Indicators • Provision of basic level of service e.g. water - Access - Affordability - Quality - Sustainability Indicators (Cont…) • Improved living conditions - Creation of Jobs - Skills Development - Reduced migration to cities - Improved Health Conditions - Access to economic opportunities Evaluation design • There are two main evaluation issues – (a) Examine the effectiveness of MIG programme on development indicators identified above, and (b) Evaluate the effect of technical support on programme uptake (and better outcomes). • For the first exercise, we plan to employ an instrumental variable methodology to use program resource allocation rules to instrument for backlog eradication at the municipality level. • The program has already been in place for two years. Thus, we really don’t have a true control group with 0 and 1 with treatment. • Instead we use a dose response approach, where we will examine the variation in amount of backlog reduction to examine changes in outcomes of interest. Evaluation design (Cont…) • Further, we will also account for the duration that the municipality has been part of the MIG program. • Using this strategy, it will be possible to examine the effect of MIG funds on outcomes of interest. • The second evaluation will be prospective in nature, where for the next years roll out, we will randomly provide technical assistance (via a programme to be determined later) for municipalities to prepare application for MIG funding and also help identify feasibility of projects. Evaluation design (Cont…) • Following the roll out in the year, we will then examine if municipalities with training increase project application success relative to those that do not. • As we have baseline data for the municipalities from the non financial census of municipalities, we can do some post assistance data collection to do a Diff in Diff type of estimation . Sample and data • • • • • • • • • MIG - Monthly Reports MIG - KPI Reports Sector Dept. DBSA (LGRC, Development Information Unit) Annual Analysis of IDPs Stats SA Provincial Offices DPLG Non Financial Census of Municipalities Human Resources Plan • DPLG/ PSC/ DPSA/ Treasury / Stats SA to coordinate the Impact Evaluation. • A Technical Team comprising the role players indicated below will be involved in the actual evaluation process: - Academic Institutions - Sector Departments - Parastatals (HSRC, DBSA, MRC etc.) - Salga - Provincial Departments and Municipalities - Community Development Workers (CDW) Time frame/working plan • A Technical Team will draft the work plan, taking into account the following issues: - Conceptualization - Design - Consultation process - Agreement on methodology - Training - Data collection and analysis - Publication - Dissemination Time frame/working plan (Cont …) • Time line - Jan 2008 • Target Date - Oct 2007 Key Activities : - Planning - Implementation - Reporting Budget • Technical Team, primarily DPLG to look into financial issues of this exercise.