CROSS-COUNTRY ARD SECTOR AAA ASSESSM ENT October 14, 2004 TABLE OF CONTENTS EXECUTIVE SUMMARY……………………………..…………………………………............i-iii 1. OBJECTIVES AND APPROACH………………...……..…………..………. ........................... 1 2. ASSESSMENT OF THE AAA PROGRAM OF WORK............................................................. 3 A. Overall Findings ................................................................................................................ 3 B. Findings By Quality Dimension........................................................................................... 4 C. Systemic and Cross-Cutting Issues...................................................................................... 9 D. Priority Areas for Improvement and Next Steps ..................................................................13 TABLES 1.1 Expenditure on AAA by ARD and Bank-wide.................................................................... 2 2.1 QAG ESW and AAA Ratings, ARD Sector............................................................................... 3 2.2 Overall Assessment of Seven Countries .............................................................................. 4 BOXES 2.1 What is Good AAA............................................................................................................... 4 2.2 The Ukraine Policy Group……………… ............................................................................ 9 2.3 Broad or Narrow Sector Work? ..........................................................................................10 2.4 A Supply-Driven Success .....................................................................................................13 ANNEXES 1. Approach Paper.......................................................................................................................15 2. List of AAA Reviewed..............................................................................................................18 3. Country Findings .....................................................................................................................19 4. Comments on the Technical Quality of Specific Studies in Bangladesh, Mexico and Tanzania......40 EXECUTIVE SUMMARY 1. An assessment of Analytical and Advisory Activities (AAA) was carried out by QAG in June/July 2004 for the Agriculture and Rural Development (ARD) sector, to review the quality of sector work in the context of the Bank’s strategy paper, Reaching the Rural Poor, approved by the Board in October 2002. The QAG review examined the “coherence” between the ARD sector strategy and a sample of recent sector tasks undertaken in seven countries, focusing on aspects such as technical quality of ARD analytic work, contributions to the CAS, PRSP and country rural strategies, as well as impact on the Borrower, Bank lending strategy and on other donors and stakeholders. 2. Considerable improvement in the quality of AAA activities managed by ARD was registered during the last two ESW assessments (FY00-02) managed by QAG. Results from the current review show continued improvement and the overall rating for 20 activities managed by ARD regional units was 95% Satisfactory or better (all ratings in this report are based on QAG’s new six point scale). While the quality of individual tasks is high, the amalgamation of tasks at the country level reveals shortcomings in coherence and impact. Of the seven countries examined in this review, three (Mexico, Morocco and Ukraine) had satisfactory ARD sector work overall, though no country was rated Highly Satisfactory. Two countries (Kazakhstan and Tanzania) had Moderately Satisfactory ratings, and two countries (Bangladesh and Ghana) were rated, respectively, Moderately Unsatisfactory and Unsatisfactory for quality of ARD sector work. 3. Implementing the goals of the new rural strategy remains a work in progress. Reaching the Rural Poor was a major exercise to reposition and refocus the ARD sector as a priority locus of business activity for the World Bank based on the preponderance of poverty in rural areas. The strategy challenged those inside (and outside) the sector to reassert the strategic importance of rural development through enhanced performance in lending and AAA. It emphasized that “improvements in the well-being of the poor will only be possible through enhancement of their productive, social and environmental assets. This means increasing the productivity and growth of both the farm and non-farm economies.” 4. Attaining the goals set out in the Reaching the Rural Poor strategy is bound to be a slow process. With many of the tasks included in the sample having been launched prior to Board approval of the new direction, the strategy is too new to have had a significant impact on the strategic direction of these activities, or on the volume of AAA in the sector. From an overall perspective, however, the Panel found that in several instances the effectiveness of ARD sector work at the country level lacks consistency and coherence over time, is poorly integrated with lending programs, and reveals weak linkages with other sectors and other donors’ programs. 5. The Panel recognizes that the means for achieving many of the strategy’s objectives depend on budgetary decisions taken by Country Directors and thus lie outside the control of ARD managers and staff. But some are within their control, and that provides the Panel with an opportunity to offer recommendations that may assist in achieving the goals. At the same time, the Panel wishes to alert country departments of their role in making the ARD strategy work by according it the priority it warrants in country work programs. The ARD strategy is a Board- approved document that clearly identifies the fact that poverty is more prevalent in rural areas. Cross-Country ARD Sector AAA Assessment ii Yet the Panel found that rural poverty is still dealt with in a cursory manner in CASs, CEMs, and even PRSPs. 6. Key findings emerging from this review include the following points: • Technical quality of analytic work at the task level was uniformly high for tasks managed by ARD units, but most of the tasks managed by non-ARD units that were identified as having sections pertinent to the rural sector were not particularly relevant. • ARD AAA is too often fragmented, ad hoc, financed off-budget, and poorly documented in the Bank's internal records, weakening institutional memory and undermining the coherency of AAA work programs. • Sector "silos" are very apparent in the AAA work program, with little or no evidence of interaction between sector departments. • The integration of ARD AAA into lending was highly variable. Three countries showed good integration of sector work (Morocco, Mexico, Ukraine) but two had only tenuous links to the portfolio (Bangladesh, Ghana). Sector-wide, coherence and integration was the weakest dimension of AAA work. • While AAA often shows good impact on other donors, the Bank's tendency to want to always take the lead, and often to overlook analytic work done outside the Bank, has probably undermined true synergy in many countries. • Reports too often were not translated into local languages or formally published, which reduced impact. Also, the Bank needs to make more use of Web-based dissemination for AAA reports. • The quality of management input into AAA tasks was variable. Larger analytic tasks showed strong involvement from the conceptual stage through to implementation and dissemination, but in smaller tasks management attention was often hard to detect after the initial concept stage. • The Bank appears to have a very porous institutional memory and an entirely inadequate filing system (electronic or otherwise). Many reports were missing or could only be located with difficulty. 7. The following recommendations are addressed to specific Bank audiences that have a role to play in helping strengthen the impact of AAA work on reducing rural poverty: For the ARD Sector Board • Continue to publicize and disseminate best practice in AAA. For ARD Regional Sector Units • Review measures that would help ensure the availability of experienced sector economists that can devote most, if not, all their time to AAA activities. Cross-Country ARD Sector AAA Assessment iii • Sector staff should continue to champion the importance of AAA by emphasizing the preponderance of poverty in rural locations and the value of ARD AAA in meeting the Bank’s poverty alleviation objectives. • Greater use should be made of QERs, especially for less experienced task managers. This should be combined with more extensive use of external reviewers. • Make more extensive use of studies conducted outside the Bank, conduct literature searches before beginning a task, and consider updating old studies before launching new ones covering the same topic. • Greater efforts are needed to accurately record AAA tasks in the SAP and to ensure that reports are properly archived and readily available to the staff and the client. For Country Departments • Achieving the Bank’s poverty reduction objectives requires greater prominence for the rural sector in country programs. This calls for a more consistent allocation of funds to ARD AAA to maintain the increase seen in FY04. • Ensure that rural poverty aspects are properly integrated into the analytic work of other sectors and guard against the tendency for staff to remain fixed within narrow sector “silos.” • Earmark funds for dissemination of AAA findings and recommendations and for posting them on the web. For QAG • This pilot review was not able to fully address some issues due to the limited interaction with staff and managers. This aspect should be reviewed in any future assessment of this type. • The impact of sector AAA on country lending programs and other stakeholders is best examined in conjunction with field visits that provide a much deeper basis for informed judgments. I. OBJECTIVES AND APPROACH 1.1 This exercise is a pilot review of the Agriculture and Rural Development (ARD) Analytical and Advisory Activitie s (AAA) undertaken during FY01-04 in seven countries and five regions, broadly representative of the different conditions being faced by ARD staff. The sample also included rural- related AAA studies conducted by other units. The objectives of the review are to assess the overall quality and adequacy of AAA in the rural sector, the impact of AAA and the Sector Strategy Paper, Reaching the Rural Poor, on country rural strategy and activities of other donors, and its contribution to the formation of the Bank’s lending strategy/project design in the sector. Although many of the tasks in the sample were launched before the sector strategy paper was approved by the Board, the Panel found that most of them were closely aligned with the broader vision of the strategy. One objective on which the Panel could not make a judgment through the pilot assessment process was the extent to which the rural sector strategy paper and AAA activities have brought about changes in the content and quality of the Bank’s lending activities in the rural sector. Documentation available on the entire AAA process was gathered and read by the panelists. Limited information was collected on the topic of “Bank inputs and processes,” but no country management staff were interviewed. With the exception of Bangladesh and Mexico, no field visits were undertaken. Five countries were chosen on the basis of their inclusion in a current or previous QAG country AAA assessment, and two countries were suggested by ARD as representing examples of very good and rather inadequate AAA. (Detailed terms of reference are provided in the Approach Paper, which is Annex 1 of this report.) 1.2 The review was conducted by a team of seven sector specialists and employed a phased approach. First, 31 individual tasks (27 ESW and 4 NLTA) were assessed. Out of this total, 20 activities were managed by ARD units, while the balance were tasks managed by other units but identified as having relevance to work being carried out in the rural/agriculture sector. It should be emphasized that the non-ARD AAA activities reviewed here were examined strictly from the point of view of their contribution to advancing the ARD analytic agenda; this was not an assessment of their individual merit. 1.3 In the second phase, individual task results were aggregated at the country level with panelists paying close attention to the comprehensiveness of the AAA program, its likely impact on, and strategic relevance to, the country’s rural strategy, taking into account the sector strategy paper approved by the Board and referred to above. The panel also looked at the quality of the Bank’s management of the ARD AAA program in each country. (Results from this part of the assessment must be handled with caution since in at least three countries only a small number of ARD AAA activities were carried out, while in others some of the tasks were only indirectly related to the rural sector.) The assessment’s third phase is the overall sector performance. With the studies included in the sample accounting for about 13% of total AAA activities conducted in the sector, the sample size is not large enough to yield robust results at the Bank-wide level, but it does provide an opportunity to learn several lessons and establish trends. Finally, three AAA tasks were examined by an independent specialist in agriculture and rural development to identify strengths and weaknesses related to technical quality and policy relevance (see Annex 4). 1.4 Table 1.1 provides a breakdown of the expenditure on AAA tasks managed by the ARD units and Bank-wide. At this time, there are no benchmarks to determine the appropriate level of sector activities and funding, because these must be determined on a country-by-country basis according to need. As can be seen, until FY04 the amount allocated to ARD AAA has been essentially flat. In FY04, there has been a significant increase in expenditure on ARD AAA, with about half the increase coming from trust funds and the other half from the Bank’s budget, but at Cross-Country ARD Sector AAA Assessment 2 this time it is unclear if this recent increase is a temporary ‘blip’ or a longer-term shift to a higher level of funding, though the increase is in line with the recommendations of this report. TABLE 1.1 EXPENDITURE ON AAA BY ARD AND BANK-WIDE FY00-04 ($ million) ARD as ARD Bank % of Bank FY00 9.2 196.6 5% FY01 6.8 181.1 4% FY02 9.4 195.7 5% FY03 10.3 253.3 4% FYO4 16.4 245.4 7% 1.5 In line with the agreement reached with the ARD Sector Board, this pilot review was a desk study and interviews with task and sector managers were limited to a single meeting. This introduced some limitations. Conducting the assessment without interviewing all sector and country staff involved in a task often made it impossible to pursue central issues in detail. There were no individual task discussions (though some had previously been assessed by QAG), and the lack of a field visit meant that the opinions of the client and other stakeholders could not be assessed directly. Moreover, in three of the sample countries, only a small number of ARD AAA activities were carried out, while in others, some of the tasks were only indirectly related to the rural sector. Finally, robustness of the findings from aggregating the seven country assessments is affected by the small sample size. 1.6 Ratings were carried out using QAG’s newly-introduced six-point rating scale, with the following guidelines for assessment: 1 – Highly Satisfactory Best practice in most respects and no significant deficiencies 2 – Satisfactory Satisfactory or better on all aspects 3 – Moderately Satisfactory Satisfactory or better on all key aspects but significant missed opportunities 4 – Moderately Unsatisfactory Significant deficiencies in a few key aspects 5 – Unsatisfactory Significant deficiencies in several key aspects 6 – Highly Unsatisfactory A broad pattern of deficiencies Cross-Country ARD Sector AAA Assessment 3 II. ASSESSMENT OF THE AAA PROGRAM OF WORK A. OVERALL F INDINGS 2.1 Recent QAG assessments of ESW tasks (FY99-02) have found the quality of ARD analytic work to be close to the Bank-wide average (Table 2.1). The present assessment, which is based on a review of 31 tasks in seven countries, found that tasks managed by ARD units were of high quality (95% satisfactory rating for 16 ESW tasks and four non-lending TA) and were closely aligned with the agenda outlined in the sector strategy paper. Sections of studies managed by other units that were identified as being pertinent to the rural sector, however, were found to be of limited relevance and of inferior quality. When aggregating the individual tasks at the country level, the review found a pattern of deficiencies in analytic work and limited impact on the country/Bank strategic approach to the sector. Although the sample is too small to derive conclusive Bank-wide results, the Panel judged the quality of ARD AAA work at the country level for two out of the seven countries in the sample to be Moderately Unsatisfactory overall and the others were rated Satisfactory. The strongest aspect of the ARD AAA work program at the country level was the internal quality of analysis, while strategic relevance, coherence and integration, and likely impact demonstrated significant shortcomings. Table 2.1 ESW and AAA Ratings, ARD Sector % MODERATELY SATISFACTORY OR BETTER QAG ESW Assessments, FY99-02 • Bank-wide rating 85% • ARD rating 81% QAG ARD Sector AAA Assessment, 2004 • ratings for 20 AAA tasks 95% 2.2 Within the sample of countries assessed by QAG, there were of course specific examples where these weaknesses were not apparent, but overall the shortcomings seem to be in these areas: • Inappropriateness of the coverage, scope and relevance of ARD AAA program in the context of the CAS framework, and in relation to the importance of the rural sector in the economy, and to the percentage of people classified as poor; • Failure to effectively engage government, local agencies and other donors in carrying out the tasks and disseminating results; • The absence of a coherent long-term strategy for undertaking AAA activities in a country, and to clearly define objectives and target audiences; and • Limited impact of the analytical work on the client and the Bank due to shortcomings in dissemination, the short “shelf-life” of reports, and inadequate systems for archiving, updating, and accessing AAA reports. Cross-Country ARD Sector AAA Assessment 4 Box 2.1 What Is Good AAA? Drawing on its accumulated experience in assessing AAA, QAG has identified key characteristics of good sector work. Although they were formulated for ESW, they are equally true for AAA and were used in this assessment: • Clarity of purpose, intended audiences, and expected impact • Operational relevance and client receptivity • Clients and stakeholders as partners with well defined roles • Sound, customized analysis blending local knowledge with global insights • Substantive conclusions, clearly presented and argued • An effective program of dissemination and follow-up • Realistic budget and timetables • Agreed quality assurance mechanisms and monitoring milestones B. Findings by Quality Dimension 2.3 As pointed out in paragraph 1.5 results from aggregating the findings from the seven countries included in the sample are not statistically robust and should be interpreted with care. An assessment of this limited sample shows the strongest aspects at the country level to be the internal quality of activities, dialogue and dissemination and Bank inputs. Areas in need of improvement are strategic relevance, coherence and integration, and likely impact (Table 2.2). The following paragraphs summarize the findings for each of the six dimensions used in the assessment. On just about every dimension of the review, the findings include examples of good practice but also many areas in need of improvement. TABLE 2.2 OVERALL ASSESSMENT OF S EVEN COUNTRIES Overall Rating 4 MODERATELY UNSATISFACTORY Strategic Relevance 4 Moderately Unsatisfactory Internal Quality 2 Satisfactory Dialogue and Dissemination 3 Moderately Satisfactory Coherence and Integration 4 Moderately Unsatisfactory Likely impact 4 Moderately Unsatisfactory Bank Inputs 3 Moderately Satisfactory 2.4 Strategic Relevance. This dimension measures the appropriateness of coverage, scope and relevance of sector AAA programs in the context of the CAS framework and countries’ PRSP/CDF or equivalent strategy, and relevance of sector AAA to the Bank’s sector strategy paper. • Examples of Good Practice: (i) Strategic planning. The Morocco sector studies followed the CAS outline of a three-year program to expand and complement the available analysis and contributed to the development of strategic planning; (ii) Focus on poverty aspects. In Tanzania, where there is considerable rural poverty and a history of zero sector growth, the AAA program was focused on changing incentives to promote Cross-Country ARD Sector AAA Assessment 5 growth in a manner that takes account of poverty implications; and (iii) Sector Strategy Paper. Most tasks are closely aligned with the broad vision of “Reaching the Rural Poor,” are consistent with that paper’s strategic options, and have broadened the analysis from agriculture to rural development more generally. The Kazakhstan Livestock study is a good example of a sound strategic choice of analytic orientation with excellent coverage of the poor population. • Aspects Requiring Improvement: (i) Coverage of sector studies. The totality of the AAA program does not address integrated agriculture and rural development and growth issues in a coherent manner, and frequently overlooks key issues in the sector; (ii) Descriptive studies. A number of studies included in the sample are descriptive of the current situation rather than being forward looking and aimed at helping government overcome emerging problems; (iii) Conclusions and Recommendations . In many studies the conclusions and recommendations tend to be quite broad, giving the borrower few indications of priorities and sequencing of recommendations; and (iv) Studies alignment with the CAS. In some instances, the AAA activities were not aligned with the poverty reduction objectives of the CAS, and in the case of Ghana, where poverty is predominantly rural, the absence of an ARD AAA program is a serious shortcoming. 2.5 Internal Quality. This dimension examines the technical quality of sector AAA analysis, and the quality of the empirical evidence used (both quantitative and qualitative). • Example of Good Practice: (i) Use of earlier studies. In Ukraine, AAA tasks as a group built on strategic work carried out in the 1990s, addressed key issues, and were high quality. The work greatly benefits from its use of empirical evidence, notably the database established within the Agriculture Policy Group; and (ii) Studies rated Highly Satisfactory. Other AAA tasks rated highly satisfactory include the Bangladesh Climate Change Study, the Morocco Agriculture Sector Note, and the Morocco Cereals report. • Aspects Requiring Improvement: (i) Use of Data. Inadequate attention to the collection of raw data within the country (this could frequently be best achieved by engaging local institutions as was done in Kazakhstan and Ukraine); (ii) International Experience . Many studies fail to make effective use of examples from, and convincing explanation of the experience in, other countries that have successfully addressed similar sector problems; (iii) Actionable Recommendations . Studies frequently contain numerous recommendations that are not very explicit and “actionable,” and (iv) Literature Search. Studies are making limited use of empirical evidence and frequently fail to make extensive search of sector-relevant literature (including non-Bank analytic work, which is overlooked in many studies). 2.6 Dialogue and Dissemination. This dimension examines the contribution of sector AAA programs to promoting policy dialogue and institutional development at the country level, the quality of client participation, and partnership arrangements with other donors. • Examples of Good Practice: (i) Client led studies. Where studies are managed by the country (or independent local units) and with broad participation from a wide range of stakeholders, ownership is enhanced, especially where this is combined with the Bank assuming a supportive role and furnishing the local experts with worldwide example of how other countries have dealt with similar issues; (ii) Dissemination of studies: Dissemination efforts through workshops and the translation of reports into the local language enhance the impact of the studies on policy makers in the country; and (iii) Collaboration with donors. There are many examples of joint financing of studies with Cross-Country ARD Sector AAA Assessment 6 other donors but by far the most effective dialogue occurs when the donors are actively participating in the preparation of a study. In general, donors’ programs appear to be influenced by the findings of Bank AAA. Individual tasks noted for good attention to dissemination include the Mexico Land Policy Review, the Kazakhstan Fisheries sector study, and the Bangladesh Poverty Assessment. • Aspects Requiring Improvement: (i) Borrower participation: Active client participation in the preparation of studies, which generally enhances ownership, is limited in most instances; (ii) Shelf life of studies. Many reports included in the sample have a far too-short shelf life in an environment where policy issues really do not change that much across several years. There are few examples of attempts to keep reports alive and to update (rather than re-create) them at appropriate intervals; (iii) Dissemination of studies. The arrangements for disseminating studies are often an afterthought. Efforts are unplanned, unbudgeted, and simply not undertaken in a workmanlike manner. As a result, client uptake is weak. Reports are rarely formally published, and few are available in short policy note format (translated to the local language) and posted on the web. There are even instances where policy studies intended to influence policy makers were not delivered to government; and (iv) Capacity building. Few of the AAA programs include capacity building as an explicit objective. Where this has taken place (Ukraine, Morocco, and to a lesser extent Kazakhstan), it has substantial impact on the country’s analytical capacity and the ownership of the studies and their recommendations. (See Box 2.2 for an example of an innovative capacity building effort that has proven highly effective.) 2.7 Coherence and Integration. This dimension assesses the coherence of sector AAA in terms of inter-linkages across other AAA tasks and sector or thematic areas, integration of sector AAA with lending programs, synergies with other donor programs, coherence of sector AAA over time, and consistency of effort. • Examples of Good Practice: (i) Impact on Bank lending. In Ukraine, AAA work has had direct impact on the Bank’s lending program; and (ii) Programmatic approach. In Morocco, a three-year programmatic approach to AAA has resulted in excellent coherence of analytic work over time, while consistency of effort benefited from designating a lead sector economist with adequate resources for this role. • Aspects Requiring Improvement: (i) Linkages with other sectors. Frequently, linkages with other sectors are not readily apparent and in fact AAA tasks are often “enclave”-type undertakings. In the cases of PERs, for example, which are supposed to pull together themes for various sectors, the technical input from ARD sector specialists is often limited to reviewing drafts at a late stage when such inputs are least effective. Even in countries in which the Bank is heavily involved in areas such as rural infrastructure, rural education and health, integration of ARD sector work was often hard to detect; and (ii) Linkages with lending operations . Sector studies are frequently undertaken to justify/support operations in advanced stage of preparation instead of preceding such preparation efforts. 2.8 Likely Impact. This dimension measures the likely impact of AAA programs on clients, on the Bank, on other donor programs, and on other stakeholders and beneficiaries (NGOs, civil society, the private sector). • Examples of Good Practice: (i) Client led studies: Attempts to building ownership and coalitions for change have the biggest impact where studies are channeled though local Cross-Country ARD Sector AAA Assessment 7 working groups and participatory methods are used for analytic tasks; (ii) Impact on the Bank and Client. Morocco’s p rogrammatic approach to AAA has improved impact within the Bank, and could have broader benefits if the approach is adopted in other sectors and countries. In Ukraine, partnerships with key donors are well developed and policy priorities are clearly agreed within the group. The establishment of the Policy Analysis Group is a best practice, an innovative example of enhancing country capacity and knowledge transfer. Individual tasks which were highly rated for likely impact include the Mexico Crop Insurance report, the Mexico Land Policy review, the Ukraine Agriculture Policy Note, the Morocco Agriculture sector note, and the Bangladesh poverty assessment. • Aspects Requiring Improvement: (i) Client impact. Studies that fail to have an impact on the borrower suffer from inadequate client focus, absence of an effective and long- term strategy for dissemination, the absence of clear conclusions and “actionable” and sequenced recommendations; and (ii) Protecting the Bank’s Knowledge base. Failure to complete ana lytic tasks and to ensure ready access to final reports diminishes the impact of sector studies on the Bank. 2.9 Bank Inputs and Processes: This dimension examines the adequacy of funding, the efficiency of resource use, the quality of support from other Bank units, managerial attention to AAA quality, the effectiveness of the peer review process, monitoring and evaluation of sector AAA, and the quality of AAA information in the SAP and Business Warehouse. • Examples of Good Practice: (i) Management Input. In general, it appears that managerial attention is quite high at the inception phase of large tasks, and in these instances there is evidence of lively discussion on the merits and composition of the activity in point. Where country management is also involved in AAA tasks, this frequently pays off in terms of high quality product and high impact; (ii) Efficient use of resources. Major tasks are making efficient use of available resources, and (iii) Client and stakeholder feedback. In Ukraine, the Agricultural Policy Group has proven to be a very effective conduit for sharing views and soliciting feedback on sector analytic work. At the individual task level, no AAA tasks were rated highly satisfactory with respect to Bank inputs and processes. • Aspects Requiring Improvement: (i) Multi-year budgeting. Under current Bank systems, task teams are unable to make efficient use of resources because of uncertainty of budget allocation from one fiscal year to the next; (ii) Country team’s participation. Greater active involvement from country economists and other senior members of country team in the selection and preparation of AAA activities would probably help to strengthen the integration of the tasks with others being undertaken in the country; (iii) Monitoring and Evaluation. Consistent monitoring of ARD’s sector activities at the country level and the consistency of these activities over time was particularly lacking; (iv) Peer reviews. The Bank’s peer review process appears to have deteriorated; evidence points to cursory reviews, infrequent attendance by designated reviewers in review meetings, and patchy use of external reviewers; and (v) Implementation and Dissemination. Management attention to AAA tasks seems to fall off after the initial concept review, which weakens the impact of the work. 2.10 There are several implications arising from the findings on Bank inputs. As shown in Table 1.1, with the exception of FY04, budgetary allocations (both TF and BB) for AAA activities in the sector have remained essentially flat; this was also true for the Bank as a whole though here the sharp increase in expenditures preceded that in ARD by one year. The Panel was Cross-Country ARD Sector AAA Assessment 8 unable to judge whether the sudden increase in budget in FY04 is the beginning of a trend or a ‘blip,’ but noted that about half the increase was from trust funds, which would seem to indicate an aggressive strategy on the part of staff to secure more resources rather than an explicit policy decision by country departments to provide enhanced budgets to ARD AAA. Unless the level of expenditures is sustained at about the FY04 level, the Panel is concerned about the prospects for improving and sustaining the Bank’s analytic and policy commitments in agriculture and rural development, for several reasons. • First, it is increasingly difficult for sector managers to retain well-qualified economists who are fully dedicated to sector work. At best, sector economists who are fungible with lending work are selected, and when this occurs it is inevitable that first priority is given to lending activities. This is a logical sector management reaction to limited and uncertain AAA budgets, but it has serious implications for the flow of quality AAA work and the generation of knowledge. There are several examples of experienced sector economists leaving the sector for PREM or other positions and so being “lost” to the Network. • Second, there are serious deficiencies at the AAA work program level which would not be redressed by occasional increases in funding. The review came across only one example of, say, a planned, rolling three-year work program of AAA and a notional work program, loosely defined, ready to respond according to resource availability. The general absence of a meaningful AAA work program means that it is very difficult to assemble a comprehensive and cumulative knowledge base for the country, and to maintain an effective staff skill mix focusing on analytic work. Thus AAA work tends to be reactive rather than proactive, and inconsistent in focus from year to year. • Third, under the Bank’s present structure and budget, task managers of AAA more often simply manage the process, outsourcing the fieldwork and writing to consultants. The Panel raises the question of the implications of this. Where will the sector get its next generation of task managers skilled in sector work, policy analysis, and policy dialogue? What are the implications for quality and the “knowledge bank” if this should persist? • Fourth, insufficient funding for AAA also means that task managers must spend considerable time chasing and managing trust funds and other off-budget sources of funding. This would not appear to be the best use of their time, nor is it appropriate that a core Bank activity and area of comparative advantage should be so exposed to the whims of trust funding. (As seen in Table 1, however, compared with the rest of the Bank BB funding accounted for a much higher share of the total.) Cross-Country ARD Sector AAA Assessment 9 BOX 2.2: THE UKRAINE POLICY GROUP Ukraine was one of the slowest agricultural sector reformers in the Former Soviet Union during the 1990s. The Bank’s traditional approaches to inducing policy change were not working. We were seen as outsiders who did not understand their problems. The decision was made to establish a policy analysis unit, with a staff of about 10 Western-trained Ukrainian nationals. The goal of this unit was to carry out policy analysis for key policy makers who were having problems getting access to analytical support for their ideas. Several lessons were learned: • National Policy Advisors are Critical. In the Ukrainian context, policy advice provided by foreign advisors had minimal impact. Western trained national policy analysts could more effectively assimilate new policy ideas, and communicate them to senior policymakers. • Re-training the Old Guard is not effective. This project demonstrated that the focus on educating a new generation yields far better results than trying to reform the “Old Guard,” and that it may often be more effective to create new institutions than to try to reform old ones. • Support of a “Champion” is Essential. The policy analysis unit was fortunate in that it had a series of enthusiastic “champions” at the highest levels of Government who supported them and made use of their policy advice. • Financial Autonomy is Important. Financial autonomy has been important to the group in terms of its ability to provide objective, independent advice and also to move their support to wherever it was required by reformers in Government or the Parliament. • Donor Coordination Improved. Having a commonly-funded policy advisory unit was important in harmonizing donors’ policy advice. • Cost Effectiveness. For the Bank, this was a low cost - high impact intervention. Using about $700,000 in Bank Budget over 6 years, the region mobilized an additional $5 million in funding from USAID, EU and DFID. The policy analysis unit has produced hundreds of quick turnaround reports, comments on laws, and meetings with policymakers which have significantly improved the agricultural policy environment. C. Systemic and Cross-Cutting Issues 2.11 One over-arching issue encountered in the assessment was the rather low level of importance assigned to ARD AAA in country programs in relation to the importance of the rural sector in the economy and as a source of the poor. In many respects agriculture and rural development appear to have taken or been assigned a secondary role, even though these are sectors that might appropriately be seen as a core business line for the Bank, with high importance for poverty reduction and the prospects for achieving the Millennium Development Goals. Panel members expressed concern over the disconnect between the importance of agriculture and rural development to economic performance, growth, and poverty reduction, and the amount of budget allocation to ARD AAA. In many countries, only one piece of ARD AAA is conducted per year, and in some years, none are undertaken. It is true that new activities, such as PRSPs, do have significant ARD components yet do not get “counted” as ARD activities. Yet the Panel questions whether, as the “Knowledge Bank” on which many other donors also depend, Cross-Country ARD Sector AAA Assessment 10 the World Bank is investing sufficiently in ARD knowledge and whether it is fulfilling its Board mandated strategy. This is a matter for the attention of country departments. 2.12 Directly related to the question of defining the audience is clarity of objectives. ESW and NLTA have multiple objectives depending on the particular piece of work and context. A well- designed piece of AAA identifies up-front the prime audience and casts the report with the prime audience in mind. What the Panel failed to find was a systematic discussion at inception of: (i) the use to which a report would be put, (ii) a clear identification of the target audience, and (iii) how the target audience might affect the design of the work. (See Box 2.3 and Annex 4 for further details). Seldom is the prime purpose and audience clearly spelled out in the concept paper, and thus many Bank reports miss an opportunity to send focused and sharp messages to the prime client. This also has consequences for the dissemination strategy, which often seems to lack focus and follow -up. 2.13 Related to this is the issue of demand-driven versus supply-driven AAA work. In the recent past, it was recognized that the Bank undertook too much supply-driven AAA and country uptake and ownership tended to suffer. “Reaching the Rural Poor” concluded that to be effective, rural strategies must be “owned” by the country, and preferably the team undertaking the work should include government officials and local experts. The Morocco rural strategy is a good example of such an approach. But there are also dangers of taking a demand-driven approach to the extreme where the Bank has very weak or little influence on the analytic agenda. In countries where decisions on sectoral investment allocation are made by political elites who are not very interested in agriculture or rural issues, adherence by the Bank to the principle of demand-driven sector work tends to exacerbate such biases. In these countries, the ARD sector work often tends to be under-funded relative to its importance to the economy and to poverty. 2.14 Sometimes this becomes extreme, such as in Kazakhstan, where only work approved and co-financed by the Kazakh government is undertaken. The Panel found that although AAA is being carried out on several special topics that inform the Bank’s lending in the country, there BOX 2.3: BROAD OR NARROW SECTOR WORK? In practice, a feasible agenda for sector reviews has to focus either on broad-ranging description with relatively superficial analysis (“wide and shallow”), or a relatively narrow focus with more detailed analysis of causes of problems (“narrow and deep”). The prevalent approach to sector work appears to be the wide-and-shallow type. When the sector is as complex as the rural sector, this descriptive task alone is a large undertaking. Combining this with in-country expert opinion and the Bank’s general experience-based positions on policy matters, one model of sector work is a report that presents a large amount of data along with fairly generic policy recommendations. The narrow-and-deep model of sector work is a report that focuses on a few issues, and analyzes them in detail. An example of this genre is the paper on Mexico’s program of privatizing land holdings. The work involved two substantial surveys of residents affected by the policy changes, an econometric analysis of the data, and conclusions based narrowly on the findings. The principal problem with narrow-and-deep sector work is that it is costly per conclusion. It would be impossible to fund such work for every lending program and policy reform supported by the Bank. The principal problem with wide-and-shallow sector work is that many of the recommendations given are weakly evidence-based and may fail to convince a skeptic that they are is preferred to alternatives. Thus, what genre of sector work is preferable? Clearly, both genres have a function that provides value to the Bank and its clients. The important point is that well- managed sector work will identify, early on, the clear purpose of the work and this will inform the decision on wide versus deep. Cross-Country ARD Sector AAA Assessment 11 has been no recent comprehensive sector strategy, survey, or assessment. The risk in such a situation is that the Bank conducts studies of limited relevance and is unable to undertake broad policy/strategy pieces of work that serve to provide a vision for the sector. 1 In so doing, the Bank is losing knowledge and compromising its unique strength of having a broad and encompassing view of the sector. It can no longer provide informed strategic advice to certain governments and is in danger of losing its seat at the agriculture and rural development policy table. 2.15 AAA work reviewed in Mexico provides an interesting counter-example of supply-driven AAA, where the Bank unilaterally undertook a major piece of informal sector work that became hugely important to the incoming government and provided a roadmap for Bank intervention and dialogue (see Box 2.4). The panel concludes that a balance is required between supply-and demand-driven approaches, and that careful judgment is needed on that balance. Some work that is initiated unilaterally by the Bank to fill knowledge gaps or to respond to topics ignored by the country is needed. At the same time, extreme demand driven approaches as is seen in Kazakhstan come at a cost to the Bank’s knowledge and comprehensive view. As in much of life, polar solutions are seldom ideal. 2.16 Cross-sectoral collaboration in ARD AAA is seldom being practiced. The panel could find only one example where cross-support from specialists in other sectors was employed. There is considerable evidence of ARD staff collaborating with others in activities such as the PRSP, CAS, CEM and PERs, but there is very little the other way round. The sectoral silos remain robust in the Bank. The present structure and task budgeting makes cross support problematic. Staff aspire to task management and there appear to be no incentives for collaboration. Yet sectors do not operate i n isolation and strong inter-relationships and overlaps occur across sectors and expertise. Ways to facilitate cross support would benefit the Bank. The use of specialist anchor staff is not systematic, either. Some regions use anchor and DEC staff well, but they are the exception rather than the rule. The evidence seems to be that the Bank is not bringing state-of-the-art expertise to the task at hand and therefore the tasks often do not represent cutting-edge knowledge on the topic. 2.17 Related to cross-sectoral collaboration is the use of external expertise. In some cases, international experts were used as reviewers or were part of the team. But this seemed too infrequent. Further, subcontracting AAA or parts of AAA (data collection and analys is, say) to local think- tanks/consulting firms was the exception rather than the rule. In instances where this was practiced, it appeared to be an effective way to leverage resources while at the same time contributing to local knowledge-enhancement and country ownership. Some TTLs use external expertise well on teams but very infrequently as peer reviewers. ARD once had a roster of internal and external reviewers that staff could draw on but this list appears to have disappeared. Clearly external revie wers who are knowledgeable on international development and Bank practice would enhance the quality of AAA. The panel recommends that staff be more pro active in using external reviewers. 2.18 Another aspect of overly-inward looking Bank AAA work is neglect of non-Bank analytic studies. The Panel was struck by the fact that in not one of the seven interviews was there a single mention of ESW undertaken by others that was used by the Bank. This includes donors, multilaterals, academics, and think tanks. It was noted that while ESW reports review the Bank’s literature well, there seldom is mention of literature external to the Bank. It is as if “if it is not done by us, it doesn’t exist.” Clearly, authors of ARD ESW are missing valuable sources of information, lessons of experience, empirical findings and theoretical developments when they restrict their review to in-house work. 1 In Kazakhstan, for example, a very good piece of work, requested and partly financed by the government, was done on fisheries. This was an expensive piece of ESW yet will only benefit a few thousand fishermen. Over the period reviewed, no similar study has been conducted on agriculture, a sector with many multiples of the number of people. Cross-Country ARD Sector AAA Assessment 12 2.19 Country Departments and ARD. One purpose for producing “Reaching the Rural Poor” was to inform a wider audience in the Bank of the importance of the rural sector to the objectives of the Bank. In particular, it was hoped that the evidence-base showing poverty to be concentrated in rural areas might influence country departments to accord a higher priority to work in rural space. Despite a vigorous dissemination campaign by ARD when the report was published, there is no evidence from the sample that country departments have yet been influenced by the new rural strategy. Indeed, in one instance the Panel learned that even the rural sector manager had not yet read the report! 2.20 Quality Enhancement Reviews. None of the reports reviewed had taken advantage of a QER. ARD could play a role in promoting QERs. Since the feedback from a QER is substantially different from that received from a peer review, QERs are very instructive and are likely to be especially useful in the case of an inexperienced task manager. They could be regarded as a “mentoring” tool. Undertaking a QER at concept or at least at the “white cover” stage would enhance the quality of AAA and provide managers greater confidence on the quality of the work. 2.21 Accessibility of Documents. The Bank appears to have a very porous institutional memory and an entirely inadequate filing system (electronic or otherwise). Many reports were missing or could only be located with difficulty. In one instance, a report was located because a panelist knew the author who had since retired, and obtained a copy from him; in another, a consultant had to be tracked down as the TTL did not have a copy; in a third case, only because a panelist knew the report existed was the report found – it was not recorded in the system. If staff are unaware of the existence of studies, how much less so are country counterparts? There is a massive institutional failure and a massive loss to the institution in reports lost, shelved, unused or under-used, or otherwise inaccessible to both staff and clients. Better ways to systematically save and access AAA products should be found. 2.22 Further, the lead-up documentation to AAA is frequently missing. Project concept notes are either non-existent or missing from files, as are the minutes of peer review meetings. This makes assessment by QAG and sector managers particularly difficult. Better managerial oversight of AAA work is needed. The Bank should consider the parallel with lending where a sector manager signs off on the PAD before it goes out. Sector managers could also sign off on AAA. Cross-Country ARD Sector AAA Assessment 13 BOX 2. 4: A SUPPLY-DRIVEN SUCCESS As demonstrated by recent events in Mexico, the transition to a new government offers a timely opportunity for the Bank to synthesis the p ortfolio of AAA work, whether or not a government requests it. Before the 2000 elections, the Mexico Country Department arranged for a stocktaking of issues drawing on existing knowledge of staff working on different sectors in Mexico. This consisted of a wide ranging series of policy notes, prepared outside the formal AAA program, largely as an unfunded mandate. As part of this process, comprehensive policy notes were prepared on major agricultural policy and institutional issues. These notes were based on the extensive AAA program that had been conducted in Mexico on agricultural and rural issues. When the election led to a change in Presidency for the first time in 70 years in Mexico, the Bank’s policy notes served as a briefing for the new Government. Later, the new Government agreed to have the policy notes published as a book, "Mexico-A Comprehensive Development Agenda for the New Era" (2001). The book is widely acknowledged with favorable citations, and represents a successful AAA contribution. It enabled the Bank to gain a "seat at the table" with the new Presidency. It also led to a major change in openness in Mexico's Presidency towards the Bank's analytical work. The book became an important element of the Bank's analytical contribution in Mexico although it is not recorded in the list of tasks undertaken by the Region. D. Priority Areas for Improvement and Next Steps 2.23 The following recommendations are addressed to specific Bank audiences that have a role to play in helping strengthen the impact of AAA work on reducing rural poverty: For the ARD Sector Board • Continue to publicize and disseminate best practice in AAA; and • Review measures that would help ensure the availability of experienced sector economists that can devote most if not all their time to AAA activities. For ARD Regional Sector Units • Sector managers need to be more proactive in ensur ing that sector staff follow Bank processes designed to enhance AAA quality, including proactive use of peer review systems; • Sector managers should earmark funds for dissemination of AAA findings and recommendations; • Sector staff should continue to champion the importance of AAA by emphasizing the preponderance of poverty in rural locations and the value of ARD AAA in meeting the Bank’s poverty alleviation objectives; • Greater use should be made of QERs, especially for less experienced task managers. This should be combined with more extensive use of external reviewers; Cross-Country ARD Sector AAA Assessment 14 • Make more extensive use of studies conducted outside the Bank and consider updating old studies before launching new ones covering the same topic; and • More attention is needed toward accurately recording AAA tasks and ensuring that reports are properly archived and readily available to the staff and the client. For Other Regional Sector Units • More attention is needed to ensuring that rural poverty aspects are properly integrated into the analytic work of other sectors, to overcome the tendency for staff to remain fixed within narrow sectoral “silos.” For Country Departments • Greater attention needs to be paid to the importance of the rural sector in country programs if country units are to fulfill the Bank mandate of poverty reduction. This will require a more consistent allocation of funds to ARD AAA to maintain the increase seen in FY04. For QAG • This pilot review was not able to fully address some issues due to the limited interaction with staff and managers. This aspect should be reviewed in any future assessment of this type; and • The impact of sector AAA on country lending programs and other stakeholders is probably best examined in conjunction with field visits which provide a much deeper basis for informed judgments. Cross-Country ARD Sector AAA Assessment 15 Annex 1 AGRICULTURE AND RURAL DEVELOPMENT PILOT R EVIEW O F THE Q UALITY AND EFFECTIVENESS OF ARD AAA Approach Paper Background 1. The Agriculture and Rural Development Sector Board has requested the Quality Assurance Group to conduct a pilot review of the impact of its SSP and ARD AAA activities 2 on: a) Country strategy and the formulation of sector strategies by the country and the Bank; and b) The integration of ARD AAA activities into Bank country operations, including policy dialogue and lending. 2. Table 1 below shows the volume of ARD AAA activities managed by the regions. What is less clear, however, is the impact ARD AAA has had on the Bank’s approach to the agricultural sector overall and to the formulation of strategies at country level. Nor is there a clear indication of how the Sector Strategy Paper influenced AAA and lending activities. There are two further uncertainties associated with the ARD AAA activities. The first relates to the correlation between the quality and quantity of these activities and the coverage of the rural issues in a country’s CAS and PRSP (according to assessments conducted by QAG during FY98-02, the 34 ESW activities in the rural sector received a 71% overall satisfactory rating compared with a Bank-wide overall rating of 88%). The second is the correlation between the quality of ARD AAA and the quality of project design and overall performance at exit. Table 1: Region specific ARD1 AAA Activities Managed by Regions: FY00-03 No of Activities Total Cost NLTA ESW 2 Other ESW 3 Total ($'000) FY00 5 22 6 33 5,282 FY01 2 16 15 33 7,634 FY02 12 9 12 33 5,578 FY03 16 13 18 47 7,268 Notes: Source Business Warehouse; 1. Activities designated as the responsibility of the Agriculture and Rural Sector Board; 2. refers to the reports output type; 3. ESW output types: Consultations; Consultations/Country dialogue; Conference/Workshop; Other; Policy Note. 2 ARD AAA stands for all AAA recorded as RDV, whether they are managed by ARD regional units or by other sector/country units. Cross-Country ARD Sector AAA Assessment 16 APPROACH 3. Included in the review will be all ARD AAA activities conducted during the five- year period FY01-04 and their likely impact on the rural strategy and lending. Activities will be reviewed in the context of a selected number of countries. Main objectives of the review will be to evaluate the following aspects: • Impact of the Strategy Papers (Vision to Action and Reaching the Rural Poor) on ARD AAA and lending activities; • Overall quality and adequacy of ARD AAA activities; • ARD AAA contribution to and impact on the CAS and PRSP, and understanding of the factors leading to the impact; • Impact of ARD AAA activities on the country’s rural strategy and on the activities of other donors; and • ARD AAA contribution to the formation of the Bank’s lending strategy in the sector and impact on the quality of project design and lending volume and understanding of the factors leading to the impact (AAA activities completed during FY 00-01 will be examined for actual impact on projects approved by the Board in FY03-04 and the rest for their potential to influence future lending). 4. The review will eva luate the impact of AAA activities conducted by both ARD regional units and those managed by other sector/country units (e.g., CEM, PER, Poverty Assessments), including regional and global ESW that may have relevance to the country’s rural sector strategy and lending. Particular attention will be paid to perceived shortfalls in the level of ARD AAA activities in the country and to the identification of areas where additional studies could enhance the quality of Bank activities and support the activities of the member country. In order to reduce the burden of this review on regional staff, preference was given in the selection of countries to countries where QAG has recently completed a Country AAA assessment. Selected countries, which are subject to review and approval by the concerned regions, include: Mexico and Ukraine Country AAA completed Kazakhstan Country AAA completed Bangladesh and Tanzania Country AAA ongoing Ghana No AAA assessment and relevant ARD AAA available Morocco No AAA assessment available 5. The review will be conducted by a core team comprising a senior retired Bank staff member and an outside reviewer. Specialized reviewers, with sector/project expertise, will be engaged as needed to examine in more detail individual activities. Support to the core panel will be provided by the ARD anchor. An initial desk review of all activities previously evaluated by QAG will be carried out with a view to identify those areas where additional work is called for. Where the Country AAA assessment is ongoing, the core panel will work closely with the members of those panels. Except where a Country AAA assessment is ongoing, the present review will only entail one meeting between the panel and the staff and managers responsible for the ARD work in the country. At this time no separate field visits Cross-Country ARD Sector AAA Assessment 17 are expected. A list of primary studies to be examined during the review in the seven selected countries is attached. 6. Questionnaires to be used in conducting the review will be modeled on the Country AAA assessment. They will evaluate quality and impact at the task, country and Bankwide levels. However, ratings will be strictly for QAG’s piloting purposes and will not be shared with the Regions or the Sector Board. QAG is currently preparing these questionnaires, and drafts will be sent to the Sector Board for review and comment. 7. At the end of the assignment, the core panel will produce individual country reports summarizing the findings and recommendations for the country. It will also produce an overall report outlining the major findings of the assessment. In a separate report to QAG, the panel will evaluate the review process, provide a judgment on whether or not this approach could be replicated to other sectors, and recommend additional measures to strengthen the process. COST ESTIMATES 8. Inputs to the review will be provided by the core panel, specialized reviewers, ARD support, and QAG administrative/management inp uts. Estimated costs associated with each of these aspects are as follows: a) Core panel: 2 persons x 25 days = 50 days b) Sector panelists: 15 in-depth reviews (1.0 day/activity), and 30 activities conducted by non-ARD units (0.5 day/activity) = 30 days. (In the case of the two countries where the review is ongoing, the costs will be covered by the AAA review). c) ARD input = 30 staff days d) QAG management and administration = 40 staff days The above works out to a total of 150 days or roughly $105,000. TIMING 9. It is suggested that the review should commence in mid-May so that the core panel could interact with the two on-going AAA reviews. The individual country assessments will be aggregated into an overall report by mid-July. PANELISTS Amnon Golan (Moderator) Malcolm Bale (Coordinator) John Haywood Rory O’Sullivan David Steeds Derek Byerlee Fred Swartzendruber Bruce Gardner Annex 2 Cross-Country ARD Sector AAA Assessment 18 LIST OF AAA REVIEWED FY Proj. ID Title TTL Output Cost Bangladesh FY01 P071573 Banglades h Rural Decentralization CONSTANTINO Consultations 58 FY01 P072118 Bangladesh - Agriculture Policy Dialogue CHOWDHURY Consultations 44 FY01 P067728 Natural Resources Management Note CHOWDHURY Policy Note 36 FY02 P075239 Rural Development Sector Dialogue EPWORTH NLTA 54 FY03 P074941 Ag. growth. and rural pov. dyn:A reg. p DIOP Policy Note 74 FY04 P078291 Promoting Rural Non-Farm Growth ESW FY04 P088653 Scaling up Microfinance ESW GHANA FY01 P068979 Poverty Update and Profile JONES CoConference FY03 P077543 Country Gender Assessment MUTEMBA Policy Note P078688 Poverty Note DEMERY Report FY04 P085043 Ghana Energy PSIA KEENER Policy Note P077582 CEM: Poverty, Growth, Budgetary CARNEIRO Report P083876 Public Expenditure Review ANDRADE Mexico FY01 P060321 RUR.FIN.IN MARG.AREA BRIZZI Report 200 FY01 P060324 FOOD MARKETING BRIZZI Report 346 FY01 P071457 MX-Land Policy Review LAVADENZ PACCIERI Policy Note 276 FY01 P072427 MX-Crop Insurance VARANGIS Report 165 FY02 P074770 MX Sugar LARSON Policy Note 119 FY02 P074768 Land Administration LAVADENZ PACCIERI NLTA 87 FY03 P074698 MX Coffee VARANGIS Report 122 FY04 P085943 Rural Poverty Analysis ESW Morocco FY03 P073174 MA-Agriculture Sector Note WARD Policy Note 175 FY03 P077456 FOREST AND LAND REVIEW PSWARAYI-RIDDIHOUGH Policy Note 87 FY03 P081087 MA: Agricultural Reform and Irrigation WARD Report 163 FY04 MA: Agricultural Reform 2 Report FY04 Agriculture Reform NLTA Tanzania FY02 P072131 RURAL DEVELOPMENT TOWNSEND Report 359 FY04 P085776 SEA for Sustainable Coastal Livelihoods BOYLE NLTA 20 FY04 P080072 Rural Development Report FY04 P087208 Agribusiness Supply Chain Analysis Ukraine FY01 P072194 LOCAL CAPACITY BLDG TA (PHASE 2) SHUKER NLTA 102 FY02 P074847 RUR FIN POL DIAL TA CHAVES NLTA 72 FY04 P077461 AGRIC POLICY NOTE SHUKER Policy Note 208 Kazakhstan FY03 P077376 Forestry and Rangeland Management FY03 P078300 Livestock Sector Study Jungbluth FY04 P083363 Fisheries Sector Study Sutton Annex 3 COUNTRY FINDINGS Cross-Country ARD Sector AAA Assessment 19 BANGLADESH S ECTOR AAA COUNTRY ASSESSMENT 1. Summary Comments. Serious deficiencies were found in the overall quality of Agriculture and Rural Development AAA in Bangladesh during the period 2001-2004. However, this finding has to be considered in the context of the special circumstances of country-Bank relations prevailing at the time, the state of the country’s agriculture, and the state of sector knowledge. From ARD’s perspective, the relationship between the Bank and Bangladesh was not considered conducive during FY01-03 to engaging more substantially in either new lending or AAA activities. Three out of the seven studies carried out during this period and identified by the Region as being relevant to the rural sector were found not to be AAA activities (they were either consultation with government or codes that covered staff time in maintaining a dialogue with their counterpart during this difficult period). The Region agrees that they have to address this coding problem. 2. During the 1980s and ‘90s, the Bank played a key role in Bangladesh agriculture in bringing about liberalization of markets and prices and privatization of input supply and irrigation. Employing high quality sector work and a large team of specialists in Dhaka, the Bank helped the Government of Bangladesh (GOB) attack constraints to agricultural growth and backed the effort with a vibrant pipeline of projects. Throughout the 1990s, economic growth accelerated and by 2000 Bangladesh had quite remarkably, achieved food-grain self- sufficiency. However, deteriorating governance soured government-Bank relations so that from 2000 to 2003 lending was low case, and funding for AAA was restricted mostly to PREM-managed tasks such as the Public Expenditure Review (PER) and its associated Poverty Assessment (PA). 3. A new CAS, produced in 2001, conformed well to the Rural Sector Strategy, and set “an integrated approach to rural development, including supporting growth in agriculture and non- farm activities” as “one of the three most powerful lines of attack against poverty.” Notwithstanding, this theoretical thrust, however, only minimal resources were provided to support agriculture. There was good support for rural infrastructure, and rural health and education, which stimulated the private sector but did nothing to sustain earlier gains in agriculture. Importantly, there was no integrated approach to rural development and poverty reduction that included agriculture. Consequently, the portfolio dwindled; sector work fragmented, and demoralized agricultural staff left the Region. Given the hiatus in 2000-03, action is even more urgent now. Another noteworthy factor is the Bank’s previous instrumental role in helping develop a good donor coordina tion program in Bangladesh. In the future, this facility could be used to interact with government, perhaps through a dedicated workshop, to define and prioritize issues constraining rural advancement and proposing a coherent program of AAA tasks. 4. A new team of Bank country and sector management took over responsibility for the Bangladesh program in 2003 and started to inject life back into the agricultural program. It should be recognized, therefore that this assessment reviews AAA, at a particularly difficult time in Bank-GOB relations, when the sector’s impressive performance had induced complacency in the country and in the Bank, and when Bank management was in transition. These factors combined to produce a period of AAA weakness that must be considered an aberration in the history of Bank- Bangladesh ARD relations. 5. Strategic Relevance. Studies with relevance to the ARD sector (PER, Poverty Assessment and Climate Change) were focused on CAS objectives but the rest of the program was Cross-Country ARD Sector AAA Assessment 20 supply-driven and descriptive of the current situation rather than being forward looking and aimed at helping government overcome emerging problems. Importantly, the totality of the AAA program failed to address integrated rural development and rural growth in a coherent manner. The CAS clearly articulated a prime objective of reducing rural poverty but this was not matched by coherent, prioritized AAA work geared to producing practical answers to rural problems. This lack of relevance to immediate issues was cited as a key reason for poor relations between Bank’s sector staff and the line ministry. 6. The Poverty Assessment fed recommendations directly and purposefully into the I- PRSP and the Climate Change task produced valuable indicators for a long-term vision of development. However, AAA failed to address priority needs and the rural sector still lacks answers to vital questions concerning crop diversification, agro-processing and marketing, stimulating the non- farm sector, how best to introduce new technology, improving land and labor productivity, efficient water management, and how best to provide credit for middle level farmers and entrepreneurs. It should be noted, however, that new studies launched in FY04 will address many of these issues but as they were still in progress when the current review was completed, they could not be assessed by the Panel. 7. Internal Quality. Agricultural data in Bangladesh are accessible, sound and up to date. They provide a wealth of information for high quality analysis, as evidenced by the Agriculture Growth and Rural Poverty Dynamics study. The quality of analysis in AAA was mostly satisfactory. A number of studies used the Bangladesh Bureau of Statistics (BBS) and international experts, for example IFPRI economists for the Agriculture Growth Study, to undertake analyses. Small studies, although technically sound, tended to be descriptive of the current situation and to bring no new analysis to the agricultural debate. 8. Recommendations followed logically from analyses and conclusions, but were mostly broad and general. A complaint from the MOA was that recommendations lacked specificity and practicality regarding what should be done and when. Sequencing of recommendations was rare. Most AAA reports avoid being prescriptive, deliberately leaving decisions to government. Although this is correct, AAA should nevertheless aim to produce options for debate, otherwise recommendations become too vague; which leaves the potential role for project intervention equally vague. 9. A key factor influencing quality was management commitment to, and support of, the task. Where the task from its inception was considered important by the Bank and by stakeholders, e.g., the PER, adequate funding for employing a skilled team and for getting good feedback resulted in high quality product. 10. Likely Impact. AAA tasks varied greatly in their likely impact on clients. Importantly, it was often unclear just who was the client for a particular AAA task. Some tasks were directed at government decision makers and civil society, whereas others appeared undecided between the government and Bank management. This ambiguity of client focus resulted in ineffective presentations and weak impact. By contrast, AAA tasks that involved PREM interaction with the Planning Commission and Ministry of Finance had good impact at senior government levels. Where, however, AAA was supply-driven and had little support from Bank management or champions within government, it tended to fizzle out and have zero impact. Capacity building efforts were effective in some AAA activities, for example where BBS was involved, but sometimes appear to have taken place in virtual isolation from government staff. Cross-Country ARD Sector AAA Assessment 21 11. Government ownership, dialogue and dissemination were particularly weak elements in much of the AAA program. It is vital to the health of the AAA program that in future, government should be true partners in prioritizing and designing AAA. Specific resources should be made available for effective interaction with stakeholder. The impact of AAA on the agricultural lending program was minimal. There was a complete disconnect between completed studies and the ongoing ARD portfolio during the period under review. AAA tasks did not tackle emerging problems so both the Ministry of Agriculture and Bank management lost confidence and interest in the program. 12. Donors look to the Bank for leadership in agricultural strategy and use quality AAA findings as indicators in their own programs. Impact of the PER and Poverty Assessment studies on donors was considerable. However, small pieces of supply-driven AAA had little or no visible influence on donors’ programs. 13. Comprehensiveness. In the Panel’s view, the AAA program was fragmented, supply-driven and ad hoc during this difficult period when the Bank virtually withdrew from the sector. The sum total of studies, which omitted some key issues (see paragraph 6), in no way covered the needs of a forward- looking ARD program. The government and the Bank’s agricultural staff, particularly those few remaining in Dhaka, recognized the need to sustain sector gains but lack of adequate funding and lack of a critical mass of skilled staff prevented meaningful AAA being carried out. 14. Planning and Bank Processes. During the period 2000-2003 planning for the overall AAA program was weak. There was no coherence in the program and it was not geared to the strategic needs of the country or to the development of a meaningful lending program. Planning of the individual larger pieces of AAA – the PER, Poverty Assessment and Climate Study – was good. At the outset, plans for these tasks defined the client and provided resources for stakeholder interaction and results dissemination. Other pieces, however, were supply-driven and suffered accordingly. 15. The Poverty Assessment demonstrated what could be achieved even during a period of poor Bank-government relations. It conformed precisely to the CAS, was geared to the - immediate needs of the I PRSP, was well planned and resourced, and was given good support by Country and Sector Management. Consequently, it was taken seriously by government and stakeholders, which made feedback and ownership meaningful. The impact of this AAA was considerable. 16. The Climate Change Study, which could have easily turned out to be somewhat academic, was also well planned and resourced, given support and taken seriously by management, government and stakeholders. Again, its impact on changing attitudes in long term planning of government and donors could be considerable. The contrast between how these AAA tasks were managed, compared with the rest of the program, provides clear lessons about ‘seriousness’ and impact of AAA. 17. The vicious downward spiral that set in during 2000-2003, whereby lack of support for agriculture led to lack of staff capacity, which led to lack of sector strategy and further lack of support for the sector, must be reversed. The impressive gains in agriculture made during the ‘80s and ‘90s can only be sustained with donor support for further modernizing the sector. But building an effective pipeline will require coherent and coordinated AAA, undertaken by skilled professionals. This will, of course, require adequate funding, but more Cross-Country ARD Sector AAA Assessment 22 importantly it will require experienced staff backed by consistent management attention and support. Cross-Country ARD Sector AAA Assessment 23 GHANA S ECTOR AAA COUNTRY ASSESSMENT 18. Summary Comments. In the absence of ARD-specific AAA work, this evaluation covers relevant aspects of other AAA tasks suggested by the Region. Some of the poverty work done in Ghana was found to be of high quality, and there is now a significant body of research dating to the early 1990s, which demonstrates that poverty reduction has been rather successful in urban areas, notably in Accra, while deprivation continues or is increasing in rural areas with minimal infrastructure, poor access to markets and services, and few economic options. The recent CEM does an excellent job of cross-sectoral synthesis of this evidence and convincingly presents the case for better sector work in Ghana, especially with respect to agriculture. Another factor may also be at work in Ghana, relating to the very high level of effort in recent years dedicated to developing the CDF framework, preparing the first PRSP and PRSC, and complex multi-donor negotiations around these initiatives. This work may have crowded-out some of the sector work and management attention which otherwise might have been channeled toward strengthening the existing rural/agricultural portfolio, and strengthening the integration, coherence and dissemination of AAA activities, especially in cross-cutting areas such as poverty and social impact analysis, which are highly relevant to the ARD agenda in Ghana but are managed under other sectors. 19. Strategic vision and leadership are especially important to provide renewed focus and dynamism when a sector program runs into serious difficulties, as has been true of agriculture in Ghana during the 1990s. If the institutional judgment is that this is not an area of comparative advantage for IDA (a legitimate option), then in the interest of transparency it would be better to acknowledge this openly, and work with the Government and other donors to ensure that appropriate resources and commitment are forthcoming. 20. Strategic Relevance. The Panel noted the absence of ARD managed sector work during the review period, and the fact that none is proposed in the latest CAS (dated February 2004). In fact, the rural development staff appears to have taken a back seat on critical issues, which are at times being covered (thinly) by other units, who have considerably less technical expertise (e.g., aspects concerning non-traditional agricultural exports). At the same time, according to the Bank's poverty studies, poverty in Ghana is essentially rural (more than 80% of the poor live in rural areas), is increasingly concentrated among farming households in northern savanna areas and coastal non-forest areas, and is especially severe among female- headed subsistence farming households. The Bank and the government both agree tha t the agricultural sector is critical to achieving sustainable, broad-based growth, especially in diversifying exports and developing new products and markets for which Ghana has comparative advantage. The absence of an ARD AAA program thus represents a serious strategic omission in Ghana, as it is hard to see how poverty reduction objectives can be achieved without a major effort to address these problems. As a group, the AAA tasks reviewed here do not yet add up to a comprehensive picture of what the Bank would need to do on the analytic side to increase the chances of achieving the MDGs and to improve the present lending program. The recent Country Economic Memorandum (February 2004) makes a useful first step in this direction, by calling for more in-depth sector work "to identify the bottlenecks to agricultural growth,” and notes that "policy dialogue must advance beyond cocoa... in essence moving from subsistence agriculture to market-based high value production.” 21. Internal Quality. Tasks examined by the Panel demonstrate a strong continuity of analytic effort on poverty issues since the mid-1990s, and, taken together, would appear to Cross-Country ARD Sector AAA Assessment 24 support a strong case for rethinking the Bank's approach to rural development in Ghana. Some of the individual reports w ould be strengthened by sharpening their conclusions and recommendations, which are sometimes sidelined within a text box or table without a clear summing- up of key points and next steps. The electricity tariff social impact assessment has excellent internal quality, but the present phase I report is only partially relevant to the rural sector, and the proposed phase II has not been committed, though it would be directly focused on poor rural areas of northern Ghana. But on the whole, these non-ARD pieces maintain important Bank engagement with analytic issues that will need much more attention in the future, and represent a good starting point for reinvestment in the sector. 22. Likely Impact. Based on the present absence of an ARD AAA program, which could make use of the non-ARD tasks reviewed here, the program is unlikely to have a significant impact. The Bank's rural projects in Ghana have performed poorly in recent years, and many of these deficiencies are exacerbated by the lack of relevant sector work to inform project design and improve the choice of targets, instruments, and partners. For example, peer reviewers criticized a CDD operation now in preparation as taking a very outdated approach, and missing opportunities to build upon the Bank's own best practices. The task team argues that land tenure and legal issues unique to Ghana seriously constrained design choices, yet this is precisely the kind of situation which would call for focused and timely AAA work prior to project preparation. Similar design issues are found in the new land administration project, which is designed to tackle a complex problem that would greatly benefit from up- front analytic work. Indeed, the 2004 CEM, which was reviewed for this exercise, comments that land administratio n efforts are likely to fail in the absence of a range of upstream improvements in agricultural input supply as well as downstream improvements in marketing and infrastructure. 23. Comprehensiveness. Shortcomings on this aspect are similar to those identified above. The most comprehensive sector work in Ghana by far is poverty work done in the overall context of the GPRS, but the non-ARD tasks reviewed here in no way substitute for a properly- focused effort to understand why previous rural and agricultural operations have performed poorly, and to identify alternative approaches which might offer better chances of achieving the MDGs. 24. Planning and Bank Processes. The AAA tasks reviewed show little evidence of attention from the ARD sector managers and limited coordination of cross-sectoral issues. At least one study appears to have been closed at a draft stage without soliciting comments from the client or planning any dissemination activities. During the review period, the country team appears to have been extremely busy in support of the PRSP/PRSC and CDF initiatives, but these have not been guided toward developing a coherent ARD AAA program, which in turn could serve as a basis for improving the troubled rural portfolio in Ghana. Records are often difficult to locate, and in some cases, there remains little institutional memory about how a particular task evolved and why. Cross-Country ARD Sector AAA Assessment 25 KAZAKHSTAN S ECTOR AAA COUNTRY ASSESSMENT 25. Summary Comments. Since 2001, and the abrupt change in Bank-Kazakh relations following Kazakhstan's oil boom, sector work in Kazakhstan has been carried out jointly with the government under the Joint Economic Research Program (JERP). This involves the selection of an agreed sector work program and the joint financing of tasks. Several aspects of this arrangement are noteworthy. First, there are few member countries where governments agree to co-finance (50% in this case) sector work, and this arrangement, which has been brokered with Kazakhstan, is a most interesting development with possible lessons for other MIC’s. Second, because studies are 100% demand driven, the likelihood that senior officials will actually read and use them is far greater than in most other countries. But third, on the downside, because of the demand driven nature of the arrangement, the Bank seems to have limited influence over the selection of studies entering the work program. A pertinent question in this case is the extent to which sector work in this kind of arrangement should also be driven by Bank policy preference (such as the poverty driven imperative of the ARD strategy). The three tasks reviewed here, covering Fisheries, Livestock, and Forestry/Rangelands were part of this jointly implemented and agreed program. These tasks involved the participation of local cons ultants and experts working with Bank staff and consultants. The reports were clearly of good technical quality, with the Bank evidently providing effective leadership in collecting and harmonizing disparate contributions and weaving them into a coherent framework while addressing the key policy issues using cross country experience. But the linkage of these studies to the CAS and overall Bank policy needs to be examined. 26. Although the program has the clear advantage of being demand driven, the Panel was concerned that its overall thrust may not cover the highest sectoral priorities for stimulating economic development in the rural areas and combating poverty and queried the flexibility that the Bank had for the allocation of WB funds in its dealings with the JERP. 27. Strategic Relevance. While the three tasks all contribute to specific areas of concern in the rural space they differ in their potential impact, and the Panel considers that other tasks may have been more relevant in addressing sector -wide and poverty-related issues. The livestock report covers a highly relevant and important subsector, which contributes as much as 40% of agriculture GDP and contains a considerable proportion of the rural poor. On the other hand, the fisheries report, while addressing important technical challenges in the sector (preserving the Caspian sturgeon, an effort that calls for close collaboration with the other countries bordering the sea, and the co-management of small lakes), deals with a topic that may not be of the highest priority in Kazakhstan (officially, only 13,200 people are engaged in commercial fishing). While the report recognizes that artisanal fishermen are also important, and tries to investigate them, it does not cover this group in any depth or as a major focus, as the ARD strategy would recommend, and fails to address the concerns of persons in the poverty target group. But at the same time the report provides and excellent technical assessment of the sector, which was surely appreciated by the JERP administrators and those working in the sector. The Forestry/ Rangelands study was in the end split into two. Rangelands, of very high relevance and importance on economic, poverty and environmental grounds, and forestry that in Kazakhstan is important more on environmental grounds (the Panel noted, however, that partly as a result of the latter study, a Forestry Project is now at an advanced state of preparation). Activities dealing directly with agriculture and agricultural products were missing from this AAA program: agricultural production, agricultural research, agricultural extension, marketing, and seed production. While some of these will have been handled in earlier studies, and others were Cross-Country ARD Sector AAA Assessment 26 handled in the context of project preparation, there are still key unresolved issues in Kazakhstan in these subsectors of wide importance and the Panel wondered if this program could not have been better targeted. Perhaps this will raise challenges for the future work program to be agreed under the JERP Program. Elsewhere, it was noted that regional studies covering microfinance and irrigation had contributed to a better understanding of key ARD related issues. Both these studies were useful additions to Kazakhstan's AAA program but are not a substitute to tasks concentrating on country specific issues. 28. Internal Quality. Overall, the technical quality of the reports is good and they seem to address the key issues in the subsectors in which they are working. Technical reports written by local consultants have been used judiciously to provide a solid technical underpinning to the policy recommendations made. External comparisons have been well selected to put the sectors in the relevant perspective. 29. Likely Impact. The Panel considered that information contained in these reports could have a significant impact on the chosen subsectors provided the Kazakh authorities follow up the recommendations made. The problem is that the extent to which this will actually happen is still unknown. The outcome will depend to some extent on the ability of the Bank and local experts to "sell" the findings of the reports and it is still too early to know the outcome in this regard. One potential important factor that will help a successful outcome is the emphasis in all the reports on improvements that can be made in the Public Investment Program for the subsector being considered. This linkage with the key national instrument for making things happen could facilitate a positive outcome. Overall, it would it would be advantageous for the national counterparts in the JERP Program to be fully involved in dissemination as well as design of this AAA program. 30. Comprehensiveness. The Panel had concerns about the comprehensiveness of the AAA program in the ARD sector, even allowing for separate regional studies that have included aspects of the rural sector. For Kazakhstan, this then raised the question of the influence that the Bank had in setting the AAA agenda in the context of the JERP. As mentioned above, the current program does not cover key subsectors for economic growth and poverty reduction in the rural areas, which puts into question the adequacy of the Bank's AAA program in the country. 31. Planning and Bank Processes. A number of the issues raised above, including the lack of attention to certain key subsectors, highlight the difficulty of planning the AAA program in the JERP environment. The government may be less interested in working on key issues (e.g., with much of the land being tied up in large holdings, what can be done to improve the livelihood of small farmers), which face difficult policy choices. Since AAA activities in Kazakhstan receive about one third of all BB resources, it is important for the final program to mirror Bank priorities as much as it responds to the needs of the Government. The Panel is aware that this would be difficult in the Kazakhstan context but every effort should be made to make this happen. 32. While pointing out potential risks (see above), the panel commended the close working relationship with the client in designing and implementing the AAA program through the JERP. It is expected that this joint working arrangement would bring benefits in subsequent dissemination efforts, the transfer of knowledge, and the technical quality of the final product. All the reports appeared strong on technical aspects with excellent policy roadmaps for the Kazakhstan’s decision makers. There was widespread evidence in supporting documentation of good coordination with other development partners, which should strengthen the CDF approach of other partners. Cross-Country ARD Sector AAA Assessment 27 33. The cost effectiveness of AAA needs to be assessed when designing the program. For the ARD sector, this assessment will often be in terms of resources allocated to tasks compared with the number of potential beneficiaries, particularly the poor. This will help keep the AAA program in line with the Bank's ARD strategy. Dissemination remains a vital task if AAA is to have the full impact hoped for. This may well require a multi-year dissemination pr ogram, coming back yearly with an assessment of progress and policy change adoption. This could greatly enhance the effectiveness and payoff from AAA. Cross-Country ARD Sector AAA Assessment 28 M EXICO S ECTOR AAA COUNTRY ASSESSMENT 34. Summary Comments. Mexico is an advanced middle- income country with considerable capacity for analytical work. However, the Bank is valued for its AAA, especially in work that is on the frontier in terms of analytical skills and that provides access to international good practice. In addition, lending to Mexico is non-additional in ministry budgets, so that AAA work is demanded and valued largely as a stand-alone product, rather than as a means for enhancing the quality of lending. 35. ARD AAA work in Mexico emerged as part of the post- liberalization agenda, after implementation of NAFTA. It has been one of the largest ARD AAA programs, and with its emphasis on reforms of product and factor markets, was clearly strategic in terms of CAS and Mexican goals. Overall, the program of 7 AAA tasks delivered during FY01-04 has had considerable impacts on policy and capacity building, and to some extent on the lending program. However, emphasis on AAA has fallen off sharply beginning in FY03, in part due to a decline in overall AAA, and resulted in a significant reduction in the ARD share of AAA from 24% in FY00-FY02 to 7% in FY03-FY04. The current program is not well positioned to engage in policy dialogue on major emerging issues such as competitiveness, more effective use of the large public expenditures on the sector, and adjustments required for full implementation of NAFTA by 2008. 36. Strategic Relevance. The AAA program emphasizing product markets (grains, perishables), commodity studies (coffee, sugar), and factor markets (rural finance, insurance, and land) clearly responded to the post- liberalization agenda to develop efficient and competitive markets, in the wake of state withdrawal from the sector. Agriculture and rural development was highlighted as a strategic priority in the 1999 CAS, which guided the selection of tasks to FY02. Likewise, the task on rural poverty initiated in FY03 (but not yet delivered), was consistent both with the CAS priorities, as well as the newly-approved Bank corporate strategy, Reaching the Rural Poor. Some of the tasks, notably sugar, crop insurance, and land policy, also responded to strong demands from the GOM. 37. Internal Quality. QAG has previously evaluated five of the seven completed tasks, and generally rated them favorably, with the tasks on land policy rated as highly satisfactory. Some of the tasks were weak in recommendations, providing too many general recommendations that did not provide sufficient guidance on priorities, the role of the public sector, and institutional options for effective implementation. 38. Likely Impact. The AAA program has had considerable impacts. Three of the individual tasks have been rated highly satisfactory in terms of policy changes and capacity building (land, sugar, and crop insurance). In spite of the non-additionality of lending, the task on rural finance has been used in the design of two operations, and the task on land is also likely to be followed by a lending operation. Several of the tasks (insurance, land, and coffee), have also resulted in changes in overall government strategies toward the sector, and the work on coffee, food markets, and crop insurance has influenced private sector decisions. Overall, however, impacts may have been even higher with a more effective and longer run strategy on dissemination. Only one of the reports was formally published, and none are available in short policy notes (in Spanish) or on the web. 39. Comprehensiveness. The AAA program was quite comprehensive through FY02. This is reflected in the compilation of the book for the incoming government at the end of Cross-Country ARD Sector AAA Assessment 29 2000, which provides an excellent overview of the major challenges for agricultural and rural development, and concrete recommendations for moving forward. Relative to the overall AAA program for Mexico, a much higher rate of rural AAA listed in the CASs was comp leted in this period. Rural staff have also made significant contributions to AAA led by other sectors, especially the Southern States Strategy. 40. While the program has shifted directions since FY02 toward other key priorities, notably rural poverty and water, the panel feels that the Bank has lost momentum in some of the core issues for the sector, relating to effective use of public expenditures, institutional capacity to address competitiveness within a value chain framework, and adjustment to full implementation of NAFTA. 41. Planning and the Bank Processes. Although the overall AAA program in Mexico was rated by QAG as lacking coherence and strategic direction, especially on macro- level issues, this was less of a problem at the sectoral level. A judicious use of demand and supply driven approaches were applied, that resulted in a reasonably coherent program that at the same time responded to client requests. More recently, some of that coherence has been lost, as the Bank appears to be reluctant to push for AAA on issues of clear strategic relevance for the sector, such as competitiveness, which has appeared in the last three CASs but never implemented. 42. Major findings from the current review are as follows: Strong Aspects • Effective deployment of high quality Bank staff, often from the anchor, DEC or universities, to carry out complex tasks (such as crop insurance) that are of good technical quality; • Generally good dialogue with Mexican officials in several ministries, with good continuity over time to FY02; • Tasks are having a useful impact, with tangible evidence of results on the ground. Aspects Needing Attention • Dialogue needs to be sustained over time on core issues, and with a variety of ministries and organizations, even with changes in personnel in both the Bank and in the GOM; • Increased attention to the share of agriculture and rural development in AAA (since FY02); and • Allocation of resources up front to dissemination, to allow final publication and translation of policy notes that could be made ava ilable to a wider group of stakeholders, and to maintain institutional memory. Cross-Country ARD Sector AAA Assessment 30 Systemic Lessons • AAA should be regarded as a stand-alone product that can have significant impacts, regardless of the amount and quality of subsequent lending; • Preparation of policy notes for incoming governments is an effective tool for synthesizing AAA work and providing immediate input into policy dialogue; • The formulation of AAA programs requires both demand and supply led approaches to setting priorities and developing a coherent program; and • The Bank should systematically tap experiences and good practice in institutional and policy reform in OECD countries, to inform policy dialogue in advanced middle- income countries such as Mexico. Cross-Country ARD Sector AAA Assessment 31 M OROCCO: SECTOR AAA COUNTRY ASSESSMENT 43. Summary Comments. The Panel assessed three tasks completed during the review period. These tasks have slightly confusing names and will be referred to here as policy note, cereals, and irrigation. A fourth task produced a document entitled Integrated Forestry Development in the Middle East and North Africa. This is a regional task, with a particular emphasis on Morocco (and Tunisia) only in an annex with good material on land ownership, land fragmentation, and inactive land markets but very little on the relevance of land issues to forestry development. Although coded and funded as part of the Morocco AAA program, the country to which the report’s forest recommendations are least relevant is Morocco, since Morocco already has a good forestry strategy and a Bank-financed project is under preparation. Thus, it was an anomaly that this work was included in the Morocco AAA program. 44. A major recommendation of the review is that so long as government takes no action on large-scale irrigation or on the more difficult aspects of cereals policy reform, the Bank should consider deferring any new analytical work and concentrate instead on trying to get action taken on the work already done, even if this requires commissioning further work in these areas. Among the key findings of the review are the following: • The rural dimension has to be brought more explicitly into design of the AAA program for ARD. Perhaps this is already being done informally, but it needs to be made explicit; • A multi- year programmatic approach to sector work, while still requiring confirmation in annual work programs, provides important comfort to sector managers and staff that they can embark on a program of work with the client knowing that the bank will stay the course; • A two-part presentation of the written product in a client- led process, as done for both cereals and irrigation, enables the Bank to meet the challenge of preserving its objectivity. Part one of the presentation summarizes the results of the work undertaken jointly, while part two offers a (constructive) critique of the work; • A well- timed, custom- made, policy note can be very effective in engaging a new government in consideration of policy reforms in contentious areas; • In cases where collaborative work with other donors might detract from perceived objectivity, such as Moroccan officials’ skepticism about the motives of USAID and EU regarding reduction of protection of cereals, the Bank should do the work alone (with the client); and • On processes, the peer review process for the three activities reviewed by the Panel was very mixed, and the quality of information in the SAP and BW was unsatisfactory. Cross-Country ARD Sector AAA Assessment 32 45. Strategic Relevance. A major rural development strategy had been undertaken in 1998, which was judged (and is still judged) to be highly relevant. Building on this strategy, the July 2001 CAS envisaged completion of work on agricultural incentives in 2001, undertaking work on agriculture/water strategy in 2002, and “expanded” ICRs on large-scale irrigation in 2001 and rural water supply in 2003. The scope of this analytical work was fully appropriate. 46. Given the CASs emphasis on focusing on the rural poverty challenge, and a history of zero rural growth in the 90s, the AAA program was rightly focused on changing incent ives to promote growth, in a manner that would take account of poverty implications. This was the thrust of the policy note that was put to the new government at the end of 2002. At Government’s request, the Bank agreed to conduct joint analytical work on cereals trade and price policies and large-scale irrigation management first, to be followed by research and extension, with institutional restructuring to come later. None of this work was undertaken for the purpose of underpinning lending, although lending could well arise in respect of, for instance, direct income support during the cereals policy transition. Undertaken as a fully collaborative exercise with two government-working groups, the program was designed to maximize knowledge sharing. Overall, relevance was highly satisfactory. 47. Internal Quality. Without breaking any new ground, the three written outputs contain good quality analysis. The policy note produced for the new government was a model of the genre, synthesizing a vast amount of material in 15 pages, with a clear statement of recommendations, and indications of how the Bank would be willing to help. Both the cereals and irrigation papers are very good short statements of complex issues, drawing on worldwide experience in developing policy options, and not shrinking from indicating differences between government and Bank positions. 48. No surveys were launched to look for new evidence, but appropriate consultancies were arranged to fill gaps in, for instance, legal constraints on irrigation management reform, and modeling the effects of cereal pricing reforms on production systems. Related work under the Poverty Study (completed in April 2004) analyzed the distributional effects of cereals reforms, and made it possible to pinpoint the income effects as a clear guide to refining needed accompanying measures. Each of the three main pieces of work, and the work on forestry too, contain good discussions of options followed by clear statements of conclusions and recommendation. These stateme nts are mutually consistent in striving for pro-poor growth. Overall, quality was fully satisfactory. 49. Likely Impact. Impact on the client is somewhat mixed. Impact in terms of building ownership and coalitions for change has already been achieved, via the composition of the working groups and the fully participatory methods that they adopted. Bank participation was low-key, and rightly so. Some changes in government policy have recently been announced in respect of aspects of cereals marketing reform, including some pilot activities, but nothing has yet been decided for large-scale irrigation. In respect of cereals, moreover, the more difficult areas of trade and consumer subsidies are still pending. Liberalization of the agricultural price system, which could have been chosen as a success indicator, has still not yet been fully attained. By contrast, impacts on capacity building and knowledge transfer have already been attained. Cross-Country ARD Sector AAA Assessment 33 50. The impact on the Bank is likely to be high. The programmatic approach to AAA has clearly been effective, and is likely to be adopted by other sectors in Morocco (if not already done) and in other countries within the ARD sector. The country knowledge base of the Bank and its staff has greatly benefited from operating with client- led working groups. 51. Comprehensiveness. Reference has already been made to the sequencing of analytical work, starting with cereals and large-scale irrigation, followed by research and extension and institutional restructuring. Building on the 1998 strategy, the CAS laid out a fairly modest lending program ($100 million and 2-3 projects per year) but with about one- third for ARD. These included three successive operations for community-based development in projects in small-scale irrigation, rainfed areas, and forestry and natural resources management. Each of these projects has, in fact, come to fruition. This suggests that no further analytical work, beyond project preparation, was needed in these subject- matter areas and that the selectivity applied to design of the AAA program was highly appropriate. 52. These remarks apply to the agricultural aspects of ARD but nothing has been said about the non-agricultural aspects. The CAS had envisaged an expanded ICR on rural water supply, and a rural infrastructure project in FY03. The panel has seen no documents on these or related matters and is therefore concerned about the comprehensiveness of the whole AAA program for ARD. 53. Planning and Bank Processes. The multi- year programmatic approach to design of sectoral AAA seems to be working very effectively. Noteworthy aspects of the program in Morocco include: • Appropriateness of coverage and scope of the sector AAA program; • Relevance of the program to the CAS and the SSP, including water and forest sector policies; • The contribution of the program to promoting policy dialogue; • Quality of client participation in defining the scope of the work, carrying it out, and formulating conclusions and recommendations; and • The coherence of the program over time and the consistency of effort. Cross-Country ARD Sector AAA Assessment 34 TANZANIA S ECTOR AAA COUNTRY ASSESSMENT 54. Summary Comments. This assessment is based on three AAA tasks. One, entitled Rural Development Sector Work (June 2002), was really two pieces of work: a review of institutional arrangements fo r promoting cross-sectoral rural development; and an examination of (relatively poor) agricultural export performance. This is a high quality product. The other two tasks are: a poverty and social impact analysis of crop boards, as a follow-up to the work on agricultural exports; and an analysis of agribusiness supply chains. Both these tasks are incomplete. For the former, for which the first draft report is expected shortly, scope and strategic relevance are highly satisfactory but it is too early to m ake assessments of any other aspects, although they are also likely to of high quality. For the latter, for which a first draft report recently became available, the Panel identified a number of deficiencies, though these may well be addressed during the Regional review of the draft report. 55. While the AAA program was not undertaken to underpin lending, it is surprising that the actual (in contrast to CAS-planned) lending program included a Participatory Agricultural Development and Empowerment project that seems to have no tie to the CAS or to any AAA. New lending in research and extension is being contemplated and it is surprising that prior sector work has not been initiated instead of, for instance, work on supply chains. As for the non-agricultural aspects of ARD, the CAS envisaged nothing beyond (indirectly) fiscal decentralization. The panel has seen no documents on any other related matters and is concerned about the comprehensiveness of the whole AAA program for ARD. 56. Key findings from the Panel’s review of the Tanzania program include: • Overall, the program was strongly linked to initiating/moving the dialogue forward. Dialogue with Government on exports seems to have been stalled when reforms were rolled back in 1999-00. The Bank’s AAA program was intended to restart it, and subsequently to deepen it though poverty and social impact analysis. The work on institutional arrangements for rural development was also intended to move the dialogue forward; • The work on exports (both tasks) and supply chains was cutting edge, and undertaken to share knowledge and build capacity; • Broadening of the agenda from agriculture to rural development was the right move (and in line with the Bank's sector strategy); • The work on crop export boards is innovative. It addresses the contentious and often neglected area of second-generation reform of marketing institutions. If successful, it will become a best-practice case to be scaled up in those countries that are wary of full liberalization but unsure of what institutional and regulatory arrangements to keep in place; • An aspect common to work on exports and supply chains was shortcomings in the quality/use of empirical evidence. Egregious errors need to be cleaned up in the supply chains report; Cross-Country ARD Sector AAA Assessment 35 • Another aspect common to exports and supply chains was an inadequate conclusions and recommendations section. For exports, the panel would have welcomed greater differentiation in conclusions and recommendations between the crop boards. For supply chains, it is important that a clear statement of conclusions and recommendations be made in the text and succinctly put into the summary; • Likely impact would be enhanced if conclusions and recommendations were clearly stated; and • The rural dimension has to be brought more explicitly into design of the AAA program for ARD. Perhaps this is already being done informally, but it needs to be made explicit. 57. Strategic Relevance. The CAS had envisaged only one piece of AAA in ARD: a rural development strategy in 2002 to follow up on two major pieces of agricultural sector work that had been completed before mid 2000. This broadening of emphasis from agriculture to rural development was sound. The budget of $175,000 implied spending of only $60,000/year, surely too little for a country the size of Tanzania and for a sector of such importance for poverty reduction. In the event, actual spending of $190,000/year on four tasks was a far more appropriate (about $150,000 of this total was funded by BB). Much of the actual program (two tasks and about 70% of spending) was on agricultural exports, hardly a top priority for poverty reduction, but perhaps the earlier work had dealt adequately with commodities for domestic markets. 58. Among items not included in the program, some analysis of the scope for growth would have been warranted. With the Ministry of Finance unwilling to invest in rural development so long as agriculture under performed by not achieving growth of 7%/year, and with the CMU having reservations about rural development, a short analytical piece could have been very useful showing what growth could reasonably by expected and what policies and public expenditures would be needed to achieve it. This could have drawn heavily on the two major pieces of work that had been completed mid-2000. 59. The rationale for undertaking work on supply chains is not clear, in large measure because no concept paper could be found, but also because the draft report does not make the case convincingly. Although small cost ($40,000), an alternative could have been to take a fresh look, via an expanded analytical ICR or as a self-standing ESW piece, at the disappointing performance of research and extension. 60. Internal Quality. The work on institutional arrangements for rural development was not analytically demanding but was done competently. The work on agricultural exports was in many ways of high quality, but could have been improved by (e.g., examining strengths and weaknesses of cooperatives and their relations with crop boards, and providing illustratio ns of good marketing practices from other countries). In addition, the report points out significant differences in export crop performance in Ethiopia, Kenya, Tanzania, and Uganda, but does not convincingly explain them. The work on supply chains contains little real analysis of performance of the chains with respect to producer and consumer welfare, and how the chains might be made more effective. Limited, and poor use of, empirical evidence is a problem in the work on supply chains. Absence of clear conclusions and recommendations is also a problem, which, if not corrected, would prevent any likely impact. Cross-Country ARD Sector AAA Assessment 36 In export crops, however, conclusions and recommendations were not candidly stated for tactical reasons, but should nonetheless have been better differentiated between the boards. 61. Likely Impact. There are many vested interests in preserving the status quo regarding crop boards, and many vested bureaucratic interests in preserving the status quo regarding "turf" in rural development. These two pieces of work have, as yet, had only moderate impact on building ownership and coalitions for change in these respects. There has been some impact on reducing the burden of local taxes on exports, although possible backsliding by way of substituting fees for reduced taxes will have to be watched. There have been some minor improvements in the behavior of crop boards, but significant change will not happen until the PSIA work on options for the boards has been completed and digested. As for impact on the Bank, some recommendations were incorporated in the PRSC agenda, and the continuing work will feed into the next CAS. The work was not designed to underpin lending, except for supply chains, although lending in other areas could well ensue in, for instance, transitional arrangements for crop boards' operations. In addition, farmers' groups and private businesses have been provided with thorough analytic work on agricultural exports that they may be able to use to lobby government more effectively. 62. Comprehens iveness. The heavy focus on agricultural exports seems out of proportion to the scope for poverty reduction, but perhaps earlier work dealt adequately with commodities for domestic markets. The work on supply chains has some overlap/complementarity with exports and crop boards, but this is not explained. The impression is that these tasks were undertaken quite separately. Other than the work on institutional arrangements for rural development, the panel does not know of other AAA tasks dealing with other aspects of rural development. 63. Planning and Bank Processes. The inability to find concept papers and reviewers' contributions for the initiation of work on agricultural exports, and later supply chains, may just be an administrative lapse. If not, it reflects something seriously amiss in the planning of AAA. At least, initiation of the work on institutional arrangements for rural development, and on crop boards, was given serious consideration as demonstrated in the supporting documents. Cross-Country ARD Sector AAA Assessment 37 UKRAINE S ECTOR AAA COUNTRY ASSESSMENT 64. Summary Comments. While the operating environment in Ukraine is complicated by the frequent changes of government and wild swings in the policy environment, the Bank has adjusted well to this situation through the creation of the Agriculture Policy Analysis Group, a group of western-trained Ukrainian economists who act as proxies for the Bank in agricultural analytical work and policy dialogue with government officials. Challenges remain in terms of how to place this institution, now three to four years old, on a sustainable financial footing, but the Region is to be congratulated on the innovative and entrepreneurial way in which this capability has been developed and the way it is used to enhance policy change. 65. This aside, the Panel is concerned that strategic and analytic knowledge of the rural sector in Ukraine within the Bank is being eroded. Perhaps because of the success of the Policy Group, AAA funded through bank budget over the review period has been minimal -- to the extent that it is zero in some years. The sector manager cannot operate a coherent and ongoing sectoral AAA work program when funding is insufficient and intermittent. An added difficulty occurs when tasks are multi- year but funding comes annually with no guarantee on the level of funding in the second or subsequent years. 66. Strategic Relevance. The AAA is set in a context of an intermittently dysfunctional policy dialogue. Despite this, the tasks are relevant and consistent with the CAS they fit well into the objective of being current on policy advice and agricultural strategy in the event of an opening in the policy dialogue. But coverage appears thin with only three pieces of AAA shown as tasks in the last four years. Staff are aware of gaps that exist because of budget constraints. There is little evidence of ex-ante success indicators. Work is undertaken in a timely manner. Given the very unstable operating environment, the AAA has focused on key issues and the work of the “Policy Analysis Group” in Kiev, has been very instrumental in arresting poor policy proposals, preventing back-sliding, and conveying World Bank policy messages. The work is well aligned with the country SSP (now becoming somewhat dated and being replaced by a new document produced in collaboration with OECD), but a remaining concern is that the sectoral knowledge is being degraded by severely-constrained AAA budget. 67. Internal Quality. Although little ESW has been completed recently, that which has been done is of very high quality and addresses key issues. Well-qualified and experienced specialist economists have undertaken complex analytical tasks and produced very good reports. While the Panel did not make an independent assessment, it appears that the quality of work undertaken by the Agricultural Policy Group is equally good. This group has also built up a database, which has benefits for World Bank sectoral economists when they undertake AAA. 68. Likely Impact. The AAA work is designed and carried out in a way that offers the promise of significant impact on the client. The difficulty arises from the (sometimes) unresponsiveness or even hostility of the client to this work and the weak ability to implement policy change even when a willing government is in office. The establishment of the Policy Analysis Group is an innovative approach and an excellent example of enhancing capacity and knowledge transfer. It is a best practice. The AAA work is highly relevant to the Bank and appears to have informed lending operations. The limitation arises from the highly constrained budget for AAA in ARD. The Bank has close consultations with key Cross-Country ARD Sector AAA Assessment 38 donors and joint funding of AAA is often undertaken. Thus the opportunity for AAA to impact donors exists. Partnerships with key donors are well developed and policy priorities appear to be quite clearly agreed upon. Civil society organizations are quite nascent in Ukraine. The opportunity to have an impact on them is quite limited. On the other hand, the emerging private sector stands to bene fit considerably from AAA, which has led into policy changes and operations. 69. Comprehensiveness. The unit has done a good job identifying priority areas that require analysis. Work has become tightly focused on three cross-cutting themes common to the entire CD. But it has not been possible to conduct a coherent work program as resources for AAA have limited work to less than one activity per FY. In this fiscal year, we understand that there is no budget allocation for ARD AAA. Staff acknowledge gaps in knowledge and an aging portfolio of sector strategy work. Further, and of concern to the institution, many pieces of AAA are informal and often unrecorded (as they may be financed off-budget). The concern is that this work does not become part of the institutional memory and is quickly lost to the institution. 70. Planning and Bank Processes. In the Panel's view, this is a very weak link in the country program. For a country where agriculture is so critical to national employment and growth, and where agriculture is so in need of fundamental reforms if it is to deliver those benefits, it is indeed surprising to observe that the sector receives such limited budget support and that dedicated sectoral economists on staff are fewer and fewer. The few pieces of AAA undertaken have been well planned and executed, but a sectoral AAA work program and plan really does not exist. Budget allocations are intermittent and, when made, inadequate. Staff are forced to contract-out much of the work and to spend considerable time soliciting and managing alternative non-BB financial support. 71. The September 2003 CAS for Ukraine notes several areas of interest (a forward agenda) identified by local policymakers as priorities. Some directly mention agriculture (land titling, secure transactions) while others affect agriculture less directly but nonetheless importantly (environmental sustainability, energy reforms, targeted poverty interventions, and improvements in the business environment). The AAA undertaken on the rural sector is consistent with these priorities. The ARD AAA that has been undertaken appears to have had an important influence on the thinking of policymakers in Ukraine and has contributed directly to operations. Most operations related to agriculture have taken place within the context of the three PALs that have been the centerpiece of operations, and these components have been informed by prior ARD sector work. This is a particularly strong aspect of the ARD work, a finding that is common with that of the general AAA assessment for Ukraine undertaken in 2003. The major issue requiring attention is the disconnect between the importance of agriculture to economic performance and growth – the imperative of making a successful transition to a market-driven economy in the rural sector – and the amount of BB resources allocated in the country program to agricultural AAA. In recent years, the budget allocation for agricultural AAA has financed less than one piece of work per year, and even then task managers must seek outside resources to undertake the work at the scope and quality expected of bank reports. In other words, necessary, front- line work that can be legitimately funded from BB is being under funded, forcing diligent staff to seek off-budget funding and to work off budget to complete AAA tasks. 72. The lack of regular and sufficient funding for ARD AAA has staffing implications. It is increasingly difficult for sectoral units to employ dedicated (or even 'a' dedicated) sectoral Cross-Country ARD Sector AAA Assessment 39 or policy economists. When individuals have joint responsibilities with lending, projects tend to crowd out sector work. And lack of sector work means that the Banks analytical expertise in ARD is being seriously undermined. It may get to a situation where we cannot credibly fill a seat at the rural policy table. Cross-Country ARD Sector AAA Assessment 40 ANNEX 4 COMMENTS ON THE TECHNICAL QUALITY OF SPECIFIC STUDIES IN BANGLADESH , MEXICO , AND TANZANIA COMMENTS ON “PROMOTING THE R URAL NON-FARM S ECTOR IN BANGLADESH Main Report, Volume II 73. This paper contains a huge compilation of data and description about rural Bangladesh, and the economic connections and relationships among agriculture, the rural non- farm economy, and the rest of the country. It looks not only at the current situation but also trends linking past changes and current conditions, notably how trends in productivity, trade, investment, urbanization, and policies have affected the rural poor. A problem however is that the bulk of the analysis consists of associations rather than identification of underlying causal factors, which ends up providing disappointingly little grounding for policy or lending recommendations. To review the Main Contents 74. The introductory section (pages 1-10) contains basic history and facts, and most basic for the rest of the paper, a discussion of the conceptual framework for the work. This framework however consists mainly of definitions of concepts such as “rural area” and “non- farm” which of course help in keeping straight the categorization of data, but do not accomplish an ana lytical task. Figure 1.1 provides a visualization of “the rural- urban and farm- non-farm continuum,” which shows linkages in five connected locations from village to metropolitan city (one of two dimensions), and from primary production to final goods and services (the second dimension). This organizational setup is a necessary first step in analyzing the economic situation in Bangladesh when the focus is to be on a largely non- standard object of study like the rural non-farm sector, but here it is, conceptually, also the last step taken. More in the way of conceptual framework is needed. For example, page 5 states that “average labor productivity” is higher in RNF activities as compared to agriculture. Presumably, this is measured by output per worker although this isn’t stated. What we need from an analytical framework is a capability to interpret an indicator such as labor productivity. It could be a matter of factor ratios (more capital per worker in RNF), or relatively higher skills in RNF work, or disequilibrium in labor markets (with excess labor in agriculture). One has to conceptualize these or other reasons in order to formulate hypotheses about what policies or investments make sense for the RNF sector or for a decision to give RNF investment priority over alternatives (like agricultural technology investment). Average labor productivity being higher doesn’t necessarily imply that the value of marginal product of labor is higher in RNF (though it may well be). The paper should have pointed to the distinction and tried to assess the facts of this matter. This is the kind of conceptualization necessary to ask the most useful questions of the data, and the lack of that in the paper ends up being costly in terms of relevance of the findings. 75. On page 5, data are cited showing that the RNF sector generates a high share of rural income, and that share is growing relative to the agricultural sector, and that as quoted above labor productivity is higher. Evidence is cited that higher income is associated with RNF Cross-Country ARD Sector AAA Assessment 41 growth, inferring that RNF growth has been pro-poor. What is the implication? It seems to be that it is a good idea to encourage RNF growth. Yet Figure 1.2 shows that agricultural income predominates for the lowest deciles of the income distribution. So if we have two possible alternative policies or investment strategies for reducing rural poverty, one focused on increasing income from agriculture and the other on increased income in the RNF sector. Which is preferred? Pages 6-7 argue that growth of the RNF sector is essential. So one might conclude that RNF promotion is preferred. Yet the first “emerging opportunity” put forth is about yield growth in agriculture, as “an essential pre-condition for accelerated RNF growth” (page 8). The remainder of the introductory section goes back to RNF promotion, but I was mystified as to what the rest of the paper was going to do to help make choices about policy or investment alternatives. The concluding section of the introduction says the paper “focuses particularly on identifying constraints to promoting” the RNF sector in Bangladesh, and will conclude with a strategy for promoting that sector (pages 8-9). So the paper by-passes key issues of priority mentioned above. This is fair enough, but it does limit the relevance of the work. 76. Section 2 describes and mobilizes data on the linkages between agriculture and the RNF sector. The “bottom line” for analytical purposes is a set of constraints to growth of “agriculture related RNF activities” (page 16). These are: (1) low yields and lack of year- round supply of high value crops; (2) low yields in livestock; (3) lack of food quality assurance; (4) improvements needed in handling, storage, processing, and transporting of perishable commodities; (5) improvements needed in quality of inputs; (6) lack of a “fully integrated supply chains for fruits and vegetables;” (7) tariffs, taxes, and regulations that discriminate against (small-scale) agro-processors; and (8) needed infrastructure and finance for RNF linkages forward and backward. These are reasonably likely constraints but the factual basis for choosing them, or prioritization among them, is not provided. 77. Section 3 addresses poverty, inequality, and access to the RNF economy. Data outlined in the introductory section are elaborated upon, with more locational detail about employment and income sources. Constraints to better access are identified as: lack of education, distance to source of RNF employment and existence of markets in nearby villages, electricity availability, credit availability, and gender-related constraints (page 25). The analytical basis for identifying these constraints is a regression analysis of household data on non-farm employment, with statistically significant variables taken to imply the constraints listed. That is, if a variable is significantly and positively related to non-farm employment, that is taken as evidence that increasing the correlates of that variable will increase employment, which is reasonable. 78. Section 4 provides further regional detail, and brings in the urbanized parts of Bangladesh. The section concludes not with constraints but with contributors to more balanced (regionally more uniform) growth and income levels. These factors are an efficient transportation system, political and fiscal decentralization, increased openness to international trade, and strategic investment is smaller towns. Here the basis for choosing these factors is less apparent than in Section 3, and for some factors, notably the role of international trade, evidence is entirely lacking 79. Section 5 considers the climate for investment in rural areas, particularly for small or medium-scale enterprises. Growth of such enterprises is taken to be a promising avenue of RNF development. Survey responses indicated that constraints to such growth are lack of access to finance, exposure to flood and other disaster, electricity, road conditions, and Cross-Country ARD Sector AAA Assessment 42 (deficiencies of) transport to market. Governance issues (regulatory burden and corruption) were seen as bigger problems by large, urban-based firms. Proprietor education is important to the success of smaller enterprises. 80. Section 6 covers service provision to rural enterprises. Many of the same constraints discussed in previous sections are important here also, notably transportation and finance, but in this section with more emphasis on communications, particularly telecommunications. This section also sees the need for “strategic public intervention in providing the regulatory framework and in ensuring access of the poor to services” (page 71) to an extent not approached in the preceding sections. 81. Section 7 (pages 75-84) is where all the preceding strands are pulled together to outline “strategy and options for promoting the rural non- farm sector” (page 75). Here the doubts engendered by the wide range but too casual substantiation of the findings of the preceding sections come to fruition (on non-fruition). The recommended strategy has two main thrusts: (i) “improving the rural investment climate,” and (ii) “devising an institutional framework for efficient delivery of services” (page 77). 82. The components of (i) are: (Section 7.6) removal of anti-rural bias in trade and macro-economic policies; removal of constraints in physical and financial infrastructure, notably in (Section 7.7), roads, (Section 7.8) electricity, (Section 7.9) telecommunications, and rural finance (Section 7.10); (Section 7.12) improving disaster mitigation; and (Section 7.11) providing public goods (information, research in agriculture) and a regulatory framework for the agro-food supply chain. The discussion then goes on to (ii) recommendation for implementing the strategy through institutional changes, including involvement of RNF stakeholder groups in government decision- making, decentralization of public decision- making to local levels, a framework for NGO participation, and new ways to provide business development services (pages 83-84). 83. The strategy is reasonable but the ties back to the analytical work of the paper are weak overall and for some components hardly exist. A skeptical reader would ask, why these investments and policies and not others? From the point of view of task managers, we really do not have evidence on rates of return (or benefits and costs) to be expected from the recommended actions, even in rough terms. Perhaps though this is too much to ask of even a fairly large piece of economic sector work. The paper does lay out a lot of policy and investment options, placed in the context of the specific situation of Bangladesh today. It nonetheless remains the case that in order to prioritize among the ideas put forward, it would take substantial further analysis beyond what this paper has provided. 84. One may also ask about a basic presumption of the paper – that investment and policies that encourage the growth of RNF are preferable to a growth and poverty reduction strategy for Bangladesh that does not aim specifically at RNF development. On this question the paper cites, without much detail, a literature on RNF sectors in other countries that is said to support the wisdom of RNF focus. I rather doubt the decisiveness of the findings of that literature in allowing us to conclude that the menu of Section 7 will be a more effective poverty reduction strategy for rural people than, for example, investment in research on rice or other big internationally traded crops, improved elementary education of rural youth, investment in public health-related activities, to name a few. Not that the paper could have examined all alternatives; I just think it should have examined in more economic-return detail the benefits and costs of the topics the paper did cover. Cross-Country ARD Sector AAA Assessment 43 COMMENTS ON “M EXICO LAND POLICY – A DECADE AFTER THE EJIDO R EFORM” 85. This is an excellent piece of work. I will go over its main points as I see them and then give my assessment. I am looking at the paper with a view, as requested, to its analytical quality, usefulness as an instrument for policymakers in the country, and extent to which it provides useful information to Bank managers in country sector strategy and to task managers in the design of lending activities (note from Amnon Golan). 86. The paper contains a lot of interesting and relevant factual and historical background in its Sections 2 and 3 (pages 2-34), but the heart of its analytical contribution is in Section 4 (pages 34-51), “Impact of the legal changes of 1992 and the PROCEDE.” This is a really nice exercise in social science research. It attempts to provide on-the-ground empirical evidence on the effects of Mexico’s ambitious program to establish (something approaching) individual property rights in land ownership by residents of ejidos, which had been, roughly, under a regime of collective ownership. This change was made in hopes of economic gains for the farmers but was controversial, as the paper describes, because of fears of the kind that typically arise when private-property/market institutions are given greater scope. It is therefore quite important to assess with as much care and objectivity as possible what has happened in the decade since 2002 and what is plausibly attributable to the policy changes. 87. The paper’s analysis is based on a large (1297 observation) panel of households surveyed twice, in 1994 and 1997 (so we really only get to five years after 1992). The surveys, done with FAO and the University of California (not the “University of Berkeley” as page 35 has it), appear to have been carried out with high standards and must have been costly to conduct. The survey asked not only about technical and economic aspects of agricultural production, but also about a range of socio-economic characteristics of the households and about respondents’ subjective perceptions regarding the PROCEDE (Table 6). 88. A reasonably sophisticated econometric analysis estimated effects of the policy changes on participation in land rental and sales markets, access to credit, household income growth, and access to natural resources (common lands). The findings were of greater participation in land markets, no loss of access to common lands, and no improvement in access to credit. In addition, the subjective responses indicated improvements in “social peace,” presumably because of settling uncertainties about who had rights to what. Most importantly, the study found reasonable strong evidence that the policy changes increased household incomes, and argued, to me convincingly, that this was likely because the greater tenure security meant people could work off the farm without the risk that by leaving their land relatively less fully tended to take such jobs they would open themselves up to loss of land rights to others in the community. 89. The recommendations section of the paper is reasonable, but many of them do not follow directly from the findings of the analytical work. The real contribution of the paper in my view is the support it gives to the wisdom of the 1992 policy changes and the PROCEDE process (notwithstanding the political problems of incentives within that process that the paper straightforwardly discusses). Turning to the criteria for evaluation of the paper Cross-Country ARD Sector AAA Assessment 44 90. Analytical quality: excellent, on the frontiers of applied econometric work in policy analysis, first rate attention to detail in the surveys, substantively appropriate asking of questions and creative ways of marshaling data to answer them. 91. Usefulness to bank managers: sector strategy. Very high marks. A prime difficulty with policy recommendations by the Bank is that they can be taken as too much assertion based on general principles without fact-based consideration of a particular country’s situation. The findings of this paper go a long way to answering that sort of criticism. Recommendations for further implementation of PROCEDE-like policies now have a much more solid grounding than they would have in the absence of the research. 92. Usefulness to task managers. Here the limited scope of the paper makes the usefulness much more constricted. The paper provides no support for (or against) most of the kinds of loans the Bank makes. The area most directly relevant might be investment in financial institutions to overcome credit constraints facing rural households. But even here the negative finding of the research – that PROCEDE and related strengthening of property rights in land have not enabled a significant increase in land-based lending (because, as I understand it, the ownership rights cannot be conveyed to a bank or other lender through foreclosure, so hamstringing the use of land as collateral) – has direct implications for the expected returns to rural- finance institutions. 93. The contrast with the Bangladesh paper is striking. I was critical of that paper because it did not provide the kind of analytical concentration and quantified findings that the Mexico paper does. On the other hand, the Bangladesh paper took on a much wider range of issues and options. Would the Bank rather have what I would characterize as a relatively narrow and deep analysis, as the Mexico paper provides, or a wide and shallow one as the Bangladesh paper provides? The exercise so far is inclining me to prefer the narrow and deep approach for economic and sector work. The broader assessment is necessary to see the range of what might be possible. But, without being fully familiar with the range of preparatory work the Bank does, I believe the broader assessment is more effectively done by the country experts within the Bank as part of the broader country assessment papers, and the economic sector work is more usefully focused on a relatively narrow range of options which it should analyze in depth sufficiently to be able to come up with some quantification of potential benefits and costs of policy or investment alternatives. COMMENTS ON “TANZANIA RURAL D EVELOPMENT” 94. This paper is not a description of a single project of sector work, but rather contains two parts: a report on a fairly broad program of activity highlighting rural development, and a focused analysis of agricultural export growth, as “a key pillar of the strategy to reduce income poverty” in Tanzania (page vi). 95. The first major section of the work (pages 5-26) addresses Tanzania’s rural development strategy and its implementation. For purposes of carrying out analytical work, it is most helpful that the Government has prepared this strategy. Having the Government’s own strategy in place limits the huge range of alternative policies and programs that might otherwise have to be considered. The strategy has four “pillars”: (1) promoting widely shared growth across components of the rural economy, from small farms to tourism to fisheries; (2) increasing access to services, from education and health to technology and energy; (3) reducing risks and vulnerability; and (4) good governance. Each of these covers Cross-Country ARD Sector AAA Assessment 45 a lot of territory, so implementation requires narrowing down the sphere of action as well as spelling out implementation processes. 96. In addition to narrowing the scope in a broad sense, the Government of Tanzania has already accomplished some necessary further focus with respect to the first pillar through its agricultural sector development strategy, aimed at improved farm income and reduced rural poverty through improved productivity and profitability of the agricultural sector. The Report, after several page s devoted to institutional issues of implementation which are sensible but not analytical challenging (or even interesting in my view), takes on the substantively central question of prioritization of implementation alternatives. This is done principally by prioritizing expenditure options. 97. The discussion of priorities begins with those the Government identified in its broad Poverty Reduction Strategy, namely, primary education, primary health care, agricultural research and extension, rural roads, water, judiciary, and HIV/AIDS. The Report takes the appropriate analytical approach of stating the chief criterion for prioritization as being choice of options where the marginal returns to public expenditure are highest, and proceeding to the point that the marginal return to additional expenditures is equal across expenditure areas (page 16). The practical analytical question though is how to carry out this approach. Given the huge data and analytical requirements of applying these principles, the paper starts out with the reasonable approach of reviewing what has been learned from this kind of prioritizing activity in other countries, notably, China and India, which have been intensively studied, and some particular quantitative studies in Africa. This information is then nicely merged with considerations about Tanzania to argue that two areas of public investment offer notably high returns: agricultural research and rural roads (pages 17-18). The remainder of Section 2 discusses prioritization further but goes not carry out any further quantitative analysis. It is nonetheless quite valuable to sketch out the criteria and methods of doing such analysis, and underlining the importance of an analytical economic (as opposed to political or rhetorical) approach to prioritization. 98. The second major section of the Report (pages 27-59), and the five Annexes (pages 70-202) that make up the bulk of the Report, focus on agricultural exports. This section provides an excellent compendium of facts and trends, and does this not only for aggregate agricultural exports but also for individual key commodities. Moreover, the discussion moves beyond the domestic factual basics to international comparisons and analysis of the economic behavior of Tanzanian producers. These steps are essential in the knowledge base for policy, because one has to know how Tanzanian agriculture will respond to alternative price and other policies in order to gauge the benefits and costs of the policies, and this is knowledge that is all too often lacking in discussion of policy alternatives even for the advanced industrial economies. The sub-section on supply response and the DRC indicator of competitiveness is a nice illustration of how to combine an intuitive analysis with key facts, and findings from a surprisingly extensive outside literature, to provide parameters one needs to carry out a quantitative analysis of policy alternatives. 99. The conclusion and recommendations (Sub-section 3.11) reviews the situation and ends with six bulleted recommendations (page 59). Given the wealth of well- focused detail that precedes those recommendations, it is disappointing that the recommendations do not pull the analysis together to quantify what is at stake in the policy choices recommended. For example, the first recommendation is that “taxes on export crops must be reduced.” Instead of just presenting this in the imperative mode, it would have been far preferable to Cross-Country ARD Sector AAA Assessment 46 present at least a rough quantification of the gains to be had by reducing those taxes. For example, if Tanzania were to tax its coffee exports 10 percent less, how much more would producers get, how much more would they produce in response, what is that additional value generated for the Tanzanian economy, and how does that compare to the tax revenues lost. For most commodities in most developing countries, there would not be sufficient analytical information even to begin an answer. This paper provides a great deal of the necessary information for Tanzania, so it is frustrating to see the final steps not taken (or if remaining defects of knowledge prevent that, the paper spelling out what additional data are needed and how they would be used). 100. The remaining five recommendations are broader, and not capable of benefit-cost quantification given the information available. But the example of the first could be made more helpful by giving policymakers a better grip on what kind of information and analysis would be required to assess the real gains available from those policies. The recommendations on investment and growth could have been expanded upon briefly to discuss the increase in complexity necessary to assess the dynamic effects of policies, where the big gains are likely to be reaped only after many years of consistently followed policies. 101. Following up the thread of the preceding remarks, it would have been useful for the paper to have had a brief concluding section that integrated the two main sections. The first section sets up general “best practice” procedures for policy prio ritization. The second section provides a good case study of how to implement those procedures using the highly relevant case of exported commodities. The implementation exercise is useful not only for its specific content but because it gives concreteness to the more general ideas of the first section. Without the implementation exercise, the general prioritization could easily appear to government officials or others influential in decision- making to be too “academic” or lacking in practicality. A brief concluding section could have stressed the division of intellectual labor between the two main sections (and would have given further impetus to pushing the export policy section toward a more finished benefit-cost analysis). 102. Although I have ended up being critical of the Report for not going as far as it could have gone, this does not detract from the high quality of the material that the Report contains. In usefulness to Bank managers in rural development strategy formulation, and to task managers in lending, the data and approach to analysis of data provide an excellent baseline of information and a benchmark for analytical contributions needed in the evaluation of particular policies or lending programs. The usefulness to Tanzanian policy professio nals is similar but in addition is helpful, as far as it goes, as an example of best practice in practical policy analysis.
Pages to are hidden for
"CS AAA ARD"Please download to view full document