In 2005, the World Bank launched Brasil Avaliaçao (BRAVA), a modest technical assistance initiative to help the government of Brazil strengthen country systems for monitoring and evaluation (M&E). The programme also hopes to promote a culture of M&E in order to provide feedback on the performance of key programmes, to promote accountability, and to inform government spending decisions.

The BRAVA has adopted an innovative approach to promote M&E from the top down and bottom up, by supporting activities at both the institutional and programme levels.

At the institutional level, the BRAVA is supporting the institutionalisation of M&E across government, as well as the development of sustainable country systems for results-based M&E. This involves clarifying responsibilities for coordination and implementation, establishing incentives for line ministries to carry out robust M&E, and developing solid quality standards that will encourage the utilisation and dissemination of M&E information.

Many Brazilian agencies are involved in M&E and have some mandate for different aspects of M&E oversight. So far, the BRAVA has focused on identifying and strengthening the role of key ‘champions’ of M&E, through activities such as providing technical support to the Inter-Ministerial Commission for Monitoring and Evaluation, which coordinates the government’s policy and systems for M&E of federal programmes, and conducting institutional assessments of M&E systems in line ministries.

At the programme level, the BRAVA’s technical support for M&E activities emphasises the full results chain. For ‘monitoring’, the results chain includes the monitoring of inputs (financial indicators), processes (oversight and controls), and outputs and intermediate results (key success indicators). For ‘evaluation’, the results chain includes both implementation and impact. Activities to date have focused on the design of overall M&E systems, log frames and impact evaluations for several key strategic programmes.

Emerging lessons

Although the BRAVA programme is still evolving, some important lessons have already emerged.

Above all, efforts to promote M&E should work with the country’s own systems and institutional structures, rather than trying to create new ones. Support to emerging champions for M&E has resulted in greater ownership, the diffusion of impact throughout the various ministries and programmes, and increased capacities. M&E support should be demand-driven, and should involve senior officials. Ultimately, the goal of ‘BRAVA-type’ programmes should be to transfer the functions and capacities to key M&E champions in the government.

As noted above, strengthening M&E systems from both the top down and the bottom up can produce considerable synergies and demonstration effects. Within the Ministry of Social Development, for example, the BRAVA is supporting programme-specific evaluations, as well as institutional M&E assessments, that have had important cross-fertilisation effects both for our work within the Ministry and for our assistance to other line ministries and central agencies.

The right mix of tools

It is important to take a broad view of M&E, and to adapt common-sense tools tailored to the situation of each institution or programme. With some programmes, a full-fledged impact evaluation is needed to demonstrate impacts. With others, experimental evaluations are not feasible (and perhaps not cost-effective), in which case priority is given to implementation evaluations or log frame exercises to identify tractable indicators for monitoring purposes. In many cases, a range of instruments can be used in a mutually reinforcing way. Thorough discussions with the institutions and programme managers concerned can help diagnose objectives and the feasibility of tailoring the right mix of M&E tools to each situation.

A steering committee, with representatives of key central agencies and line ministries, can enhance the prioritisation of BRAVA-type activities at both programme and institutional levels. Steering committee meetings can also serve to promote dialogue about M&E initiatives and champions across the various agencies. As such, they often help to create an M&E policy community that can ensure the cross-fertilisation of experiences.

A number of other countries in Latin America and beyond are now looking to replicate and adapt BRAVA-type M&E support initiatives, and to build on the valuable lessons learned in Brazil.

Links

World Bank IEG, Evaluation Capacity Development (ECD)

World Bank: Impact Evaluation

World Bank Independent Evaluation Group (IEG)

World Bank Participatory Monitoring and Evaluation (PM&E)

World Bank Poverty Monitoring

Further reading

K. Mackay (2006) Institutionalization of Monitoring and Evaluation Systems to Improve Public Sector Management, ECD Working Paper 15, World Bank IEG.

E. May et al., eds (2006) Towards the Institutionalization of Monitoring and Evaluation in Latin America and the Caribbean. Proceedings of a World Bank/Inter-American Development Bank Conference. IBRD/World Bank.

World Bank (2004) Monitoring and Evaluation: Some Tools, Methods and Approaches, 2nd edn, World Bank IEG.

A. Zaltsman (2006) Experience with Institutionalizing Monitoring and Evaluation Systems in Five Latin American Countries: Argentina, Chile, Colombia, Costa Rica and Uruguay, ECD Working Paper 16, World Bank IEG.