Methodologies

SIDW/Charney Survey: The State of Development Evaluation

By Craig Charney | Insights | Series II | No. 1 | December 2013

What’s happening in development evaluation? How often do development projects include evaluation — and what kinds? How are evaluators chosen? How big are their budgets? How satisfactory are their results? To provide a snapshot of development evaluation today, the Society for International Development’s Washington Chapter partnered with Charney Research to survey development professionals, including funders, contractors, and evaluators, involved in development, stabilization, and humanitarian assistance.

People involved in evaluation can use the report to benchmark their work and knowledge against the rest of the field. Funders will find it helpful to measure awareness and application of recommended evaluation practices. It offers policy-makers a chance to see how far the field has come and ways to move it further forward.

KEY FINDINGS

• Just under half of development projects and proposals include both impact and performance evaluation, regardless of overall project budgets. Both types are likelier only when more resources are specifically committed to evaluations.
• Few projects include both baseline data collections and final evaluations.
• Evaluation budgets run below USAID guidelines half the time.
• Few respondents are very satisfied or dissatisfied with evaluations actually conducted; most are fairly satisfied. However, dissatisfaction is much more frequent when evaluation spending runs below the USAID guidelines.
• Development professionals are saying they need help most with three areas related to evaluation: methodology, capacity building, and resources.
• Local evaluators work with internationals on most evaluations.
• Some development professionals are unaware of USAID evaluation definitions and guidelines.
• Websites are the leading source of information on development evaluation, while the World Bank and USAID Impact blogs are the most widely accessed.
• Websites and live events are the most popular sources for evaluation education. Development professionals are quite interested in SID-W initiatives in the field, especially online groups, one-day workshops, and webinars.
• General methodological topics on development evaluation have the greatest appeal to survey respondents.

The survey was conducted online in Jan-Feb 2013 among development professionals on the SIDW mailing list. The survey was sent to 10,000 and 624 replied. Most were fairly senior: three-fifths were project managers or higher and the average number of years in the field was 15. Almost all worked on evaluation proposals or projects; the average number in 2012 was 8.