Evaluating International Programs: Answering the Toughest Questions
Abt’s combination of evaluation experts and subject-area specialists — in every field from education and agriculture to health and climate change and beyond — allows us to provide richer insights for our clients.
How can we increase the impact of international development work and make it more efficient?
International governments and donors increasingly are looking for rigorous, high-quality program evaluation to guide their work.
For more than 50 years, Abt Global has conducted hundreds of cutting-edge evaluations, both domestically and abroad, to provide clients with actionable insights based on evidence. What sets us apart?
“We are combining the best of our knowledge and substantive expertise in the international development context with our methodological evaluation expertise,” said Judy Geyer, Ph.D., an economist at Abt with broad expertise in evaluation.
Getting Rapid Feedback
In Ghana, Abt evaluators used data from the field from face-to-face interviews combined with mystery client surveys to explore self-reported and actual recommendation practices for treating childhood diarrhea. Abt researchers conducted a randomized controlled trial to determine whether mobile phone messages could improve the prescription behavior and practices. One of the biggest challenges in evaluating programs is that the answer to the question – did it work? – can come too late for meaningful action.
“Donors want to know if their work is having an impact, and it is not practical for them to wait until the end of a project to know,” said Melissa Chiappetta, who leads Abt’s Center for International Evaluation. “They need to move quickly and adapt along the way.”
To help clients do just that, Abt is employing rapid feedback monitoring and evaluation as well as utilization-focused evaluations, which engage clients throughout the evaluation to define and refine the questions they are seeking answers to about a particular program. This way, Abt can deliver results that are actionable and improve programs in real time.
One example of this is through our work with USAID Global Development Lab’s Monitoring, Evaluation, Research and Learning Innovations Program (MERLIN). The program innovates on traditional approaches to monitoring, evaluation, research and learning so that a project can benefit from learning what worked – or didn’t – in the early stages of a program’s implementation.
Applying Insights from Implementation
Abt’s combination of evaluation experts and subject-area specialists — in every field from education and agriculture to health and climate change and beyond — allows us to provide richer insights for our clients.
“We conduct evaluations with the eyes of someone who also implements programs,” said Tulika Narayan, Ph.D., an evaluator who specializes in agricultural development and applied econometrics. “You have to understand the larger context and what a program plans to do to really evaluate it.” “We conduct evaluations with the eyes of someone who also implements programs,” said Tulika Narayan, Ph.D., an evaluator who specializes in agricultural development and applied econometrics. “You have to understand the larger context and what a program plans to do to really evaluate it.”
As an example, Abt has been able to draw on its expertise in agriculture and nutrition in evaluating the multi-donor AgResults initiative. The project is designed to encourage and reward high-impact agricultural innovations that promote global food security, health and nutrition. We are evaluating the effectiveness of pilot projects in Nigeria, Kenya, Zambia, Uganda, and Vietnam that use “pull mechanisms,” or results-based prizes, to spur private sector actors to develop and market innovative agricultural technologies. Abt’s evaluation will assess the overall effectiveness of the AgResults model, as well as the impacts of each pilot on smallholder farmers. Evaluators will tailor the evaluation designs based on the needs and challenges of the various project sites.
We also have a strong legacy of incorporating rigorous research, monitoring and evaluation in our international health projects – everything from gaining insights on the best ways to treat childhood diarrhea to understanding the impact of financing mechanisms as they relate to delivering maternal and child health services.
Focusing on Innovation to Overcome Challenges
AgResults team members conduct interviews in Nigeria. AgResults is evaluating the results of “pull mechanisms,” or incentives for private companies to develop and disseminate high-impact agricultural innovations that promote food security and benefit smallholder farmers. Evaluation in the context of international development work can bring a range of challenges, including a lack of infrastructure to support such evaluations. But a little creativity and ingenuity can help solve this.
“The obstacles can become opportunities to invest in better tools,” says Stephen Bell, Ph.D., a senior fellow at Abt who has spent more than 30 years evaluating how social programs can improve the lives of people worldwide.
For example, in Mozambique, Abt conducted an evaluation for the Millennium Challenge Corporation (MCC) Farmer Income Support Project to determine whether the program could limit the ravages of a lethal coconut tree disease that threatened the livelihoods of millions of people. Evaluators used a quasi-experimental study design and employed satellite imagery to help determine the extent of the tree disease and evaluate the program’s impact.
In Ghana, Abt researchers conducted a randomized controlled trial to determine whether mobile phone messages could improve the prescription behavior and practices for the treatment of childhood diarrhea. The evaluators were able to use data from the field from face-to-face interviews combined with mystery client surveys to explore self-reported and actual recommendation practices.
In addition, we apply rigorous approaches to evaluation in low-resource environments – a common challenge in international development.
“We will find the right tool to provide the client with the right answers and a clear picture of the reliability of the data,” Narayan explains. “We can evaluate anything.”
Read more about Abt’s work in evaluation:
