Mastering Impact Evaluation: Strategies, Methods and Tools

7-8/Jul/2014  •  Maastricht

Project Leader
For more information about the activity, please contact:

Target group
This two-day seminar is directed towards civil servants from EU Member States, candidate countries, the EU institutions and other multilateral institutions, as well as auditors, consultants, staff of NGOs and other stakeholders involved in the process of impact evaluation of policies, programmes and projects.

In the past, evaluation was focused to a large extent on evaluating the mechanisms of delivering programmes or projects. In other words, they focused on the implementation process. Relatively little attention was paid to evaluating the impact of interventions that were being financed. While over the last few years substantial efforts have been made to promote and disseminate counterfactual impact evaluation practice (quantitative), there is much less sharing of good practice concerning so-called theory-based impact evaluation. However, while counterfactual impact evaluation can tell us how much of an impact an intervention has, theory-based impact evaluation is crucial to increasing our understanding of ‘why’ certain interventions work or do not work. Without this understanding, it is not possible to improve interventions. In addition, in many cases counterfactual impact evaluation is not actually a viable method, and only theory-based impact evaluation will be able to shed light on whether or not the intervention works.

The seminar will therefore focus on this theory-based impact evaluation, drawing on the latest advances made in terms of methodology. The most cited methods – Process tracing, Congruence Analysis, QCA, Pawson and Tilley’s Realist evaluation, Mayne’s Contribution Analysis, Scriven’s General elimination method – will all be covered and put into perspective. Several cases will be used to exemplify these methods. The participants will also execute many exercises to better grasp the methods. An important focus of the course will be to match the rigor associated with counterfactual impact evaluation when making causal claims.

Learning methodology
The seminar is based on a range of presentations and interactive group work to enable participants to put their knowledge into practice. The seminar integrates case studies to allow participants to assess the quality of actual evaluations and how they could have been improved.

At the end of the course, participants will be able to design and commission theory-based impact evaluations.