Project B5:

Robust methods for the evaluation of policy reforms



Professor Dr. Markus Frölich / Professor Dr. Christoph Rothe / Professor Daniel Gutknecht, Ph.D.


Marc Gillaizeau / Nicholas Barton / Mariam Nikravech / Atika Pasha  / Tereza Varejkova / Clementine Sadania / Michael Theilmann


Prof. Dr. Enno Mammen / Professor Dr. Gerard van den Berg

 Duration:01.01.2018 - 31.12.2021



Political reforms are an ongoing process. As time proceeds, earlier reforms are modified or withdrawn, and new reforms are implemented. In this process, the evaluation of reforms and policy measures plays an important role. Reforms that lead to policy changes or new policy programmes may have been based on an erroneous assessment of their effectiveness. Alternatively, such new policy programmes may lose effectiveness as a result of changing circumstances in the economy. Indeed, the societal support for reforms in general may depend on the extent to which the public believes that policies are carefully evaluated on the basis of empirical evidence. Whereas policy makers may have an interest in hiding or in depriving the public of information, objective and truthful information can help to reduce resistance to reforms in general. All of this points to the importance of high-quality policy evaluation. This in turn requires a combination of high-quality data and high-quality econometric and statistical methods. The current project aims to develop novel methods for econometric policy evaluation in combination with the empirical evaluation of policies using high-quality data. We focus on a number of issues. First, we examine the heterogeneity of individual actors and heterogeneity in policy effects. Not all individuals benefit (or are harmed) to the same extent by a particular reform (which we label as impact heterogeneity in what follows), and it is thus important to know who (or how many) lose or gain from a certain reform alternative. This means that not only do average expected gains or losses need to be assessed but also their distribution. Such information is required ex ante, but also ex post, after reforms have taken place, to assess their fiscal and/or political sustainability. Reforms might then be modified in an attempt to reduce the inequality of the reform impacts (perhaps at the cost of reducing the average impact). The second focal point concerns the fact that the empirical evaluation of reforms is hampered by endogeneity and selectivity problems. Those who benefit from exposure to a new policy might also have done well in the absence of the policy. As a result of political considerations, large-scale reforms are often implemented during economic downturns, and as the economy recovers in subsequent years, the reform may appear to have been successful even if it had no effect at all. Here, sound econometric methods that can deal with this problem are pivotal. Current methods often rely on assumptions that are convenient from a statistical or practical point of view but that are not justified from an economic point of view. Moreover, current methods often have difficulties dealing with effect heterogeneity. We will, therefore, develop novel robust methods to deal with this. This will be carried out in conjunction with the empirical evaluation of the impacts of educational and labour-market policies in Europe. The latter is the third focal point of our project. In particular, we aim to evaluate school reforms in Sweden and Germany and labour-market polices in Germany and France. For this we have access to unusually rich longitudinal population register data. Moreover, we aim to carry out randomised social experiments for the unemployed and follow them over time to observe outcomes. Finally, to study the ex ante effects of participation in policy programmes, we aim to use survey data (including the German Internet Panel, the GIP) to elicit the beliefs and expectations of potential participants.