Project B5:

Robust methods for the evaluation of policy reforms



Professor Dr. Markus Frölich / Professor Dr. Christoph Rothe 


Nicholas Barton, Ph.D. / Dr. Atika Pasha / Dr. Clementine Sadania / Dr. Arne Weiß


Prof. Dr. Enno Mammen / Professor Dr. Gerard van den Berg / Professor Daniel Gutknecht, Ph.D.

 Duration:01.01.2018 - 31.12.2021



Political reforms are an ongoing process. As time proceeds, earlier reforms are modified or withdrawn and new reforms are implemented. In this process, evaluating reforms and policy measures plays an important role. Reforms that lead to policy changes or new policy programs may have been based on an erroneous assessment of their effectiveness. Alternatively, such new policy programs may lose effectiveness as a result of changing circumstances in the economy. Indeed, the societal support for reforms in general may depend on the extent to which the public believes that policies are carefully evaluated on the basis of empirical evidence. Whereas policy makers may have an interest in hiding information from the public or in depriving the public of information, objective and truthful information can help to reduce resistance to reforms in general. High-quality policy evaluations are therefore important. These, in turn, require a combination of high-quality data and high-quality econometric and statistical methods. The current project aims to develop novel methods for econometric policy evaluation and to empirically evaluate policies using high-quality data. In this process, permitting heterogeneity is essential: Not all individuals benefit (or are harmed) to the same extent by a particular reform (which we label as impact heterogeneity in what follows), and it is thus important to know who (or how many) loses or gains from a certain reform alternative. This means that not only average expected gains or losses need to be assessed but also their distribution. Evaluation of distributional effects is crucial as political support often not only depends on average effects but also on effects on inequality. For example, in a majority vote setting, a reform would be rejected if the median effect is negative even if the average effect is positive. Econometric methods further need to account for endogeneity and selectivity problems. Those who benefit from exposure to a new policy might also have done well in the absence of the policy. As a result of political considerations, large-scale reforms are often implemented during economic downturns, and as the economy recovers in subsequent years, the reform may appear to have been successful even if it had no real effect at all. Here, sound econometric methods that can deal with this problem and consider heterogeneity are pivotal. This research will be carried out in conjunction with the empirical evaluation of the impacts of educational policies in Europe. In the third funding phase, we will focus in particular on the education and health projects in France, Germany and Sweden that have been started in the second funding period and run until 2020 with repeated longitudinal data collections.