There have long been calls for policies to be evidence-based. Christian Gade reflects on the conditions under which a policymaker is justified in claiming that a given policy is evidence-based and considers how they can be put into practice.

‘Evidence-based’ has become a buzzword, often tossed around without consideration for when a policymaker, either an individual or an organization, is justified in claiming that a specific policy (let us call it ‘Policy A’) is evidence-based. I propose that a policymaker is justified in making such a claim if, and only if, three conditions are met.

This argument (presented below) implies that the justification not only depends on evidence, but also on the policymaker’s subjective preferences. It follows that the clarification of preferences should be a primary concern in the field of evidence-based policy, suggesting that the normative dimension of this field (the outcomes desired by policymakers) ought to receive increased attention.

The three conditions

1.The policymaker possesses comparative evidence regarding the effects of their policy compared to the effects of at least one alternative policy.

Claiming that Policy A is evidence-based is to assert that it is better than an alternative, and a policymaker is only justified in making that claim if they possess at least some evidence about the effects of Policy A in relation to the alternative. In the field of evidence-based policy, ‘being better’ is a matter of effects/consequences, which places this field in the consequentialist ethical tradition. Here, consequentialism should be understood in a broad sense where the good effects/consequences may be the realisation of any kind of preferences.

2.The policy is supported by the comparative evidence according to at least one of the policymaker’s preferences in the given policy area.

For example, let us assume that the policymaker has a preference for low recidivism in the area of crime management. Thus, if the policymaker possesses evidence showing that Policy A leads to lower recidivism than Policy B, then the second condition is met. Here, it is not enough that the policy is supported by the evidence according to a preference that someone may potentially hold in the given area. If that were the case, the policymaker could, as the list of potential preferences is endless, be justified in claiming that any policy is evidence-based, which would effectively render the term ‘evidence-based’ meaningless. Thus, to avoid that the term becomes all-inclusive and meaningless, a policymaker is only justified in claiming that a specific policy is evidence-based if the policy is supported by the evidence according to at least one preference that the policymaker actually holds.

3.The policymaker can provide a sound account for this support by explaining the evidence and preferences that lay the foundation for the claim that the policy is evidence-based.

Let us continue the above example and assume both that the policymaker possesses comparative evidence showing that crime management Policy A leads to lower recidivism than Policy B, and that the policymaker has a preference for low recidivism. If that is the case, the policymaker is justified in claiming that Policy A is evidence-based if they can explain that they have the mentioned evidence and preference. However, if the policymaker cannot provide any such explanation, but simply states that ‘Policy A is evidence-based because I say so’ or something similar, then the claim is not justified. In this connection, it should be noted that the third condition does not require that policymakers can provide the account for evidence and preferences completely on their own. It is fine if they provide the account with the support of others, such as brokers.

Some may be of the view that my conditions are not strict enough, believing that it is only specific types of comparative evidence, and only specific types of preferences, that count as a foundation for justified claims about policies being evidence-based. The problem with such an approach is that, due to the existence of different epistemic and moral communities, people will probably never reach agreement on what the appropriate types of evidence and preferences are. Thus, instead of excluding any comparative evidence and preferences from the realm of evidence-based policy as a starting point, I suggest a transparency approach where the important thing is transparency about evidence and preferences. Based on such an approach, I may say that it does not matter to me that a specific policymaker is justified in claiming that their policy is evidence-based, either because I do not trust the type of evidence the policymaker possesses, or because I do not share the preferences that lay the foundation of the policymaker’s claim.

Using these conditions in practice

So, what should be done in practice if a policymaker wants to implement a policy that they can reasonably claim is evidence-based? The first step should be to identify which preferences the policymaker has in the given policy area. For example, in the area of crime management, the policymaker may have a preference for low recidivism, high victim satisfaction, low economic cost, etc. Here, researchers or others may support the policymaker in identifying what their preferences are, for instance through a dialogical process with questions for the policymaker about their preferences.

The first step should be to identify which preferences the policymaker has in the given policy area.

When the preferences have been identified, the next step should be to consider the collection of comparative evidence. Here, there are two important questions to consider. First: should evidence be collected in relation to all the identified preferences? If there are not enough resources to do that, it may be reasonable to focus on collecting evidence in relation to the preferences that the policymaker finds most important for the decision about which policy to implement. For example, if low recidivism is much more important for the policymaker than high victim satisfaction, then it may, given resource restraints, be reasonable to focus on collecting evidence on recidivism. Second: what type of evidence should be collected? Here, it is important that the selected type of evidence is trusted by the policymaker in order for the policymaker to actually base political decisions on it. This calls for a dialogue between the policymaker and the researchers who collect the evidence regarding the type of evidence to be collected.

transparency about evidence and preferences is an important foundation for dialogue about the appropriateness of political decisions.

Finally, the last step is for the policymaker to decide which policy to implement. It may be the case that based on the policymaker’s preferences and the collected comparative evidence about the relative effects of two policies, Policy A and B, the policymaker is justified in claiming that Policy A is evidence-based in relation to, let us say, recidivism, while they are justified in claiming that Policy B is evidence-based in relation to economic cost. In such cases, the policymaker must weigh the difference in recidivism (where A does better than B according to the policymaker’s preferences) up against the difference in economic cost (where B does better than A according to the policymaker’s preferences) to decide which policy to implement.

Here, it is important to emphasise that the implemented policy may not be the one that others believe ought to be implemented, either because they do not trust the evidence that lay the foundation for the implementation, or because they do not share the policymaker’s preferences. In this connection, transparency about evidence and preferences is an important foundation for dialogue about the appropriateness of political decisions.

This post draws on the author’s open access article, When is it justified to claim that a practice or policy is evidence-based? Reflections on evidence and preferences, published in Evidence & Policy.

The content generated on this blog is for information purposes only. This Article gives the views and opinions of the authors and does not reflect the views and opinions of the Impact of Social Science blog (the blog), nor of the London School of Economics and Political Science. Please review our comments policy if you have any concerns on posting a comment below.

Image credit: MedRocky on Shutterstock.