I fundamentally don’t trust Ian Duncan Smith. I do trust, however, that he holds deep beliefs that social security benefits dissuade people from working. For me, I agree, but only in the same way that I believe a lack of oxygen dissuades people from breathing. I don’t think he’s right about causality, and he and I would argue that point until hell freezes over, the existence of which would probably be our next debate.
This week’s news that IDS’s department have admitted to policy being implemented to cap household benefit payments on the basis of belief rather than evidence has made me realise how little I believe in the idea of “evidence-based policy”. Here is my beef…
The idea of statistical evidence being used as the basis of the way in which political policy should be set and developed is seems to me to be using a pseudo-scientific idea of theory, experiment and result. In the world of pure science, that works kind of fine – that is until someone comes up with an entirely new theory which blows apart established wisdom. Unless we had people who had those lightbulb moments – Copernicus, Newton, Einstein et al. – we still be stuck in flat earth thinking. If my A-level Physics memory holds me well (and it possibly doesn’t – memory itself is a very subjective construct), Einstein’s lightbulb moment for relativity came not from observation, but from a dream he had about electrocuting cows in a field.
So let’s start from the principle that policy comes not from pouring over evidence, but from ideological belief. I’ve no problem with that – I might disagree with various ideologies, but at least you know where you stand.
At that point we get into a discussion about what statistical evidence might be used to substantiate a policy. And here comes our problem as humans of being rubbish with numbers. If we believe in something, then we find ourselves subject to a concept called confirmation bias: we seek out evidence that confirms our beliefs, and discount that that counters them. I don’t believe that, as a result, we’ll ever see “evidence-based” policy – just policies based on the balance of evidence as judged from a particular political or ideological perspective.
It might be tempting to think that evidence might be a way to justify why the existing status quo should be changed – but confirmation bias rears its head again. The evidence that you may provide to say why something isn’t working may even be the very same numbers that I could use to counter the argument (I used to run an exercise that took some immigration statistics to get groups to argue different political standpoints – you’d be amazed how a well-constructed histogram can be used to argue just about anything you would like).
If we really were in a political landscape where evidence-based policy were true, everyone would also have to change how we view political debate. Theory-experiment-result approaches to public policy seem to be unacceptable in our world of political discourse because it would have to mean acceptance that our politicians would be allowed to admit something they tried didn’t work without them being thrown to the lions. That would be a welcomed change, see a level of adult debate entering into our systems of government, and appears to me to be so far away from today as to be unattainable.
But then the final challenge of this all is that my old friends Campbell and Metcalfe rear their heads again. Evidence-based policy would require measurement; measurement in policy leads to a change in the meaning of the measures (they no longer are objective barometers, but become politically-loaded success criteria) and people will bend the rules to the maximum to make sure that the measures are hit. Think “bobbies on the beat”; think NHS GPs’ appointments.
So, when I hear “evidence-based policy” from this point forward I’ll translate to “ideological-based policy supported by a skewed balance of certain evidence to make it look credible” and it will make me feel, if not happier, at least a little less angry.