I’m doing a wee bit of research at the moment into another Ignite presentation at the end of the month (more details, and opportunity to sign up here). I’m going to cover off a favourite topic of mine – the madness of measurement; basically a piece of controlled moaning.
The former I came across in David Boyle’s book The Human Element. Charles Goodhart is an economist from the London School of Economics, and a former member of the Bank of England’s Monetary Policy Committee.
The law named after him was first noted in a paper he published in 1975, and states:
“that once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play that role.”
In other words, if you pick a metric to be a target for some sort of change you are trying to implement, then the very act of making that metric the basis for success or failure removes the validity of that metric because, effectively, it’s become too “loaded”.
Campbell’s law, coined by American social scientist Donald Campbell at a similar time to Goodhart’s, states:
“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
or if you pick a metric to measure success, then people will “game” it to hit the target.
Whilst both of these laws have their origins in social, state-provided services, I don’t see any reason why they don’t equally apply in most organisational contexts – and this comes to another problem that is long running in the field of social science: how the act of observation has impact upon what you are observing.
For marketers, particularly those who are involved with data-intensive activities in the digital realm, these two laws combined lead to an interesting potential outcome: you are likely to achieve the targets for things that you measure, but in doing so the value of the thing that you are measuring will be eroded away to not be what you thought it was.
An example: online voting for “best of” type competitions – you know the thing, “The Best X of the Year” type affairs. Think how often on social networks you see messages along the lines of “we’ve been nominated for the best X of the year – vote for us now at…”. So the metric (number of votes) being used as an assessment of how good a particular thing is actually becomes an assessment of how good the nominees (usually after they’ve paid their registration fee) are at drumming up support online. Which isn’t quite the same as the original intent at all.
The solution? Well, as I’ve argued before, is to keep objectives separate from the measures used to track progress towards success of those objectives, and make sure that you always keep focus on the thing you’re trying to achieve, not the measures in place.
UPDATE: just as I published this, I received a tweet that gives another perfect illustration: