Campbell’s law, Goodhart’s law, and the trouble with observation

I’m doing a wee bit of research at the moment into another Ignite presentation at the end of the month (more details, and opportunity to sign up here). I’m going to cover off a favourite topic of mine – the madness of measurement; basically a piece of controlled moaning.

What’s cropped up has been a couple of “laws”; Goodhart’s law and Campbell’s law.

The former I came across in David Boyle’s book The Human Element. Charles Goodhart is an economist from the London School of Economics, and a former member of the Bank of England’s Monetary Policy Committee.

The law named after him was first noted in a paper he published in 1975, and states:

“that once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play that role.”

In other words, if you pick a metric to be a target for some sort of change you are trying to implement, then the very act of making that metric the basis for success or failure removes the validity of that metric because, effectively, it’s become too “loaded”.

Campbell’s law, coined by American social scientist Donald Campbell at a similar time to Goodhart’s, states:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

or if you pick a metric to measure success, then people will “game” it to hit the target.

Whilst both of these laws have their origins in social, state-provided services, I don’t see any reason why they don’t equally apply in most organisational contexts – and this comes to another problem that is long running in the field of social science: how the act of observation has impact upon what you are observing.

For marketers, particularly those who are involved with data-intensive activities in the digital realm, these two laws combined lead to an interesting potential outcome: you are likely to achieve the targets for things that you measure, but in doing so the value of the thing that you are measuring will be eroded away to not be what you thought it was.

An example: online voting for “best of” type competitions – you know the thing, “The Best X of the Year” type affairs. Think how often on social networks you see messages along the lines of “we’ve been nominated for the best X of the year – vote for us now at…”. So the metric (number of votes) being used as an assessment of how good a particular thing is actually becomes an assessment of how good the nominees (usually after they’ve paid their registration fee) are at drumming up support online. Which isn’t quite the same as the original intent at all.

The solution? Well, as I’ve argued before, is to keep objectives separate from the measures used to track progress towards success of those objectives, and make sure that you always keep focus on the thing you’re trying to achieve, not the measures in place.

UPDATE: just as I published this, I received a tweet that gives another perfect illustration:

“Hey @ballantine70 I will give you 100 Twitter Retweets from 100 unique profiles.”

15 thoughts on “Campbell’s law, Goodhart’s law, and the trouble with observation

  1. The same ‘followers’ issue hit me today, I noticed a tweet from a company that piqued my interest, as they purported to be working with us in the prison sector, @SecurityInst. 8000+ followers seemed a lot for a company with 3 tweets to their name… I don’t know which particular scheme sent these followers to that profile, but what a waste of time! And, when they get their social marketing into gear, counter-productive! It doesn’t take a lot of thinking about but you know that someone, somewhere is making money from this sort of thing. I just wish it was me…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.