Goodhart’s Law as Applied to Marketing

It’s an old saying that crops up whenever people talk about metrics: “You get what you measure.”

One corollary to that statement is “be careful what you measure.” Why? Because as anyone who has worked in marketing for any length of time can attest, target metrics can rapidly take on a life of their own.

As marketers on the team start to orient campaigns and programs around the target metrics, there are two enormous and all-to-common risks:

1. The metric takes on a meaning of its own, divorced from its actual value;

It turn’s out there’s a handy “law” that applies, called Goodhart’s, found via Boing Boing, which explains both of these clearly:

“…once you start measuring GDP as a way of gauging social welfare, people will start to figure out ways to make GDP go up without improving social welfare (say, by swapping dirty financial derivatives). Once Google starts measuring inbound links as a way of evaluating the importance of web-pages, people will figure out how to increase the inbound links to unimportant pages (splogging, blogspam). And once you measure fat or calorie content as a proxy for the healthfulness of food, manufacturers will figure out how to decrease fat and calories without making the food more healthful (reducing fat by adding sugar, reducing calories by adding poisonous artificial sweeteners).”

How exactly does this apply to marketing, and social media marketing in particular?

A quick story from personal experience to illustrate:
Some years back I worked with a marketing team that was striving to make some sense of the chaotic mass of campaigns, promotions, and marcom channels churning through the organization on a regular basis. Like many orgs, they sought to standardize on a small but (supposedly) highly relevant set of goals that they felt, drawing upon research, would most effectively drive the business. That metric was “contactable profiles” – gained, for example, when a prospective customer would fill out a profile to get access to a whitepaper, and opt in for further communication from the company.

Not unreasonable as an activity goal for a marketing team, especially back in the pre-social media days when email marketing ruled. A qualified contact, who explicitly opted in to hearing more about your products and services, is always a big deal and can be an enormous marketing asset.

The problem however quickly became apparent, and it’s a case study in Goodhart’s Law. As that one metric gained prominence in reviews and scorecards, it rapidly took on a life of it’s own. Campaigns and programs that had no business collecting personal profile data – they provided no clear value to the user in exchange for it – started to incorporate it as the primary success metric. Everyone wants to show their campaigns are contributing to the business, right?

Profile data started to get collected for the sole reason of showing how your campaign was “performing.” Obviously, contactable profiles are only valuable if there’s a clear follow up plan – what are you going to DO with that data. Also, as this was before the days of Facebook Connect and single sign-on, customers quickly grew frustrated at being hit up again and again for personal profile data at every interaction they had with the company, and in situations where they saw no clear “get” for that “give.”

Take this example back to social media marketing.
All too often we hear of campaigns which have a primary goal of “building fans and followers,” a metric that is valueless when taken out of context yet has taken on a life of its own in social media circles. The result is enormous wasted effort from brands and agencies alike who churn promotions out to get more fans or followers, and once they get them a) they have no clear plan for how to leverage that engagement for real business value, and b) their new fans/followers end up seeing little value in staying engaged long term, and the brand sees high churn and disaffected followers.

Before you embark on a marketing or social media campaign, when the success metrics are being tossed around fast and furious, ask your team a few simple questions:

 
• How does this metric connect back to actual business goals – not just marketing goals?
• How can we build in some “reality check” milestones to ensure we haven’t let the metric get out of control?
• If it’s a metric that involves a customer or use action, what clear and reasonable value are they seeing by taking it?

Can you think of any other example when marketing or social media metrics took on a life of their own and went down the Goodhart’s Law rabbit hole?

*photo by kayaker1204 on Flickr, via CC license.

No related posts.