Marketing science, selective evidence and the business of proving ourselves right.
There it is!!! The moment you find a piece of research that confirms something you’re desperate to move from subjective belief to objective fact. Even better when it comes with an impressive-looking chart that makes it so obviously understandable for the most data illiterate.
Suddenly, that opinion you’ve been defending in meetings is no longer just an opinion. It’s evidence-based thinking. There’s a percentage attached to it, perhaps a confidence interval, and ideally a quote from someone important enough to have spoken at Cannes.
You can put it in a presentation. You can reference it in a strategy document. And if anyone disagrees, well, they’re not disagreeing with you anymore. They’re disagreeing with science.
You bloody geek, you.
You have to wonder whether marketers are interested in being led by the data or in finding data that agrees with them.
The conclusion came first
Marketing has become very enthusiastic about science. We talk about evidence-based decision-making, behavioural science, statistical significance and the importance of understanding what really drives consumer behaviour.
And that’s a good thing.
For an industry that has historically been quite comfortable making enormous decisions based on instinct, experience and the opinion of whoever has the most senior job title, a little scientific rigour is probably overdue.
But how often do we approach the evidence with a genuinely open mind? Because there’s a familiar pattern emerging.
Someone has an idea they believe in. They might have years of experience supporting it, a strong personal preference or a very convincing argument for their particular doctrine. The idea gathers momentum, people become invested – but where is the evidence?
They delve into the research but not research to find out whether it’s the right recommendation.
Research to support it.
It’s a subtle distinction, but a fairly important one. By the time the evidence arrives, the decision has already acquired a certain emotional significance. People have spent time developing it, defending it and perhaps persuading others to get behind it.
Discovering that it might be wrong is no longer a useful learning opportunity. It’s an inconvenience to the agenda.
A chart for every occasion
The good thing about marketing research is that there’s an awful lot of it. Whatever your preferred philosophy, there’s a report somewhere that appears to support it.
If you believe brands should invest more in creativity, you’ll find plenty of evidence to make that case. If you believe marketing needs to be more accountable for immediate commercial results, there are reports for that too.
You can find research supporting emotional advertising, rational messaging, distinctive assets, personalisation, simplicity, consistency and almost any other concept that has enjoyed a period of popularity.
Some of it is excellent. Some of it is less convincing. And some of it seems to have been designed primarily to give the company publishing it something interesting to talk about at its next conference.
The challenge isn’t necessarily a lack of evidence. It’s knowing what the evidence actually tells us.
A study might reveal a relationship between two things without establishing that one caused the other. It might examine a particular type of business, in a particular market, under particular conditions. Or it might rely on information supplied by people with a fairly strong incentive to report that things went well.
None of this automatically makes the research worthless. But it does mean we should probably read a little further than the headline before declaring that it has settled the argument.
A graph can make an opinion look remarkably like a fact. Especially when nobody asks what went into it.
The small print is where the fun starts
The way we talk about marketing research is almost as interesting as the research itself.
A report is published, someone extracts the most compelling statistic and within days it becomes an established industry truth. The original findings might have been carefully qualified. There may have been limitations, caveats and important details about the sample. But caveats don’t travel particularly well on LinkedIn.
“Under these specific conditions, we observed a relationship that may warrant further investigation” doesn’t have quite the same appeal as “SCIENCE PROVES MARKETERS HAVE BEEN DOING IT WRONG.”
And so a nuanced finding gradually becomes a rule. A rule becomes a framework. A framework becomes a keynote presentation. And before long, people are using it to explain what every business should be doing, regardless of whether those businesses have much in common with the ones originally studied.
We’ve managed to turn research, which should help us ask better questions, into something that occasionally discourages questions altogether.
What happens when the data disagrees?
It’s relatively easy to champion evidence-based marketing when the evidence supports what you wanted to do anyway. But what happens when it doesn’t?
What happens when research suggests that the proposition you’ve spent months developing isn’t particularly compelling? Or when a campaign you personally love produces disappointing results?
Perhaps the data suggests that the less exciting creative route is more effective, or that customers value something entirely different from what the marketing team committed to.
That’s when being data-led becomes considerably less comfortable. Because the real test isn’t whether we’re willing to quote research in support of our decisions. It’s whether we’re willing to change our decisions because of the research.
Of course, data can be misleading. Research can be poorly designed, measurements can be incomplete and short-term results don’t always capture longer-term effects.
Questioning a finding is perfectly reasonable. But I suspect we’re sometimes more rigorous in questioning the evidence that contradicts us than the evidence that confirms what we already believe. A favourable result is a valuable insight. An unfavourable result requires further investigation. Preferably until we get a more favourable result.
Science isn’t supposed to make us feel clever
One of the things I find appealing about genuine scientific thinking is that it requires a certain amount of humility. You begin with an idea about how something works, examine the evidence and accept that you might be wrong. Sometimes you discover something useful. Sometimes you discover that your original assumption was nonsense.
Marketing, however, doesn’t always make being wrong feel particularly rewarding. We like conviction. We like decisive leadership. We like people who can stand in front of a room and explain, with absolute confidence, why their strategy is the right one.
We want the credibility that comes with science without always embracing the uncertainty that makes scientific thinking useful in the first place.
I’m not arguing we should ignore the research
I’m not suggesting we abandon marketing science. Quite the opposite.
Good research can challenge assumptions, reveal patterns we’d otherwise miss and help us make better decisions. Understanding what has worked across different brands and categories is valuable, particularly when it stops us repeatedly making the same expensive mistakes.
And not every business has the resources to run sophisticated experiments or commission its own research. Learning from the work of others is sensible.
But there’s a difference between using research to inform our judgement and using it to replace our judgement. That’s where the thinking begins, not where it ends.
The most useful question we can ask when confronted with an impressive piece of marketing research isn’t “How can we apply this?” It’s “How confident should we be that this applies?”
Less exciting, admittedly.
The most valuable data might be the data we don’t like
The marketers who get the most from evidence aren’t the ones who can quote the greatest number of studies. They’re the ones who are prepared to be surprised.
The ones who can look at an unexpected result without immediately trying to explain it away. Who can distinguish between a useful general principle and a universal rule. Who are curious enough to investigate why something didn’t work rather than simply finding a more agreeable metric.
That requires a different kind of confidence. Not the confidence of always having the answer, but the confidence to discover that your answer might be wrong.
Which sounds straightforward enough.
Until you’ve already presented the answer to the board.
And another thing…
The next time we find a piece of research that perfectly supports something we’ve been arguing for years, perhaps we should ask ourselves an uncomfortable question.
Would I be quite so convinced by this research if it told me the opposite?
Because being evidence-based is easy when the evidence agrees with you.
It’s what you do when it doesn’t that counts.
Thanks for reading. And Another Thing… is a weekly look at the strange things marketers believe, the peculiar things we do, and the uncomfortable questions we probably ought to ask.
A bit of curiosity, the occasional unpopular opinion, and hopefully something worth thinking about.
If you enjoyed this, subscribe for more. And if you disagreed, even better. That’s what the comments are for.
