What I learned about measurement from 3 industry veterans (and why you should stop multiplying ROI by 3)
A few weeks ago, we hosted a measurement roundtable at Ballpoint HQ.
We had three speakers:
Rich Cousins – director growth at Thriva, for whom we’ve worked on an MMM and various lift study experiments
Sameer Modha – ex-Google, now helping provide measurement chops to telly ads
Franky Athill – startup veteran CMO, and founder of Attribution Lab.
It was a one-hour panel, no slides, just a great conversation about measurement.
Later this week, I’ll be sharing the Substack podcast + YouTube versions, but until then, here is my roundup from the event.
1. When to start doing MMM
The age-old question. For Rich, Thriva only started focusing about proper measurement when two things happened simultaneously:
They expanded into podcast sponsorships and other upper-funnel activity
Monthly spend hit six figures
Not when they hit a certain revenue milestone. Not when the board asked for it. When channel complexity met spending scale.
Before that? They were fine with platform metrics and a post-purchase survey asking “how did you hear about us?”
(Side note: nobody uses podcast promo codes and so if you’re measuring activity based on that, then STOP RIGHT NOW).
2. How classic MMMs outputs get multiplied up
Sameer shared a great hypothetical anecdotal MMM debrief from his past:
Modelling team: “The MMM shows the ROI is 0.7”
Marketing guy: ”hang on, 0.7, doesn’t that mean I lost money?”
Modelling team: “No, no, don’t worry, we just multiply it by three.”
Marketing guy: “Sorry what, you just multiply it?”
Modelling team: “Yeah yeah, we multiply it by three because that’s the industry norm for the sort of extension to the long term effect.”
While the room laughed, he pointed out that to a certain extent, it’s what most digital marketers do with their LTV:CAC forecasting….
3. Why your attribution model is lying
At Thriva, they tried building an attribution model. It was a blend of click attribution, post-purchase surveys, and other signals.
Rich’s verdict: “It just didn’t feel reflective of reality.”
That’s the test, isn’t it? Not whether your model is sophisticated. Whether it feels true. One takeaway I had from all three was that the data isn’t some magic answer, it’s there to be interpreted.
When your attribution says one thing but your business performance says another, the model’s wrong. Not reality.
4. If you’re spending over £1m, you can probably find a 10% efficiency on that spend
Cost is often one of the big issues, but as Franky points out, ‘if you’re spending over £1m on ads, you can probably find a 10% efficiency on that spend.”
And at the end of the day that’s how we should focus on measurement. Yes there is cost and time to put into it, but if we can find 10% gains on £1m of spend, it’s likely covered itself.
5. There’s no one answer
Franky described the ‘four witnesses’ of measurement: attribution, surveys, modelling, and experiments. (Very similar to our three pillars of the pyramid where we include surveys in modelling).
He says:
“If you want to find out how much growth you drove in a quarter, get your four witnesses in a room, and ask each of them how much growth these things drove.
”They’ll all tell you a different story, and they’ll all be lying to you with different biases.”
”So if you can understand the basics of what their biases are, you can make an informed decision.”
Understand what drove what in your business
If you’re struggling with measurement, then Ballpoint has now built four MMMs, and run dozens of lift-based experiments over the last 12 months. The results have been reshifts in budget, as well as things such as (a) increasing spend to YouTube, (b) investing more heavily in PR & influencer, (c) increasing upper funnel spend on Meta, (d) increasing spend in brand shopping. We can help with your measurement needs as a separate piece of work, even if we’re not managing your spend.
Josh Lachkovic is the founder of Ballpoint, a performance marketing agency that helps brands scale from £1-10M ARR. Visit Ballpoint to learn more, or subscribe to Early Stage Growth for weekly insights on profitable scaling.
Note: Ballpoint works with brands looking to grow their spend from £30k per month to £300k per month.