TL;DR

I overindexed on fear of failure as a first-time founder and with hindsight this was a huge contributor to the businesses’ failure.

“Why did Wine List fail?”

It’s been a while since someone asked me this but it came up the other day in conversation.

There’s no singular reason why my old startup Wine List failed.

In the months after I closed that busin…

Read more

First, let me recount a conversation about an ad that took over 3 weeks of editing to go live.

“This ad isn’t going to work” the founder said.

“Well, we think there’s good reason why it could, but let’s test it,” I replied.

“So you’re not even sure”

“No – no-one can be sure, but we believe it could”

“It’s not right how it is, it needs more work.”

There are versions of this kind of conversation everywhere. With advertising and creative, there are long conversations about whether an ad is right or not. But it’s not just in advertising and creative, there are versions of this conversation that take place in every part of business.

There can be debates about which country to launch in, what product line to develop, which feature to introduce next, what software you want to bring into customer support, the list goes on.

Everyone in these environments have roughly the same motivation: they want their thing to be a success. But these debates come down to a different set of opinions on how that will work.

And most importantly separates those who hold probabilistic ways of seeing how businesses grow vs deterministic ones.

Probabilistic vs deterministic ways of thinking

Deterministic thinkers believe that actions cause direct outcomes. They love historic data and believe there’s ways to make sense of the future.

Probabilistic thinkers understand there is huge amount of uncertainty and recognise that there’s a range of possibilities based on actions.

The deterministic thinker in these debates is the one who often advocates strongly for the “right” way to do something. At a certain point something would have happened as a result of an action and therefore it shapes that world view. “Apple has exceptional design, that’s why it works – we must have exceptional design too.”

Startup growth is probabilistic

By the nature of building a startup, you are building something new without historic data. You might be category creating. Or you might be radically improving a category.

Your growth rates should be in 3-figure percent points every year for the first five years. There is no roadmap. There is no playbook. You have to learn everything for yourself. And the world around you is constantly changing.

Building and growing a startup is highly unpredictable.

Ad winners in particular are highly unpredictable

Lots of people will debate me on this, but here’s the things I personally know to be true:

1. We currently launch about 80-100 ad concepts every week

2. The majority of ad launches we deem as fails

3. No-one I have ever met can reliably predict what ad will be a winner

4. Despite this, based on other agency and startup data, we as an agency have a reasonably high “win” rate compared to others.

5. Within individual clients’ accounts, success rates improve over time as we start learning and so therefore

6. You can improve your success based on past data

7. Despite that, there is still a huge amount of randomness

8. Outsized winners – which make up the biggest growth levers – are usually the more unpredictable part of accounts

All of the above is why our creative strategy is first and foremost an experimentation strategy. The most important thing is hypothesis → test → learn, and we should aim to do that as fast as possible, document clearly, and refactor learnings as soon as we can.

We do all of this to increase the probability of finding a successful ad. But you can’t ever guarantee success. All we can do is attempt to increase the probability of success.

The decision then comes down to – at what point is it worth the cost of derisking vs the cost of testing.

The cost of deterministic thinking

I was fortunate that my first ever job was in an agency. In that job I started to see hours as billable units of time. I was a content marketer for that agency and while I wasn’t client-facing, I became astutely aware of the cost of time.

Every minute spent doing something costs money. That time can never be repurchased. There’s no refund on it. You can only spend it that way once.

And it’s not just about the direct cost of the business. Every person in the business is expected to generate a return for that business.

While it’s very easy to consider the cost of cash leaving your bank account to go to Meta, I find that no matter how smart the thinker, it is rare for people to think about the cost and potential return on an hour of time spent.

Customer development is the time well spent

As an agency we experiment all the time. Most are microexperiments – i.e., will this UGC style work for this client this week.

But we also run macro experiments – like “what approach to creative strategy should we take?” This becomes something we might broadly A/B test across our client roster where we’ll run one creative strategy method with half our clients and another with the other half.

These samples are much murkier and not pure experimentation but they constantly refine our process and thinking.

One agency-wide known we have is that the Jobs to be Done framework is highly valuable in creating good ads. We’ve landed on that from trying different frameworks (personas, to minimum viable strategy) and view that time spent learning about customers is valuable time.

Specifically talking to customers, understanding their problems, and how they dealt with those problems before they were customers of the brand in question. This for us is valuable time spent for all involved: if the brand runs the interviews and feeds back to us, or if we run the interviews as we do with our biggest clients.

What’s the value of a member of the team?

These are five of the top 15 fastest growing businesses in the UK according to the Sunday Times:

• Rheal did £19.8m of revenue with ~ 40 employees = £495k per employee

• Trip did £20m of revenue with ~ 176 employees = £113k per employee

• Ancient + Brave did £10.2m revenue with ~67 employees = £152k per employee

• Purdy and Figg did £18.3m revenue with ~69 employees = £265k per employee

• Simmer did £7.4m revenue with 57 employees = £129k per employee

Let’s assume there’s a £45k average salary, then 2.5-11x is a reasonable revenue return per employee’s cost when averaged out.

The costs of debating and testing

There’s a handful of things we need to consider here: the costs, and the potential outcomes.

Let’s run some assumptions.

At Ballpoint, we see ads in three categories: losers, maintainers, and winners.

1. Losers are ads where the CPA is unsustainably higher than the profitability target

2. Maintainers are where the CPA is in line with the profitability target for the current level of scale

3. Rocketship winners are where the is a strong improvement of CPA and therefore potential scale.

What does this look like charted?

In this example, you can see the huge impact that rocketship winners can have. This is real client data where there’s three distinct phases of work:

1. Spend + CPA are correlated - i.e. we’re unable to scale as we’re learning

2. We start to find some light winners that allow us to have bigger spend without a big shift in CPA (maintainers)

3. We then land on rocketship winners which allow us to keep CPA as is, but over triple our daily spend within a very short timeframe.

Now, with this in mind, let’s have a think about what this looks like month-to-month in practice.

Let’s create an example, let’s assume we have an account tracking £50k per month of spend with a £10k per month test budget.

We run 16 creative concept tests per month or £625 of media spend per cost of ad created.

Our gross profit margin on ad spend is 60% (the 2.5:1 LTV:CAC ratio).

Let’s assume we’ve got 42 ads currently live.

Our outcomes are:

• Loser –

  • 55% of the time

  • Profit = £625 x 60% = £375

• Maintainer -

  • 43% of the time

  • Joins the winning ad pool, and for simplicity gets 1/42th of the ad spend = £952/month.

  • Profit in month one = £952 x 60% = £571

  • Assume a lifespan of 2 months

  • Total profit = 2x £571 = £1,142 plus the initial test profit of £375

  • Total profit = £1,517

• Rocketship winners

  • 2% of the time

  • Allows us to increase monthly spend to £75k/month or £60k of hero spend, as we also increase test budget to maintain 20% of spend

  • This now has £1,429 / month of spend per ad (£60k/42 – again simplistic)

  • Lifespan of rocketships is longer, assume 4 months

  • Ongoing profit becomes 4x £857 = £3,428 + initial test of £375 =

  • Total profit = £3,803

  • But it also increases the budget of the entire system, which means we add £12k/month more uplift permanently.

  • That means over 12 months there’s a total of £144k additional profit in our system.

  • Total winner value is now £3,803 for the individual ad + £144k for the whole system =

  • £147,803 profit

What’s the Expected Value of this ad then?

Based purely in the media spend of £625 per ad the EV of an ad test is £3,814.

For those who are unfamiliar, Expected Value is a term borrowed from probability theory. Expected value is the average result you’d expect if you repeated a decision or bet many times — like the long-term payoff of a choice. It’s really big in poker as it helps you understand the different outcomes of results.

It’s something we use increasingly at Ballpoint to help make decisions where it might otherwise look quite tricky.

The cost of debate

Let’s assume some costs here:

• Startup founder:

  • 1x hours of time editing and giving feedback

  • £100k salary

  • Hourly rate of £77

  • 2.5 expected return on time

  • £192 of cost

• Startup Head of Growth

  • 1x hours of time

  • £90k salary

  • £70 hourly rate

  • £175 of cost

• Marketing manager

  • 1x hours of time

  • £50k salary

  • £39 hourly rate

  • £95 of cost

Those 3 people spending 1 hours on editing and feedback creates £462 of extra cost.

Assume 3 scenarios here:

• Scenario A is quick sign-off (maybe just 20minutes of sign off from all parties)

• Scenario B is long sign off = £462 of cost

• Scenario C is huge levels of cost = £1.3k cost

Our EV is £3.8k

Each scenario now nets:

• A = 4.73x return

• B = 3.08x return

• C = 1.3x return

Say the cost of creating and running all your ads for the month is £6k.

That now adjusts the returns to:

• A = 2.86x return

• B = 2.04x return

• C = 0.93x return

Now, the high level of debate means we’re losing money in the whole system – even if you achieve the same rate of success, you’re going to be losing money.

Subscribe now

How to increase our chance of success

But I hear you cry: “if we debate this for longer and increase the time spent on it, we’ll increase the chance of it winning.” This is true, if we doubled the rate of winner, the EV goes positive.

But this is where the issue falls down.

Rocketships are almost impossible to predict

This is where we have to come back to my list of knowns from above.

The experimentation process is very good at improving the ratio between fails and maintainers. I.e. it is good at refining our ‘what’s working’ ads that make up large volumes of day to day performance.

The gains made to probability of finding a rocketship winner however are much lower and harder to quantify. When we’re talking about 1 concept in 100 or 1 concept in 50 being a step changer, the sample sizes aren’t that good at really revealing if we’re able to improve that rate or if it’s just down to chance. (I.e. the actual rocketship probability is 1-5% as a range).

Rocketships are hard to predict. We’ve seen the following be rocketships:

• Ugly UGC mashups

• Straight UGC videos

• Post-it statics

• Review statics

• Heavily branded long form video

• BTS video

• Product shots

I.e. ads. We’ve seen all ad types have the chance of being rocketships.

And from a messaging perspective, we’ve seen rocketships be problem aware or solution aware or product aware. Again, there’s not a reliable framework for the rocketship.

With many of our clients we engage with some light touch “which ad you bet will win?” This is an exercise that almost always elicits losers. At the start its the “unexpected ads” and then people start betting against their gut and then the conventional ad might win the time after.

As I highlighted at the start, some of the best use of time that we’ve found is customer development. Hours spend talking to customers, consolidating that learning, and then having creative strategists play around with that research is the best activity we’ve found for increasing probability of finding maintainers and winners.

The ever-looming runway

In certain environments, I can see the logic for investing more time upfront. Not just in customer development, but also for what the ad visual looks like or how the branding is represented.

I am sure that there is a way to increase the 1-5% chance of finding rocketships to maybe 3-10%, and you can do that by investing more time upfront. The EV of a 10% chance of a rocketship certainly makes the time investment pay off.

But – there is the ever-looming runway.

In my experience the difference between the ‘just test it’ and the ‘let’s get this right’ teams is stark. Because if you’re the deterministic / let’s get this right team, then that will extend to your entire team philosophy. It won’t just be ads that this applies to but every part of how you run your startup.

The cultures that follow deterministic thinking end up taking weeks to make decisions on things like ads. There’s back and forth, then something else comes up. It gets passed on to someone else to edit and amend, delays happen, and so on.

Similarly, the probabilistic team cultures see that permeate the entire business. When things to become discussions and debates, there is usually a decision made quickly to test it. Maybe it’s not a perfect decision, but it’s good enough. And then the founders, the growth leads, the operational leads, whoever it is, move on to the next thing.

The cultures that follow probabilistic thinking end up taking minutes to make decisions on things like ads. Decisions are made, sign offs are quick, ads go live. The test is under way.

Rocketship winners are always needed to run successful businesses. In early days, they’re needed to find product channel fit – without your first rocketship, you’re stuck pushing out ads that don’t quite break even for you or make money for you.

Beyond those early days, it’s how you continually unlock new levels of scale. Most businesses won’t survive on £20k of ad spend a month as the returns won’t hit break-even in your P&L. You need to get to higher levels of ad spend.

Based on a 1-in-50 chance of finding a rocketship, if you can test 4 things per week, you’ll find a rocketship every 3 months.

Let’s imagine though that long debate increased your chance of success by a factor of 2 but at the expense of testing speed and you’re now only launching 4 concepts per month. Your time-to-rocketship is now 6 months. For many startups, by month 6, it will already be the beginning of game over. There’s just not that much runway to correct after this point.

Growing a company is one of the hardest things you can do.

Really killing it with ads is insanely hard. The system that drives ad growth is incredibly complex.

I’ve offered a highly simplified version of what’s at stake here. But I’ve done this to illustrate a way to think about testing over debating. There are lots of ways to run a business. Not every business is investor-backed. Not every business is operating on a model that requires continual growth or requires ads to create the entire sustainable business for them.

If you’ve got a highly successful other business unit (maybe wholesale or other offline dtc) then you can afford to take a different approach to thinking about growth.

But if I was to run a brand where digital DTC or ecom was likely to be the biggest growth lever, then I’d be operating on the probabilistic/test side as much as possible. In my experience, the faster you can get learning through the system, the better your overall outcome.

I write a weekly newsletter focused on startup growth. I currently run an agency where we manage over £15m of ad spend across some of the most exciting new companies around. Before I did this, I was a startup founder of my own brand Wine List, I was the first hire and head of growth at Thriva, and on a fractional based was the most senior growth lead at Hims UK, Fy! and Ferly.

Subjects that excite me: incrementality, customer psychology, measurement, technology.

If you were forwarded this maybe you’d like to read it every week! Give us a follow.

Leave a comment

🔗 When you’re ready, here’s how Ballpoint can help you

→ Profitably grow paid social spend from £30k/m → £300k/m

→ Create full funnel, jobs to be done-focused creative: Meta, TikTok, YouTube

→ Improve your conversion rate with landing pages and fully managed CRO

→ Maximise LTV through strategic retention and CRM - not just sending out your emails

Email me – or visit Ballpoint to find out more.

NB: We support brands spending above £20k/month.

❤️‍🔥 Subscribe to our Substack to learn how to grow yourself

… because agencies aren’t for everyone, but our mission is to help all exciting challenger brands succeed and so we give away learnings, advice, how-tos, and reflections on the industry every week here in Early Stage Growth.

Subscribe now

"**********? That's great for a soft skill, but what's the real thing we're hiring for?"

I was helping a founder hire a head of growth a few years ago and we were discussing key traits needed.

The first thing I said was 'humility', which prompted the founder to smile and then ask what we really were looking for.

But I actually find humility is maybe the mo…

Read more

But here's what everyone misses: while we've been celebrating this model, it's actually killing our ability to adapt to what's coming next.

I used to believe in it too. As a copywriter who could wr…

Read more

The feeling of product-market fit is like switching off zero gravity mode.

Before you have it, there are times when you (the product) bounce against the planet (the market) and you sense what gravity might feel like. But before you know it you’re floating away.

Read more

Midway through my career I worked for a business which was going through multiple successive years of decline. I remember doing analysis to understand whether our share of market was decreasing or the market itself. The answer was both. Less pie, but we were losing at a faster rate. I got a few of the leadership on board with the idea of Wartime.

Ben Ho…

Read more

We both discussed how hard we found acquisition post-lockdown. I can still remember it now. £100+ CPAs for a £40 product…

Read more

For those working in tech – the founders, operators, investors, and suppliers – this has been three days in hell. VCs instructing founders to get money out the bank. The UK arm then reassured dozens of …

Read more

My background was in growth for DTC startups. I’d seen what acquiring 10,000s of customers looked like. I’d seen what failed experiments, cultures, and businesses looked like too. I should have been on good footing.

So what went wrong, and how would I do it differently again?

1. I made too many excuses for PMF

We never found product-market fit — I knew this, but I ignored it.

If there is one hill I will die on when talking to other early-stage founders it’s this one: stop making excuses for not having product-market fit.

Sean Ellis came up with a framework for PMF: ask your customers how disappointed they’d be if you disappeared. If 40% tell you they’d be very disappointed, the framework says you’ve got it. If it’s less than 40%, you don’t.

We ran this process with our entire customer base four times, and with a dozen cohorts on a monthly basis. We never got above 25%.

But I said, ‘we’re acquiring customers at X, once we hit scale, that would mean they are profitable after three months.’ (Excuse)

But I said, ‘our customers are typically older/British, how many of them would ever describe being ‘Very Disappointed’ that a company disappeared’? (Excuse)

But I said, if you look at this one cohort — we do have PMF. (Excuse)

I spent two years making excuses for not having PMF. But I knew deep down we didn’t have it.

How did I know?

2. I was terrified of turning off marketing

Every month I’d write an investor report and there would be some good stat in there. Either revenue or active subscribers were almost always going up. But I was terrified of turning off marketing.

I knew that if I did, churn would be around 20%. Revenue would drop in line.

I was terrified that even though our retention curve was flattening, it wasn’t flat. I was terrified because the margins never seemed to quite line up in practice as they did on paper.

Marketing hid many of these problems, and I was terrified every day of turning it off.

3. I acted like we had a future state of LTV/payback rather than today’s

Wine is an economies of scale business — and at low scale it has sucky margins. First six months, we had negative margins, next year was 10–15%, eventually we got to 35%.

“When we hit 5,000 customers,” I’d tell investors, “we will be able to push those margins to around 60%. And based on that, payback will be three months, and we’ll be at 3:1 LTV:CAC before the end of the year.”

Future states are important. It shows future viability of what scale means.

But I let myself think too much in the future state, and not enough in today’s.

In addition, lockdown meant 75% cheaper CPAs and about 30% better retention. Compound that with the future margin mindset, and today’s business burnt through cash way too quickly.

4. I fell in love with my first startup idea

I always thought of myself as an entrepreneur waiting for an idea. That’s a bad way to come up with startup ideas.

The process was: go through day-to-day life, wait for an idea to hit me, create a solution, experiment and validate, raise money, go.

“Ideas are worthless, execution is everything”

I’d often say this for years — and I retreat on it now. Ideas are important.

David Perell analysed the Simpsons writing process, and they said they rushed to the end of the first draft because they knew it wouldn’t be good. You need to flush out the crap, before you can get to the good stuff. Create an imperfect world, then improve it.

With Wine List, I found a problem and didn’t bother flushing out. I created the imperfect world, and ran with it from there.

Whatever business I start next, I’ll start with dozens of ideas, create success criteria up front, and test those ideas against them.

5. I fell in love with a solution — not a problem

“When I studied wine, my enjoyment of it went up by an order of magnitude,” began each pitch, “but the way I learnt was broken.”

I fell in love with the joy I felt having experienced a solution.

We always struggled to really communicate our problem-solution to our customers in an exciting way.

We found ourselves doing two jobs:

(1) teaching people why they should learn about wine, and

(2) why we would be the best way to learn

Matt Lerner told me years ago, he worries when he hears about startups ‘educating the market’.

In marketing, we know that attention is nanoseconds long. Grabbing it is difficult. Grabbing it and then telling that person they need to adapt their mental model to buy from you? That’s going to be almost impossible.

You see this in different ways: like shitty click-through rates and conversion rates below 1%.

And if you asked 100 wine drinkers ‘do you wish you knew more about wine?’ You’d find almost all of them would say yes. But if you asked if they would shift something in their life to learn, the number would go down a lot.

Sure, people wished they knew more about wine — but we never validated that problem. We validated a solution.

6. I lost my risk tolerance as soon as it became my money

In my life as a growth marketer, I almost always pushed for more budget and more risk.

“How much do you want to spend?” one boss once asked me, “how much do you want to grow?” I responded.

I had a very high risk tolerance.

But when it was my money, and I had to personally answer to my investors, that changed.

Being risk-averse in a startup is dangerous. It stops you making bigger bets like product changes or hiring new people.

I ended up optimising for runway rather than learning.

7. I lied to myself about what was needed to launch a new product

In September 2020, we tested a prototype of a 125ml-sample based ‘wine through the post’ product. It worked great. But we didn’t sell our first box of this until April 2021, why?

I told myself three lies around this product:

1. We need our own warehouse
2. We need to finish fundraising
3. We need to have a full time ops lead in place

We finished fundraising, and put an offer on a warehouse. We started hiring for the person to lead it, we met candidates, went back to the spec, and refined it.

Before we knew it, it was March. Six months after our first prototype and we still hadn’t built this product — or tested it with the market.

I took a holiday and realised that I didn’t need any of this. Getting this live was a series of tasks. I got back, and outsourced product design, logistics, and first build. Three weeks later, we had a product in market.

In May, we acquired over 1,000 customers to By The Glass at a £10 Facebook CPA. Our landing page for this converted at 6%, vs 0.8% for the core product.

I told myself three lies about launching this. I thought we needed conditions to be perfect.

Had we outsourced earlier, we could have had our first products out before Christmas 2020, and our future could have looked a lot different.

Bonus takeaways

Here’s a few final, smaller things which are worth pointing out.

i. Build recreational / celebrating time in with the team more

As a manager, I was great at building recreational time in with the team. As a founder, this slipped by the wayside.

And when I look back now, it’s not the day-to-day stuff that jumps out but the memories of our team together. The Christmas party, the time we went go-karting, the day we went around vineyards, and the warehouse party.

These things matter. They’re fun. The bonding that goes on with them helps cement relationships between colleagues that means when things are tough, they’re that little bit easier.

I wish I did these more, I wish I did them every month. Next time, I’m going to celebrate with the team more.

ii. No-code is great for MVPs, not so good for full products

“Product hacker is the new VP Engineering” was a post I was once going to write. I am glad I didn’t.

No-code is amazing. It means non-technical founders can build entire first products. But, we found as we were trying to build v2 of our site + product, it took just as long with no-code as not, and had far less capabilities.

iii. Continue to over-index on founder-investors

I had over 20 investors in our pre-seed. Half were founders or former founders. When Wine List closed down, one of them — Basubu founder, George Burgess, — called me up to say “well done, you’ve done it now, you’ve had a failed company, and you know it doesn’t matter.”

That was one of the best things I heard in those early days after announcing closure. He’s right. The fear of failure was always a hindrance — to what degree I don’t know, but fear of failure made me make worse decisions in many ways.

I optimised to ‘keep it going’ rather than make faster decisions that would have bigger impacts.

George was right, I’ve done it now, and I’m liberated from that fear again.

Those sorts of insights and feedback felt unique from the founder-investors. Other investors had value in different ways for sure, but the founder-investors were the ones I felt I could most easily turn to for help.

Thanks

I’d like to thank everyone for joining me on that journey. Thank you to the team old and new. Thank you to all of the investors who supported me every step along the way. Thank you to the customers and the community for joining us on it. And thank you to friends and family for putting up with having a founder as a partner, and friend.

For this article in particular, thank you to Matt H. Lerner, George Burgess, and Peter Sigrist for revisions, suggestions, and feedback.

Want to stay in touch?

👉 Sign up to my mailing list

Growth insights from a former founder - Revue

Some final thoughts — edit
Sadly as part of this process, this business ended insolvent and as a result we owed a handful of producers and suppliers money. It’s the real downside to an end like this, and the part of the process that I personally feel most sad towards. If you liked any of the wines we sold, please go and support them by buying their wines wherever you can.