System Two.
The graveyard

Eight and a half
thousand failures.

Here are sixty-nine of them, run for seven corporates between 2015 and 2021, with the numbers exactly as we recorded them at the time. Most cost about a thousand pounds of media to find out. That is the entire argument.

8.5k

Experiments run

16

Ventures built

8

Exits

£1k

Typical cost each

We count one experiment as one variant put live in its own AdWords cluster. That is about 8,500 tests across roughly 200–300 propositions — around thirty variants per idea. Sixty-nine of those tests are published below in full.

Read on its own, eight and a half thousand failures sounds like incompetence. Read as the top of that row, it is the only reason the eight exits exist. Nothing here was a workshop. Each one had a brand, a live site and real money spent putting real strangers in front of it.

Where a deck’s own conversion percentage disagreed with its own counts — which happens more than we would like — we have recalculated from the counts and said so.

British Gas

Rinsing Sprint #1 · six weeks · 2017

bankingsmart.me

BankingSmart

Borrow money at a fair rate and repay it by topping up your prepayment meter.

170Visits38Sign-ups22%Converted

The best result of the sprint. A second cut, run virally, did 130 visits and 8%.

uai.life

uAi

Let a computer watch the energy market and switch you automatically.

99Visits14Sign-ups14%Converted

Right proposition. Wrong customer. Sign-ups averaged 55 years old; the persona was a couple in their thirties.

kitty.insure

Kitty

Every insurance policy you own, bought and managed from one place.

184Visits11Sign-ups6%Converted

flatinabox.uk

Flat in a Box

One monthly fee and every utility in a new home set up and run for you.

100Visits5Sign-ups5%Converted

EVLife.me

EV Life

Rent an electric car for a week before you commit to buying one.

370Visits7Sign-ups2.1%Converted

Everybody wanted the Tesla.

home-hq.co

Home HQ

Watch every household bill and move you whenever a better deal appears.

234Visits5Sign-ups2.1%Converted

100% men, 60% under 25. Nothing like the household we designed it for.

homeexpert.me

Home Expert

A free, impartial consultant who specifies your smart home and installs it.

694Visits0Sign-ups0%Converted

Six hundred and ninety-four people came. Not one signed up. The team had rated this hypothesis four out of five.

Experian

Financial Stability · Aug–Sept 2018

PensionMine

Find the pensions you have lost track of and show what you pay in each month.

1.8kImpressions52Visits11Sign-ups21%Converted

The best converting idea in the sprint, off the smallest media buy in it.

Thymz — credit score

Build your credit score through your saving behaviour rather than your borrowing.

2.4kImpressions58Visits10Sign-ups17%Converted

Thymz — competitive savings

See what people like you actually save and compete with them.

8.7kImpressions317Visits35Sign-ups11%Converted

Five times the traffic of the credit-score version and two thirds of the conversion. The room preferred this one.

Thymes Royale

Save money by scratching a card, using the urge to gamble as a trojan horse.

3.3kImpressions82Visits8Sign-ups9.8%Converted

The sprint deck records this as eight visits and eight sign-ups, which is impossible. A later deck reusing the same result has 82. We have gone with 82.

Car Subscription

Subscribe to a car, insurance priced in real time, delivered and collected.

49kImpressions132Visits10Sign-ups8%Converted

The biggest media spend in the sprint bought the thinnest traffic. Five of the ten were buyers.

Flequity

Release equity from your house at 48, when you are too old for a mortgage and too young for equity release.

6.7kImpressions119Visits8Sign-ups6.7%Converted

ManukaMoney — zero hours

Pay me the same wage every week and hold the surplus back for the quiet ones.

6.2kImpressions207Visits14Clicks5%Converted

Identical proposition to the one below. Only the customer changed.

ManukaMoney — students

Take my whole student loan and drip-feed it to me across the year.

7.3kImpressions160Visits7Clicks4%Converted

Identical proposition to the one above. Only the customer changed.

Freddie the form filler

Pull the real numbers out of your bank and HMRC and fill your mortgage form in.

5.5kImpressions108Visits4Sign-ups3.7%Converted

Career Expert

Codify leadership and sell a believable, step-by-step route to it.

11.6kImpressions156Visits5Sign-ups3%Converted

ImpatientData

Get your NHS records, anonymise them and sell them to researchers on your behalf.

16.5kImpressions157Visits4Sign-ups2.5%Converted

The second-biggest media buy in the sprint and the worst result in it.

Aviva

Post-Covid Wellness · six weeks · 2021

hands-off.co

Hands Off

An AI that works out what you can spend this month and handles everything else behind the scenes.

228Clicks10Sign-ups4.4%Converted

Best result of the sprint. Sign-ups spanned 18 to 65+. The ads that pulled were about budgeting; the ad that converted was ‘make your pension last longer’.

thegood-place.com

The Good Place

A loan at a fair rate that locks you in for longer and forces you to build savings while you repay it.

53Clicks5Sign-ups9.4%Converted

Google refused the ads, so it ran through a personal-finance newsletter instead. Everyone asked for the £5k maximum, which is what happens when £5k is the top of your drop-down.

fomoco.io

Fomoco

A pension that puts a slice of your money into crypto alongside the usual funds.

49Clicks18Sign-ups36.7%Converted

The strongest signal in the sprint and the least trustworthy. Crypto ads are banned on Facebook, Google and LinkedIn, so it ran to a crypto newsletter list — an audience already sold on the idea.

rebelpensions.com

Rebel Pensions

A pension where every stock is picked on ethical criteria and the holders vote on how it is used.

9Clicks2Sign-ups

Nine clicks. Run to the same newsletter as the crypto test a week later, and the list was spent. Two sign-ups is not a result, it is a rounding error.

join-sarah.com

Sarah

A free digital adviser that sees all your money and debt and tells you what to do next.

232Clicks5Sign-ups2.2%Converted

Sign-ups ranged from 21 to 58 and wanted three different things — retirement, debt, a house deposit. Nothing to aim at.

healthprocheck.com

HealthPro Check

A health assistant that remembers your symptoms and tells you whether to worry.

315Clicks5Sign-ups1.6%Converted

79% of sign-ups were women. The winning ad was ‘free health assessment’, which is the offer converting, not the proposition.

retireright.uk

RetireRight

Expert guidance through the whole transition into retirement, not just the money.

284Clicks4Sign-ups1.4%Converted

Sign-ups clustered at 55–64, exactly as designed, and still did not want it. It was also being offered free.

moodscore.co

Moodscore

Software that reads camera, voice and keyboard data to score your team’s mental health.

221Clicks0Sign-ups0%Converted

Two hundred and twenty-one HR professionals clicked. None signed up. One of them commented on the ad: ‘This looks good but why don’t you just speak to people instead.’

Theratu

Instant paid access to a child mental-health specialist for a parent who cannot wait for a referral.

StoppedBefore it ran

The one we stopped. The demand was for acute intervention, not prevention, and we were not prepared to advertise a service to desperate parents that did not exist. Written up in the deck as a decision, not a result.

GSK

Oral Health · six weeks · 2019

thedentistwillseeyounow.com

The Dentist Will See You Now

Find and book a dentist the way you book a car — whoever is free, whenever you need one.

1.5kImpressions4.6%Clicked31%Converted

The highest converting experiment of the entire six weeks, and it ran for twenty-four hours on about seventy clicks. The commonest reason given for booking was immediate pain. The US version did 20% off ten times the impressions.

smilecrew.co.uk

Smile Crew

Good dentists and hygienists with an easy booking flow, for people who cannot get an NHS appointment.

11kImpressions605Clicks32Sign-ups5.3%Converted

The strongest read of the sprint on a proper base. The catch, written in the deck at the time: most of the interest came from people who had ignored their teeth until something went wrong.

brushwell.org

BrushWell

A premium electric toothbrush on a monthly plan, discounted the more you actually brush.

11kImpressions305Clicks11Sign-ups3.6%Converted

‘Cheap’ and the word Philips drove the conversions. We were testing someone else’s brand equity. The US version, which discounted dental insurance instead of the brush, did 6.4%.

brushsquad.org

BrushSquad Beta

The rewards toothbrush again, with the price taken off the page and a beta sign-up instead.

14.6kImpressions583Clicks21Sign-ups3.6%Converted

Removing the price more than doubled conversion against the paid version below. Which tells you what people will sign up for, not what they will buy.

brushsquad.org

BrushSquad

A toothbrush and app that lets parents set rewards so a reluctant child brushes for two minutes.

10.5kImpressions262Clicks4Sign-ups1.5%Converted

Forty-eight hours. Sign-ups were parents of children averaging 5.8 years old, weighted male, nearly half of them brushing less than once a day. Conversion was strongest in the poorest postcodes.

prescribewhite.com

Prescribe White

Prescription-strength whitening on subscription, with a professional signing it off.

7kImpressions359Clicks11Sign-ups3.1%Converted

The best-converting keywords all contained the word ‘best’. The US version did 2.6%. Whether people wanted the oversight or just the strong stuff, the test could not tell us.

toothbooths.com

Tooth Booths

An intra-oral scan in a booth on the high street, no dentist and no appointment.

26kImpressions1,070Clicks18Sign-ups1.7%Converted

Twenty-five of the twenty-six thousand impressions came from the ‘embarrassed about my teeth’ ad group. The other two angles — convenience and being busy — drew about a thousand between them.

jen-dental.com

Jen

An always-on coach, part AI and part human, that tells you whether you are brushing properly.

27.5kImpressions824Clicks10Sign-ups1.2%Converted

The wedge we actually wanted to test — ongoing coaching — produced zero sign-ups. Every click and every conversion came from the adjacent ads about booking a dentist.

toothtruths.com

Tooth Truths

A chatbot that answers the questions you are too embarrassed to ask a dentist, at 3am.

15.5kImpressions754Clicks10Sign-ups1.3%Converted

People googling precise medical terms clicked hardest. Cosmetic searches brought the volume and converted worst — so we rebuilt it around cosmetics, and it got worse still: 468 clicks, 4 sign-ups.

sonarbrush.com

Sonar

A brush that maps all sixteen zones of your mouth and shows you the ones you always miss.

5.5kImpressions258Clicks1Sign-up0.4%Converted

Killed in twenty-four hours. Enormous click-through on ‘best electric toothbrush’ and almost nothing on the other side of it. High intent to buy a toothbrush is not intent to be coached by one.

Ten further entry points, run as variants and in the US market

  • Entry pointClickedConverted
  • The Dentist Will See You Now — US1.2%20.0%
  • Tooth Booths — US2.2%4.0%
  • BrushWell — US (insurance discount)1.2%6.4%
  • BrushSquad Beta — US3.6%3.1%
  • Prescribe White — US3.3%2.6%
  • Jen — umbrella of services3.7%2.2%
  • Jen — umbrella, US1.7%1.7%
  • Jen — tooth rescue package3.4%1.2%
  • Tooth Truths — cosmetics, US3.6%1.1%
  • Tooth Truths — cosmetics6.4%0.9%

GSK

Resilience and Sleep · six weeks · 2020

Check Best Dose — melatonin

A site that tells you what dose of melatonin actually worked for people like you.

1,280Impressions125Clicks55Sign-ups44%Converted

The best result of the sprint on both measures, off the smallest media buy in it. Nine point eight per cent of everyone who saw the ad clicked. A lot of the searches were parents asking about dosing children.

Check Best Dose — CBD

The same idea for CBD oil: what dose has worked for other people with your problem.

3,329Impressions101Clicks39Sign-ups38.6%Converted

Skewed to new users over fifty, and about half offered to contribute their own data. People were taking dosage advice from Facebook groups because doctors are not allowed to give it.

Check Best Dose — tracker

A tracker that watches your sleep and side-effects while you are on CBD and adjusts the dose.

10,355Impressions275Clicks27Sign-ups9.8%Converted

Skewed hard to women over fifty. Third of three CBD tests and the weakest, which is the direction you want: the further from the raw question, the less anyone cared.

Adaptify

A supplement that wakes you up on a shift and a matching one that puts you back down.

9,539Impressions223Clicks11Sign-ups4.9%Converted

The only price-carrying test in the sprint that converted properly, at £20 a month.

Sleep Bootcamp — at home

An intensive sleep retraining programme that comes to your house.

33,810Impressions736Clicks9Sign-ups1.2%Converted

Tested at $300. Beat the identical programme in a clinic at $600 on volume, and matched it on rate. People would rather have the expert come to them.

Sleep Bootcamp — clinic

The same forty-eight hour sleep retraining programme, in a clinic.

11,974Impressions331Clicks4Sign-ups1.2%Converted

At $600 this converted four times better than the $50 online course below. Expensive and quick beats cheap and slow.

Slumber School

An online CBT-for-insomnia course with a proven clinical track record, at $50.

49,864Impressions1,150Clicks4Sign-ups0.3%Converted

The largest media buy in the sprint and the second-worst result in it. CBTi has an 85% effectiveness rate. Nobody wanted to buy it.

Sleep Consultant

An hour with an expert who works out why you cannot sleep and points you at the fix.

38,498Impressions843Clicks6Sign-ups0.7%Converted

Tested at $100.

MigraSleep

A mask that blocks light and sound and stimulates the trigeminal nerve so you can nap through a migraine.

44,694Impressions1,177Clicks6Sign-ups0.5%Converted

Forty-four thousand impressions at $200 for six sign-ups. The ads were competing with painkillers, and painkillers cost four pounds.

Sleep PA

A service that reshapes your calendar around protecting your sleep.

25,398Impressions442Clicks4Sign-ups0.9%Converted

Aimed at optimisers. ‘Burn out’ turned out to be a stronger keyword than ‘sleep’, which is a different business.

Track to Sleep — ADHD

A wearable that reads how stressed your child has been today and prescribes tonight’s bedtime routine.

12,117Impressions110Clicks3Sign-ups2.7%Converted

The lowest click-through of the sprint. ADHD and ASD keywords performed identically, so it was not an ADHD problem at all.

Track to Sleep — general

The same tracker sold to any parent rather than to parents of neurodivergent children.

24,052Impressions368Clicks6Sign-ups1.6%Converted

Twice the traffic of the targeted version and a lower rate. Free to join, but you had to already own a Fitbit.

Shooft — jet lag

A planner that reorganises eating, sleeping and exercise around long-haul travel.

6,260Impressions234Clicks6Sign-ups2.6%Converted

Ran into March 2020, when long-haul travel stopped existing. The team flagged the result as unusable at the time.

Shooft — shift work

The same planner built around a shift rota.

21,279Impressions268Clicks3Sign-ups1.1%Converted

Every single sign-up was a woman. Scheduling is a harder sell than medication.

Smart Sleep Sounds

A device that hears your child stirring and plays a sound to settle them before they wake.

7.5%Clicked1.9%Converted0%At $99

Second-highest click-through of the sprint as a free hardware beta. Put a $99 price on the same page and it recorded nothing at all.

Sleep Smart — scent

The same device, releasing calming essential oils instead of sound.

3.9%Clicked2.3%Converted0%At $99

Same collapse at $99. Two experiments, one variable, and the cleanest price read in the archive.

Barclays

Rental Market · one week · 2016

Fair Flat Share

One shared-household bills proposition, six different reasons to want it, all live at once: predicting the bills, splitting them in a click, paying each other, spotting anomalies, closing accounts at the end of a tenancy and a flat rate that settles up at the end.

1,535Users56Sign-ups3.6%Converted1Week

The wedge that pulled 56% of all the traffic converted worst of the six. The one that converted best got 69 users. If you had run any single one of these on its own you would have drawn the opposite conclusion — which is the entire reason we run them together.

Different Life

Life insurance smoke tests · 2021

Lucky Life

Life insurance with a prize draw attached, for people who will not buy life insurance.

6,324Impressions103Clicks12Completed0Paid

Three runs. Twelve people filled the form in, three of them were over the age limit, one was the same person twice, and not one of them paid. Cost per completed form: £108.

RainCoat

Straight life insurance, sold three different ways to find out which audience would bite.

5,478Impressions85Clicks2Completed£528Spent

The first cut drew 60% over-44s and nobody finished. The second was worse and cost £6.63 a click because the ads were catching people searching for ‘Aviva login’. Tightening the keywords got two forms at £17.61 a click.

eBay

Human-assisted search · three MVPs · 2015

eBay Elves

Eight people finding Christmas presents for strangers who were asking Twitter for ideas.

1,501Links sent8,325Clicks£1.5kPer day

Every link sent was clicked five and a half times. The third and best of three tests of the same idea: people want a human being to help them search, and they will forward the answer to their friends.

Fast Fashion

A fashion journalist posting real-time collections built entirely out of things already for sale on eBay.

3,300Readers1,503Clicks to eBay45%Converted

Target was 30%. Pages per session went from one to over three and repeat visits climbed week on week. Cost about £4 a click, which was the problem.

Team Want

A specialist you could talk to on Twitter who would go and find the furniture you were describing.

630Links sent440Clicks£3Per click

Twenty per cent of people favourited the reply and individual handles started coming back with new requests. Expensive per head, and the whole thing ran on eight people and a spreadsheet.

What this cost

Sixty-nine answers, and not one of them took longer than a week.

Eight sprints across six years. Every one of these had a brand, a live site, a media budget and real strangers arriving on it. A handful were worth continuing with. The rest were killed inside a week, and the killing is the point — each one would otherwise have been argued about in a boardroom for a year and then half-built.

The alternative is not cheaper. It is the same money spent finding out later.

This is a snapshot. There are eight and a half thousand more.