Incrementality testing: the playbook for replacing last-click attribution

0:00Welcome to the Leaders Insights podcast.

0:08Today's episode, Incrementality Testing, the playbook for replacing last-click attribution.

0:13Let's start with a provocation.

0:15You've said last-click attribution has been lying to marketers for years.

0:19That's a strong accusation against a tool half the industry still swears by.

0:23It's not even a controversial one anymore.

0:26Crediting the perfect attribution, the model that gives 100% of the credit to whatever

0:30ad someone touched right before they bought, is like crediting the person who opened the

0:35door for winning the marathon.

0:41They did.

0:42But the question isn't did they click.

0:44It's would they have bought anyway?

0:47A huge share of last-click conversions are people who were already coming to you.

0:51They searched your brand name, clicked your own ad, and bought.

0:55You just paid to intercept a sale you already had.

0:58So you're paying a toll on your own bridge?

1:00Exactly that.

1:02And it gets worse.

1:04Since 2021, privacy changes.

1:06Apple killing the identifiers apps use to track you.

1:09Browsers dropping third-party cookies mean you can't even see the full click path

1:13anymore.

1:15So last-click isn't just biased.

1:17It's now guessing with half the data missing.

1:20Fine.

1:21So what replaces it?

1:22You're pushing incrementality testing.

1:24Sign it before you use it.

1:26Incrementality is the extra sales your ad actually caused.

1:30The ones that wouldn't have happened without it.

1:32An incrementality test measures that directly by holding some people back from seeing the

1:37ad and comparing.

1:39That sounds like a science experiment, not a marketing plan.

1:43It is a science experiment, and that's the point.

1:46You take your audience, you split it.

1:48One group, the test group, sees the ads.

1:51The other, the control group, doesn't.

1:54Then you look at who bought in each.

1:56The difference is your true lift.

1:58Give me a concrete version.

2:00What does that look like in a real channel?

2:02Take META, Facebook, and Instagram.

2:05They offer what they call a conversion lift test, where they randomly withhold ads from

2:09a slice of your audience.

2:11Say the test group converts at 4% and the control at 3%.

2:16That one point is your incremental lift.

2:19Everything above the 3% baseline is what your spend actually bought.

2:23And that vendor number, META running the test on META's own platform, you trust it?

2:28I trust it the way I trust a restaurant reviewing its own food.

2:31It's a starting point, not gospel.

2:34The cleaner method is a geotest.

2:36You turn ads on in some cities, and often comparable ones.

2:40Portland gets the campaign, Sacramento doesn't.

2:43You compare sales.

2:44Nobody's grading their own homework there.

2:47What kind of gap are we talking about between what last click claims and what's real?

2:51When I ran this at a retail brand, our branded search, ads on our own name, showed a stellar

2:56last click return.

2:58Incrementality testing said roughly 80% of those buyers would have found us anyway.

3:04We cut that spend by a third, and revenue didn't move.

3:08That's a third of a budget line doing nothing.

3:10That's the part that stings.

3:12You're admitting you were lighting money on fire.

3:15Everyone is.

3:16They just haven't run the test to prove it.

3:18The comfortable dashboard number feels great.

3:21Incrementality is the friend who tells you the truth at the wrong moment.

3:25So how does an exec actually start without hiring a team of statisticians?

3:29Start with your single biggest channel and your most suspicious line item.

3:34Almost always branded search or retargeting.

3:36Ads chasing people who already visited your site.

3:40Those are where the lying is worst.

3:42Why those two specifically?

3:43Because both target people already leaning toward you.

3:47Retargeting shows ads to someone who was about to buy anyway.

3:51The test tells you how much you're paying to nudge a decision already made.

3:55And when the number comes back ugly?

3:57You reallocate.

3:59Move the money to a channel where the lift is real and let it run.

4:02That's the whole loop.

4:04Test, cut the fake wins, feed the real ones.

4:07One thing they can do Monday morning?

4:09Pick your branded search budget, run a two-week geo holdout, on and half your markets, off

4:15and the other half, and compare total sales.

4:18If nothing drops, you just found your first real savings.

4:21Done.

4:22Want to know where you actually stand?

4:24Take the CMO self-assessment at MBA-training.com.