.jpeg)
Some customers who see your ads were going to buy anyway. They already knew your brand, had something in their cart, or were about to search for you. Your ad platform will happily take credit for those sales. Incrementality strips them out and asks a harder question: What did this ad actually add to our total sales?
That question matters more every year. Budgets face more scrutiny, finance teams want proof that spend drives revenue, and platform-reported numbers often run higher than the real impact. Incrementality is how you separate the sales your marketing caused from the sales it was simply near.
Incrementality vs. attribution
Attribution and incrementality answer different questions, and you need both.
Attribution
- Question it answers: Which touchpoints came before a sale?
- Method: Tracks clicks and views on the path to purchase.
- Shows: Correlation.
- Speed: Daily, campaign and creative level.
- Best for: Day-to-day optimization.
Incrementality
- Question it answers: Which sales would not have happened without the ad?
- Method: Compares an exposed group with a holdout group.
- Shows: Causation.
- Speed: Weeks per test, channel level.
- Best for: Validating whether a channel works and how much to trust attributed numbers.
Think of it like basketball stats. Attribution is the box score: it tracks who touched the ball on every possession that ended in points. Incrementality is plus-minus: it compares how the team scores with a player on the court versus off it. Used together, the box score tells you what happened every night, and plus-minus tells you which players actually impact games, so you can weight the daily stats accordingly.
How incrementality is measured
Every incrementality test follows the same logic as a clinical trial: show the "treatment" (your ads) to one group, withhold it from a comparable group, and measure the difference.
- Pick the channel or campaign you want to evaluate and the KPI that matters (revenue, orders, new customers).
- Split your audience into a test group that sees ads and a control group, the holdout, that doesn't.
- Run the test long enough to capture delayed purchases.
- Compare results. The gap between test and control is your incremental lift.
Common types of incrementality tests
Geo holdout test
- How it works: Ads turn off in selected regions (DMAs, states, zip codes) while running everywhere else.
- Good to know: Measured on your own sales data and doesn’t rely on user tracking; needs careful geo matching.
Platform conversion lift study
- How it works: The ad platform randomly withholds ads from a set of users.
- Good to know: Runs inside the platform’s own measurement, so it isn’t an independent check; platforms set eligibility rules and may reject tests without enough volume.
Matched market test
- How it works: A small number of similar markets are paired, with ads on in one and off in the other.
- Good to know: Simple to set up, but fewer markets means less precision.
On/off (time-based) test
- How it works: Spend is paused for a period and results are compared before and after.
- Good to know: Easy but weak, because seasonality and promotions muddy the comparison.
Northbeam uses geo holdouts. The advantage is 3rd-party independence: every ad platform reports results through its own lens, and a geo test measures lift on your actual sales rather than asking you to take the platform's word for it. The quality of a geo test depends on how well the test and control regions match, so Northbeam pairs each test region with a control region that has behaved similarly in your historical data, accounting for conversion lag, discount patterns, and repeat purchases.
Key incrementality metrics and terms
Three numbers come out of most tests.
- Incremental lift: how much higher the test group performed than the control, as a percentage. Lift = (test − control) ÷ control.
- Incremental revenue: the dollar value of that difference, the sales your ads caused.
- Incremental ROAS (iROAS): incremental revenue ÷ ad spend. This is the number to compare against platform-reported ROAS.
A worked example
Illustrative numbers. A brand runs a geo holdout test on one channel, with the control group's results scaled to the same size as the test group:
The platform says every dollar returned $5. The test says $3.20. The difference is the sales the platform counted that would have happened anyway. That doesn't mean the channel is bad: a 3.2x iROAS can still be very profitable. It means you now know the real number to plan against.
A real example: Carpe
The subscription brand Carpe wanted to know how much of its growth Meta was actually driving. Northbeam ran an automated geo holdout test on Meta using Carpe's own attribution data. The test found 1,472 incremental new orders and $63K in incremental revenue, with lift driven almost entirely by new customers. Incremental orders came in 49% higher than a clicks-only attribution model credited, and incremental ROAS lined up with Carpe's internal estimates, which gave the team the confidence to keep scaling Meta.
Other terms you'll see
- Holdout: the share of geos or users excluded from ads. A 20% holdout means 20% see no ads.
- Conversion lag: the time between an ad running and the sale it drives. It sets how long a test must run.
- Minimum detectable effect (MDE): the smallest lift a test is designed to reliably detect.
- Confidence interval: the range the true result likely falls in. Narrow means precise; wide means uncertain.
- Halo effect: lift an ad drives on other sales channels, such as Amazon or retail stores.
When to run an incrementality test
- Before scaling a channel. Confirm it drives real growth before you double the budget.
- When attribution and gut feel disagree. A channel looks great in the dashboard but revenue isn't moving, or the reverse.
- For upper-funnel channels. CTV, YouTube, podcasts, and other awareness channels get little click-based credit, so tests show value attribution misses.
- When you sell in many places. If customers who see a Meta ad buy on Amazon or in stores, a test can capture that halo.
- When leadership asks for proof. An incrementality result is the most defensible answer to "is this spend working?"
How incrementality fits with MTA and MMM
Incrementality works best as one part of a measurement system, not a standalone report:
- Incrementality validates what works, with causal evidence.
- Media mix modeling (MMM) decides where to invest across channels.
- Multi-touch attribution (MTA) guides daily optimization at the campaign and creative level.
Test results calibrate the other two. Your iROAS becomes a factor applied to attributed ROAS and CAC, and lift results improve the MMM's estimates of channel impact. That turns a one-off test into an input you use for daily calibration.
Want to know how to run a good incrementality test? Check out this post.
Incrementality FAQ
What does incrementality mean in marketing?
Incrementality is the portion of sales or conversions directly caused by a marketing activity. It excludes sales that would have happened without the ad, such as purchases from customers who were already planning to buy.
What is the difference between incrementality and attribution?
Attribution assigns credit to the touchpoints that came before a sale. Incrementality measures whether those touchpoints caused the sale, by comparing people or regions that saw ads with a holdout group that didn't.
How do you calculate incremental ROAS?
Divide incremental revenue by ad spend. If a test shows $80,000 in incremental revenue on $25,000 of spend, iROAS is 3.2x.
What is a holdout group?
A holdout group is the set of users or regions deliberately not shown ads during a test. It shows what would have happened without the marketing, so the difference from the exposed group is the incremental effect.
How long does an incrementality test take?
It depends on how quickly the channel converts. A test has two parts: the holdout period, when ads are paused in control regions, and an observation window after ads resume to catch delayed purchases. For example, a test might run a three-week holdout followed by about two weeks of additional data capture. Upper-funnel channels with longer conversion lag need longer tests than performance channels like Meta.
Is incrementality testing only for big brands?
No, but a test needs enough volume to detect a difference: enough data points in the KPI you're measuring and meaningful spend on the channel you're testing. If a channel is too small to test today, a scale test (where you grow spend on a channel and measure the lift), can still tell you whether it's worth investing in.
Does a holdout mean turning off all my ads?
No. A geo holdout pauses ads only in a small share of regions, and the rest of your marketing runs as normal. Test design balances holdout size against test length so you put as little revenue at risk as possible.
Ready to measure what your ads really drive? Sign up for a demo today to learn more.
.avif)
%25201.avif)
.avif)
.svg)
