Incrementality testing for paid media: Geo tests, holdouts, and the minimum you can run

Incrementality testing measures the conversions a campaign caused by comparing a group that saw the ads with a control group that did not, then subtracting expected conversions from observed ones. Google's Conversion Lift requires at least 1,000 observed conversions and a $5,000 minimum campaign budget. In one Haus example, turning off a Performance Max campaign would cut sales by only 30% of what Google attributed to it.

What is incrementality testing?

Google's Conversion Lift sets two entry conditions before a study can run: at least 1,000 observed conversions and a minimum campaign budget of $5,000 USD. Modeled conversions do not help you clear the first bar, because conversions with supplementary data don't count toward the minimum. Those thresholds tell you something about the method. Incrementality needs volume, because it measures a difference between two groups.

The definition

GeoLift's documentation defines a campaign's incremental effect as the difference between what was observed and what would have happened in a world where it didn't take place, the counterfactual. You never see the counterfactual directly. You estimate it with a control group that did not see the ads. The same documentation describes randomized controlled trials (RCTs) as the gold standard of incrementality. Randomization makes the control group a fair stand-in for the world without the campaign.

Incremental conversions = observed conversions in the test group minus expected conversions without the campaign

See the incrementality glossary entry for the short version.

Why attributed conversions overstate it

Attribution assigns credit to touchpoints. It cannot tell an ad that changed a decision apart from an ad that appeared before a purchase the buyer was going to make anyway. Haus describes a Performance Max campaign that included brand search. A test showed that turning it off would cut total sales by only 30% of what Google attributes to it. The remaining 70% would have arrived through other channels or organic search.

Here is that ratio applied to an illustrative campaign of our own:

If you were paying for 400 conversions and dividing spend by 400, your true cost per incremental conversion is more than three times the reported cost per conversion. We think this gap is where most brand-heavy budgets lose money. The marketing-sourced vs. influenced method runs into the same credit problem at the pipeline level.

Turning one test into a reporting adjustment

Haus describes an incrementality factor: if incremental conversions equal 60% of platform-attributed conversions, you can apply this 0.6 multiplier to ongoing performance metrics. In practice:

1. Run the test and compute incremental conversions divided by attributed conversions for the same period.
2. Multiply daily platform-reported conversions by that factor.
3. Recompute cost per conversion and return on ad spend from the adjusted count.
4. Retest when spend, targeting or campaign mix changes, because the factor belongs to the setup you tested.

How does a conversion lift study work?

A conversion lift study splits one audience into two groups and compares them. Google describes the split as people who see your ads (the treatment group) and people who don't (the control group), and states that the difference in conversions between these 2 groups will tell you the "lift," meaning the conversions the ad caused. Meta's version works the same way: it compares the behavior of people who saw your ad (treatment group) with those who didn't (holdout group).

The holdout is what separates a lift study from a split test. An A/B test doesn't measure incremental impact because a holdout is not applied to any of the cells, so each cell can only be judged on attributed results.

User-based or geography-based in Google Ads

Google offers two study types.

For a B2B team whose real conversion is an opportunity created weeks after the click, we think the geography option fits better, because it accepts offline data.

Eligibility and the one-study rule

Supported campaign types are Display, Search, Video, Demand Gen, App Campaigns and Performance Max campaigns. App campaigns targeting iOS and Travel Ads are excluded. A campaign can be part of max one study at a time, whether that study is Brand Lift, Search Lift or Conversion Lift. If your brand team already has a Brand Lift study running on a Performance Max campaign, remove the campaign from that study before adding it to a conversion test.

Before launch, Google's study power tool estimates how likely the study is to reach a conclusive result. Google advises aiming for 90% certainty and treats results between 50% and 90% as directional.

Reading the result: a worked example

Google defines incremental cost per action (iCPA) as the total spend divided by Incremental Conversions. Here is a hypothetical study:

• Spend on the tested campaign: $30,000
• Treatment group conversions: 600
• Control group conversions, scaled to the same size: 450
• Incremental conversions: 600 minus 450 = 150
• iCPA: $30,000 / 150 = $200

If the platform reported a cost per conversion of $50 ($30,000 / 600), the gap between $50 and $200 is the number to bring to your CAC payback model.

What is a geo lift test?

A geo lift test randomizes places instead of people. You switch media on, off or up in a set of regions, leave the rest alone, and compare outcomes. The hard part is the comparison group: no single untreated city behaves exactly like your test city.

Synthetic control solves that. In GeoLift, Meta's open-source package, the method works like this: using historical information prior to the treatment, it finds the combination of untreated units that most closely replicates the treated one. The weighted blend becomes the counterfactual. Prior to the intervention the test and synthetic control are virtually identical, so the gap that opens after launch is read as the campaign's effect, in the same observed-minus-counterfactual sense covered above.

The data shape

The GeoLift walkthrough uses simulated data of 40 US cities across 90 days. Every experiment needs at least three fields:

Clean the panel first. The walkthrough warns to make sure that there are no missing variables, NAs, or locations with missing time-stamps as those will be dropped. For a B2B team, Y is usually opportunities or qualified pipeline by region, pulled from the customer relationship management (CRM) system, not form fills.

A worked setup

The walkthrough's market selection example assumed Chicago must be part of the test markets and we have up to $100,000 to run the test, with a CPIC = $7.50 obtained from a previous Lift test. CPIC is cost per incremental conversion. Our arithmetic: $100,000 divided by $7.50 caps the test at about 13,333 incremental conversions. If your pipeline cost per incremental opportunity sits in the hundreds of dollars, the same budget buys far fewer events, and you need more markets or a longer window.

Run the power analysis before launch. In geo tests the effect sizes are usually small and where there is often a significant chance to fail to find the effect of the experiment.

When a geo test replaces a lift study

Platform lift studies are often gated by the access to online signals (pixels, APIs, SDKs, etc.). A geo test needs only regional outcomes, so it fits three cases:

• Your pixel or conversion API coverage is thin or shrinking.
• The conversion that matters happens offline, such as a sales-accepted opportunity in the CRM.
• You want one design across channels. Google's Meridian GeoX supports holdback, go-dark, and heavy-up designs, and for go-dark or heavy-up you must also collect historical daily geo-level spend data.

Match the test window to your sales cycle length, or measure an earlier stage. See incrementality for the core definition.

Holdouts, geo tests and on-off tests compared

The three designs share the same logic of treatment against control. They differ in what you randomize, what data you must prepare before launch, and which budget question the result can settle. The table sets them side by side. The last two columns are our reading, not platform rules.

Two output details matter for a pipeline owner. A geography-based Conversion Lift study reports Incremental ROAS and Incremental Cost, so you can compare the result directly against your channel plan. GeoX also lets you force certain geos to be excluded to avoid large media disruptions, which is how you protect a region carrying a quarter's largest deals.

Multi-cell designs with one control

GeoX supports multi-cell execution, which saves both cost and time by comparing multiple treatment arms with a common control group.

In our reading, the shared control gives each arm more geos and halves the calendar time. The trade-off is that both arms lean on one control group, so a bad match in that group affects both readings.

Whatever mix you choose, Google's guidance is an annual testing plan that ensures tests across different channels are never overlapping. Put the cells, geos and dates on one calendar before any budget moves. Then run the result through the demand gen budget allocator.

Worked example: a 10% holdout test

This example uses illustrative inputs, not results from any campaign. One audience is split at random. One group of 10% sees no ads, and the other 90% sees the campaign as planned. The arithmetic below runs from conversion counts to the cost of each conversion the ads actually caused.

The arithmetic, line by line

1. Conversion rate, shown ads. 1,260 ÷ 180,000 = 0.70%.
2. Conversion rate, held out. 100 ÷ 20,000 = 0.50%. This is the baseline: the rate the audience converts at with no ads at all.
3. Absolute lift. 0.70% − 0.50% = 0.20 percentage points.
4. Relative lift. 0.20 ÷ 0.50 = 40%. Report it next to the absolute lift. A 40% relative lift on a small base is still a small number of extra conversions.
5. Incremental conversions. Apply the absolute lift to the exposed group only: 180,000 × 0.20% = 360. The held-out group saw no ads, so it adds no incremental conversions.
6. Cost per conversion the platform reports. $36,000 ÷ 1,260 = $28.57. This counts every conversion in the exposed group, including the ones that would have happened anyway.
7. Cost per incremental conversion. $36,000 ÷ 360 = $100.

Reading the result

The gap between $28.57 and $100 is the whole point of the test. Most of the 1,260 conversions came from people who were going to convert regardless. If your CAC payback model uses the platform figure, it is pricing the channel on conversions it did not cause.

Two checks before anyone acts on the number. First, confirm the split was random and that both groups were drawn from the same audience over the same dates. Second, look at the held-out count. One hundred conversions is a thin base, and a few conversions either way would move the baseline and every line below it. We think a holdout this size is a first reading to repeat, not a final verdict on the channel.

The $100 figure also has a second use. GeoLift's power calculator for known test locations takes a Cost Per Incremental Conversion as an input when sizing the investment for a geo test. A holdout result like this one gives you a starting value to plan the next test with, instead of guessing.

How many conversions and how many weeks a lift test needs

The conversion floor from the opening section tells you whether Google will run a study. It does not tell you whether your test can detect the lift you care about. That second question is a sample size problem, and you can solve it before you spend anything. Google's own floor sits beside a minimum campaign budget of $5,000 USD, so the arithmetic below decides whether that budget buys a readable result.

Worked example: people per group to detect a 20% lift, by baseline conversion rate

Item	Value
0.5% baseline	85862 people per group
1.0% baseline	42693 people per group
2.0% baseline	21109 people per group

Source: Take the Market calculation: two-proportion z test, 5% significance, 80% power, equal groups

n per group = [z_alpha/2 × √(2 × p̄ × (1 − p̄)) + z_beta × √(p1(1 − p1) + p2(1 − p2))]² ÷ (p2 − p1)², where p̄ is the average of p1 and p2

Here p1 is the baseline conversion rate, and p2 = p1 × (1 + relative lift). Assumptions: 95% two-sided confidence, 80% power and equal groups, with z values of 1.96 and 0.84. The table computes those z values unrounded.

import math
from statistics import NormalDist

def n_per_group(baseline, relative_lift, alpha=0.05, power=0.80):
    p1 = baseline
    p2 = baseline * (1 + relative_lift)
    z_a = NormalDist().inv_cdf(1 - alpha / 2)
    z_b = NormalDist().inv_cdf(power)
    p_bar = (p1 + p2) / 2
    top = (z_a * math.sqrt(2 * p_bar * (1 - p_bar))
           + z_b * math.sqrt(p1 * (1 - p1) + p2 * (1 - p2))) ** 2
    return math.ceil(top / (p2 - p1) ** 2)

print(n_per_group(0.01, 0.20))

Checked row: a 1.0% baseline and a 20% lift gives p2 = 1.2%. With z rounded to 1.96 and 0.84 the result is about 42,640. With unrounded z values the table shows 42,693 per group. The small gap comes from rounding.

From people to weeks

Weeks needed = (2 × n per group) ÷ weekly reach of new eligible people, plus your sales cycle in weeks. The second term matters because a B2B conversion often lands after the exposure window closes. Check your cycle against our sales cycle length benchmarks before you set an end date.

It applies to eligible Demand Gen only studies that started on or after March 2, 2026.

Google's feasibility tool reports certainty of lift on a percentage range from 50% to 95%, in 5% increments. It treats results between 50% and 90% as directional. If n is out of reach, the lever Google names is to choose conversions that happen more often. In our reading, moving from a closed-won goal to a form fill raises p1 and cuts n sharply, as the rows above show.

Monday step

Pull last quarter's baseline conversion rate from your funnel conversion benchmarks comparison or your own CRM. Pick the smallest lift worth acting on, run the function, divide 2 × n by weekly reach and add the sales cycle. If the answer is longer than a quarter, raise the lift you test for or move to a more frequent conversion.

How incrementality tests feed a marketing mix model

A lift test gives you one number for one channel over a few weeks. A marketing mix model (MMM) gives you response curves for every channel over years of history. Calibration is how the first corrects the second: the test result tells the model where a channel's effect should sit, and the model carries that anchor across the periods you did not test.

Calibration in Meridian

Meridian is Google's open-source MMM, and it takes outside evidence through Bayesian priors. The documentation says you can inject your existing business knowledge into the model through priors, and that a prior gives the model a starting point based on your information. In our reading, an incremental return on ad spend (ROAS) from a geo test is the most direct business knowledge you can feed it.

In our reading, this closes the loop: the test plan from earlier sections now gets ranked by the model it feeds.

Mapping a test to the right variable in Robyn

Robyn, Meta's open-source MMM, takes calibration against specific media variables, so the test has to match how you defined the channel. It separates four kinds of input:

The two vectors must have the same length and the same order of media, and when exposure metrics are not available, use the same as in paid_media_spends. Robyn's maintainers note that spend-to-exposure translation errors persist but that it's mitigated through the calibration of response curve from ground truth.

The practical rule

Define the channel in the model the way you split it in the test. If you ran a prospecting-only holdout on Meta, a single Meta variable cannot absorb that result cleanly. Robyn's guidance already says it would be meaningful to split Meta by retargeting and prospecting when their cost per thousand impressions differs.

A short checklist before you calibrate:

1. Name the model variable the test measured, before launch.
2. Match the test window to the model's date range.
3. Record incremental ROAS with its confidence interval, not just the point estimate.
4. Log the result next to the channel in your demand gen dashboard so the next model refresh uses it.

The minimum test a B2B team with few conversions can run

Both standard tools leave a gap for a team with few conversions. Google's measurement team wrote that MMM, or marketing mix modeling, "excels in cross-channel performance evaluation but may fall short in terms of speed and granularity of results" (Google). The same piece said privacy updates and the phase-out of third-party cookies made attribution harder to use alone for budget allocation (Google). A team below platform thresholds still needs a causal read on one channel. This is the minimum design we use.

Three choices that make a small test readable

1. Move the outcome upstream. Tests can focus on "revenue, profit, or any site or app action that drives value for your business" (Google). Closed-won deals are too few for most B2B teams. Pick the highest-volume outcome sales still accepts as real, such as sales-qualified meetings or opportunities created. Fix the definition before launch.
2. Go dark or hold back on one channel in matched regions. Cut one channel in regions that track the rest on past outcome volume. Leave every other channel unchanged in both groups. Log results by region, because "All conversions and spend for tested campaigns are required for each geo" (Google for Developers).
3. Run longer. Low weekly volume is noisy. Make the window cover at least the lag from first touch to your chosen outcome, using your own sales cycle data.

Worked example

A team creates 80 opportunities a month and closes 10 deals. It holds back paid LinkedIn for 8 weeks in regions that normally produce 25% of opportunities. Over 8 weeks the account would expect about 160 opportunities, so about 40 in the holdout regions. If the holdout regions produce 34, the channel drove about 6 opportunities there. For incremental return on ad spend, "divide your newly discovered incremental revenue by your campaign’s media spend" (Google), using your win rate and deal size to turn opportunities into revenue.

Test brief template

Write the decision rule first. In our reading, a test whose threshold is set after the result arrives does not inform the budget.

What this page does not cover

This page does not cover Brand Lift or Search Lift studies, building a marketing mix model from scratch, or the access process for Conversion Lift beyond contacting your account representative. The worked examples use illustrative numbers, not results from any campaign. Nothing here is financial or legal advice.

Related on Take the Market: Marketing-sourced versus influenced pipeline: definitions, the usual percentages and the argument to avoid; Self-reported attribution: how to design the field, code the answers and reconcile them with software attribution; Pipeline coverage calculator: the ratio you need from your own win rate and slippage.