Google says 80% confidence is enough for directional Ads tests, pilots multi-campaign AI Max experiments

Google says 80% confidence is enough for directional Ads tests, pilots multi-campaign AI Max experiments

Google AdsNovember 20, 2026
By Antonio Fernandez

TL;DR

  • Google's Ads & Commerce blog said on 19 November 2026 to use 80% confidence for directional tweaks and reserve 95% for major structural changes in Google Ads experiments.
  • Google recommends running tests four to six weeks: about one week of ramp-up, four to five weeks of measurement, then a wait for delayed conversions.
  • Al Nevarez said in the 18 November Ads Decoded episode that Google "just launched the pilot this month", letting AI Max experiments include shared budgets and portfolio bid strategies, with each campaign split 50/50.
  • Google gave no markets or availability date for the pilot. Its methodology help page describes two-tailed testing at a 95% confidence interval, while its Monitor your experiments page says advertisers can pick the interval, with 80% as the default.

Google said on 19 November 2026 that advertisers can accept an 80% confidence level for directional Google Ads experiments and should keep 95% for major structural changes, with most tests running four to six weeks. In the Ads Decoded episode behind that advice, Google search experiments specialist Al Nevarez also said a pilot that "just launched ... this month" lets AI Max experiments include campaigns on a shared budget or a portfolio bid strategy, with results read across all selected campaigns in aggregate.

The written guidance is short. Google's Ads & Commerce blog post "Build strategic A/B tests to grow your ad performance" gives three rules. The detail sits in the episode it summarises, "A/B testing in Google Ads: How to prove what works," published on the Google Ads YouTube channel on 18 November and running about 48 minutes. Host Ginny Marvin, Google's Ads Product Liaison, talks with experiments product lead Sagar Shah and Nevarez. This article covers both.

The three rules in Google's 19 November post

The blog post reduces the episode to three rules:

  1. Start with templated setups. Templates isolate one variable, split traffic 50/50 and cut setup errors.
  2. Focus on business outcomes. Measure qualified conversions rather than shallow proxies, and give tests four to six weeks to account for conversion lag.
  3. Match confidence to decision scope. In Google's words: "An 80% confidence interval works well for directional tweaks. Reserve 95% certainty for major structural overhauls."

The post closes by pointing advertisers to the Experiments tab in Google Ads. It does not mention the multi-campaign pilot, shared budgets or portfolio bidding. Those come only from the episode.

The table below collects the specific figures stated by Google in the post and the episode.

The three rules in Google's 19 November post
TopicWhat Google saidWhere
Confidence levels80%, 85% and 95% are offered; 80% for directional calls, 95% for structural changes or applying a winner to untested campaigns; switchable in reporting at any timeBlog post and episode
Test lengthFour to six weeks as a heuristic; about one week of ramp-up, four to five weeks of measurement, then wait for delayed conversionsBlog post and episode
Template traffic split50/50 within the single existing campaign, not a campaign copyEpisode
Multi-campaign AI Max pilotLaunched "this month" per the episode; shared budgets and portfolio bid strategies can be selected; each campaign split 50/50; uplift assessed in aggregateEpisode
Budget test example$100/day tested at $150/day runs control at $50 and treatment at $75Episode
Spend minimumsNo spend thresholds or minimums on budget and target experimentsEpisode

How Google calculates significance

Marvin noted that Google rarely discusses its experiment methodology, and asked Nevarez to explain it. He described the usual three steps: calculate the variance of the data, then the standard error, then the confidence intervals. Google's method for the first step is jackknife resampling, also called leave-one-out. Traffic is split into buckets, and the results are recalculated many times, each time leaving out one bucket. If removing a few users changes the variance a lot, that points to an outlier, such as a single user who converts many times or has an unusually high conversion value. Nevarez said this protects against false positives.

He also said randomisation keeps any given user in the same arm and the same bucket for the whole trial, so users are treated independently. Google's help page on the statistical methodology behind experiments, which the episode description links to, adds the bucket count: twenty buckets in the control arm and twenty in the treatment arm.

Templates versus custom experiments

Inside the Experiments page, advertisers choose between pre-built, goal-based templates and custom experiments built from scratch. The guests' advice was to look for a template first. A custom experiment lets you change anything in the treatment arm, but if you move several levers you may finish the test without knowing which one caused the result.

Templates work differently under the hood. Rather than creating a campaign copy or draft, as custom experiments do, a template splits traffic 50/50 within the single original campaign. Google said this can reduce setup errors and may shorten the learning period because the test runs on the campaign you started with.

For custom search and display experiments, the sync feature is on by default. It copies changes made to the control campaign into the treatment campaign so the arms do not drift apart over a test of six to eight weeks or longer. The episode explained that sync can only be set at the start, when both arms still match. That is also why templates help: with a custom experiment, Google's systems can infer what changed between arms but cannot always tell what the advertiser meant to test.

The AI Max template and the multi-campaign pilot

Nevarez said the AI Max for Search template, found under experiments for campaign features, "has become the most popular way to test AI Max." The setup is short. Select a campaign, choose which campaign-level AI Max settings to enable in the treatment arm (text customisation and final URL expansion, with brand controls optional), keep or change the default key metrics used for significance, and decide whether a winning treatment applies automatically or manually. Search term matching is an ad group setting and cannot be changed while building the experiment.

The news is in campaign selection. Experiments can now include one campaign or several, and Nevarez said: "we just launched the pilot this month to actually allow you to select a shared budget or a bidding portfolio as well." With a multi-campaign selection, each campaign is split 50/50 as though it were a single campaign, and conversion uplift is assessed in aggregate. He gave two benefits. Concerns about traffic moving between campaigns go away because all of them are in the test together. And pooling campaigns adds conversion volume, which means more statistical power and possibly a faster conclusion. Marvin added that shared budgets had been an obstacle to using experiments at all.

What the episode did not say: which markets or accounts are in the pilot, how to join it, or when it will be generally available. Treat those as unknown.

Budget and target experiments

Google also covered the newer budget and bidding target experiments for Search, which test spend and target changes live instead of relying on Performance Planner forecasts. Depending on the campaign's bid strategy and whether it is budget-constrained, the setup asks for a new budget or a new target.

Nevarez's budget example: a campaign at $100 a day tested at $150 runs the control arm on $50 and the treatment arm on $75 during the trial, half of each budget. If the advertiser applies the result, the campaign moves to $150. For target tests, a tCPA moving from $20 to $30 simply sets the treatment arm to $30. Advertisers on target CPA typically judge on conversions and those on target ROAS on conversion value, though the key metrics are configurable and the report shows all metrics regardless. Google imposes no spend thresholds or minimums.

Testing manual CPC against Smart Bidding

When the base campaign uses manual CPC, the guidance was to set the treatment target from historical performance, for example a $15 CPA or 100% ROAS if that is what the campaign achieves, and to leave out the most recent conversion-lag period when calculating it. Including recent days with incomplete conversions would push the target higher than needed. A target set too tight can slow the treatment's ramp-up and blur the result.

How long to run a test

The four-to-six-week figure is described as a heuristic, not a rule. The aim is enough conversion volume for the confidence intervals to narrow. The rough schedule given in the episode: about a week of ramp-up, four to five weeks of measurement, then end the experiment and wait for delayed conversions before comparing arms. Low-volume accounts may need longer.

Choosing a confidence level

Marvin said Google now offers 80%, 85% and 95% confidence intervals. The answer given was that 80% is typically good enough when combined with the advertiser's judgment and the goal is a directional improvement without breaking anything. For automated features such as AI Max, which keep learning, strong certainty may not be needed. If the advertiser plans to apply the treatment to campaigns that were not in the test, 95% is the stricter choice. The level can be changed in reporting at any time, and the confidence intervals update straight away.

Google's help page on experiment methodology describes two-tailed testing "using the 95% confidence interval." A separate help page, Monitor your experiments, says advertisers can pick their own confidence interval and that 80% is the default. It is worth checking which level a given report is using.

Other experiment types covered

  • Ad variations: built for one isolated change at scale, such as a headline, description or final URL across many campaigns or a whole account.
  • Asset and video tests: Marvin mentioned asset uplift experiments, video asset experiments with "up to, I think, 20 groups" (her own hedge), and a Performance Max final URL expansion experiment.
  • Retail Performance Max: a feed-only campaign can be tested against feed plus assets, as long as budgets, targets and other settings match.
  • Performance Max and Demand Gen uplift experiments: these assign "comparable campaigns" (matched on signals such as domain, conversion actions and location targeting) to both arms, so the result measures the all-in effect of adding the campaign, including traffic shifted from existing campaigns.

When a test shows no improvement

Nevarez's checklist, in order: review all metrics in the report, since something may have improved that was not the primary metric; confirm conversion tracking works; check the test ran the recommended four to six weeks; for AI Max, rerun with text customisation and final URL expansion enabled if they were off; check whether the treatment arm became budget-constrained; review search terms for negatives; and, if volume is the problem, add more campaigns through the multi-campaign option.

Two further points came up. A 50/50 traffic split does not mean equal spend: Shah's example was a $10 CPA arm getting 20 conversions and spending $200 against a control at 10 conversions and $100. And after applying a winning treatment, a short "post-experiment slump" can happen while the base campaign learns. The episode also warned that CPCs can rise under Smart Bidding while CPA or ROAS stays on target.

What this means for Thai marketers

Google did not mention Thailand, and it did not say where the multi-campaign pilot is available, so do not assume it is in Thai accounts yet. The rest of the guidance applies to any Google Ads account.

For low-volume accounts, the 80% option and the four-to-six-week guideline matter more than they do for large advertisers, because a 95% threshold on thin data can keep a test running for months without a verdict. The trade-off is plain: 80% means accepting more risk of a false winner, which is fine for a reversible tweak and less fine for an account-wide restructure.

Two practical checks follow from the episode. First, make sure the conversion action being tested is the one the business cares about; the guests singled out lead-gen accounts that track a form fill when the real goal is a qualified lead. Clean GA4 and conversion measurement comes before any experiment. Second, if campaigns share a budget or a portfolio strategy, watch for the multi-campaign option in the Experiments page rather than splitting campaigns apart just to run a test. Teams that want help designing tests can look at how Google Ads management builds experiments into the regular account workflow.

FAQ

Is 80% confidence enough for a Google Ads experiment?

Google says yes for directional tweaks. Its 19 November post recommends 80% for directional changes and 95% for major structural changes, and in the episode Google added that 95% is the safer choice when applying a winner to campaigns that were not in the test.

How long should a Google Ads A/B test run?

Google recommends four to six weeks. In the episode it described about one week of ramp-up, four to five weeks of measurement, then a wait for delayed conversions, and said low-volume accounts may need longer.

Can I run an AI Max experiment on campaigns with a shared budget?

Possibly, if your account is in the pilot. Al Nevarez said a pilot launched "this month" allows shared budgets and portfolio bid strategies in AI Max experiments, but Google did not say which accounts or markets are included.

Why did my treatment arm spend more than the control in a 50/50 test?

Because the split is by traffic and users, not by spend. If the treatment finds more conversions at the same CPA, it spends more within the campaign budget, which Google described as expected behaviour.

Is the multi-campaign pilot available in Thailand?

Not stated. Google did not give markets, eligibility or a general availability date for the pilot.

Bottom line

The blog post gives the rules; the episode gives the numbers and the new pilot. The most useful change for many accounts is the official acceptance of 80% confidence for reversible decisions, paired with a clear schedule for how long to wait. If you want a second opinion on how to structure tests in your account, Relevant Audience can help plan them.

Antonio Fernandez

Antonio Fernandez

Founder and CEO of Relevant Audience. With over 15 years of experience in digital marketing strategy, he leads teams across southeast Asia in delivering exceptional results for clients through performance-focused digital solutions.

Share to:
Copy link:

Read us often? Add Relevant Audience as a preferred source so our articles surface more in your Google results.