Craig Campbell SEO Training & Consultancy

Tutorials

SEO testing: assess a change without confusing correlation with proof

By Craig Campbell · 15 Nov 2026

5 min read

SEO testing means evaluating a planned website change against a defined outcome. A before-and-after chart can show what changed, but it cannot on its own prove what caused the change.

This beginner guide distinguishes monitoring from a stronger controlled comparison. It is an editorial planning workflow, not a statistical test engine. Sources were checked on 15 November 2026. All business names and figures below are fictional and illustrative only.

1. State one question and one primary measure

Describe the change and why it might help. For example: “Clearer category titles may help relevant searchers identify these pages.” Choose a primary measure before seeing the result, and record useful secondary measures and safety checks.

  • For search visibility, use compatible Search Console page/query metrics.
  • For enquiries, check analytics and enquiry tracking separately.
  • Do not treat search clicks and analytics sessions as interchangeable.
  • Do not choose whichever metric looks best afterwards.

2. Decide what the site can actually support

A small site with little traffic may only support careful monitoring. Larger sites with many comparable pages may support page-group SEO split testing: change one group while leaving a suitable comparison group unchanged.

That is different from visitor A/B testing, where users see different versions of the same page. Random page groups, suitable pre-test comparisons and an appropriate analysis method matter; a simple uplift calculation is not a significance test.

  • Compare pages with similar purpose, demand and pre-change behaviour.
  • Allow for seasonality, day-of-week effects and query differences.
  • Check that cross-linking or shared templates will not silently change the comparison group.
  • Use specialist analysis if the decision needs a defensible causal estimate.

3. Record the plan and change log

Write down the affected URLs, comparison pages, metric filters, start date, review criteria and rollback plan. Avoid changing titles, content and templates together if you want to understand the effect of a title change.

There is no universal minimum duration or sample size. Recrawling, traffic volume and variability differ. If the available data is too sparse, report an inconclusive result rather than extending the claim.

Fictional UK example — illustrative only.

A fictional Manchester lighting shop chooses comparable pendant-light category pages. It changes only their title wording, keeps a separate group unchanged, and records a planned review after enough post-recrawl data is available. A weekend sale is logged as a possible confounder.

4. Protect search access during visitor tests

Google’s guidance for website A/B tests recommends avoiding cloaking, using canonical links when alternate test URLs exist, using temporary 302 redirects rather than permanent 301 redirects for temporary tests, and removing test variations once the experiment is finished.

Those are safeguards for website variations, not instructions to canonicalise distinct category pages in a page-group SEO experiment. Do not show Googlebot a special version designed to manipulate indexing.

5. Read the result in context

Fictional UK example — illustrative only.

The fictional changed group rises from 1,000 to 1,150 clicks: 15%. Its comparison group rises from 1,000 to 1,100: 10%. A crude ratio comparison is (1.15 ÷ 1.10 − 1) × 100 ≈ 4.5%. This descriptive number is not a validated causal estimate, confidence interval or proof that the edit worked.

Check whether page groups remained comparable, whether rankings and query mix moved, and whether the change was crawled. Shared market changes, a promotion or an algorithm update may explain part of the movement.

Use the Search Console guide to avoid comparing incompatible filters, and the forecasting guide to keep assumptions visible.

6. Report a decision and its uncertainty

  • Keep: evidence supports the change, with acceptable safety metrics.
  • Roll back: evidence or user impact suggests a problem.
  • Inconclusive: the data or comparison is not strong enough.
  • Retest: a more focused question or better page group is needed.

Document the result in your monthly report. State what was changed, what was measured, what else happened and what you cannot conclude.

Your first test

Start with a small, reversible improvement such as the title-tag workflow. If you are correcting an inaccurate claim, fix it because it is inaccurate: factual accuracy should not depend on whether a traffic test produces a winner.

Sources

Let's work together.

Whether you need an SEO partner, a consultant, a speaker for your event or training for your team — get in touch and let's chat.

Get in touch
Get in touch