Analytics and testing
A/B test QA checklist and common pitfalls
Before launching, verify the selected page is live, the variant is saved, both versions work, and your intended action appears in Analysis. Test changes are scoped to Variant B. Saving an edit pauses the test, and repeated refreshes normally keep a browser in its assigned version.
Check the page and test scope
Open the named test from A/B tests and confirm its page. Use Original A to inspect the comparison page and Variant B when editing. Write one concrete change and one expected observation before you build. A broad redesign makes a difference harder to explain. Confirm that the prompt bar says it is editing the variant rather than the main project.
Test both versions before launch
Use ordinary, empty, and boundary inputs in both versions. Check form validation, calculator results, links, keyboard use, and mobile layout. For a quote-form test, submit the same example in each version and verify the expected saved record or result. A better-looking variant can still contain a broken action. Wait for the completed saved result; generation progress alone is not evidence that the variant is ready.
Verify actual live status
In Publishing, confirm the tested page is live. Return to the test, choose Go Live, and wait for the safety check and Live status. Saved edits pause a test, so verify status after every change. Activating another test for the same page pauses the previous one. Use Pause if you need to stop the comparison; visitors then receive the original page.
Avoid misleading assignment checks
Share the normal live link. Repeated refreshes generally keep the same browser in the same version, because assignment is sticky per visitor and test. Use the editor’s A/B controls to inspect both designs directly. Different browser contexts can receive different assignments but are not guaranteed to alternate. Cookie restrictions, especially in embeds, may prevent persistence. A small sample also need not divide into exactly equal counts.
Validate the measurement and decision
Trigger a clearly named action on the live test, then open Analysis and inspect its event block. Counts can include repeated clicks and your own testing. Rates divide occurrences by visitors, and the event breakdown is limited to recent matching records. Review those limits before claiming improvement. If you choose Make Variant 100%, review the confirmation carefully: applying the variant changes the main page and ends the test, rather than merely pausing it.