Skip to content

Analytics and testing

Understanding A/B test metrics

Div-idy documentation Updated

Open a test’s Analysis to compare Original A and Variant B. The tables report observed activity, including views, ad clicks, AI use, average time, and recorded event groups. The displayed Conversion rate is occurrences divided by unique visitors, not a guaranteed percentage of people who completed a goal.

Open the right comparison

In A/B tests, find the named test and choose View analysis. Verify the page and test name before interpreting the tables. You can star useful metric blocks to move them higher. The comparison belongs to that test; the general Analytics tool covers broader project activity. Data generated by your own visits to the live page can contribute to the test just like other recorded traffic.

Understand counts and repeated actions

Unique visitors estimates recorded browsers in each version. Views and event totals can include repeated actions. A table showing 30 actions from 20 visitors reports a 150% rate because it divides actions by visitors; it does not mean 150% of people converted. Distinguish form submissions from verified saved records, and AI interactions from successful answers. Event names describe the recorded action, not the quality of its outcome.

Read Growth as a relative comparison

Suppose A records 10 actions from 100 visitors and B records 15 from 100. Their observed rates are 10% and 15%, and Growth is 50%: B’s rate is half again as large as A’s. The absolute difference is five percentage points. When A’s baseline is zero, the interface displays zero growth rather than a meaningful relative lift. Always read the underlying counts alongside the percentage.

Know which observations enter a metric

Supported published-page events carry the test and version shown to that visitor. Event blocks use the latest 300 matching event records, so they are not necessarily lifetime event totals. Avg time (s) uses recorded sessions containing an event attributed to the test; it is not an average of every test view. These differences can explain why views, event blocks, and time do not describe identical sets of visits.

Use results without overstating certainty

The split aims for 50/50 assignment, but a small audience can be uneven and repeat visits stay with the assigned version while cookies persist. Growth is descriptive, not a significance calculation or automatic winner declaration. Consider audience size, repeated actions, device behavior, and whether your intended goal was actually completed. Verify both versions still work before interpreting a difference as improvement or choosing a permanent change.