Your Black Friday variant converts more shoppers. The team celebrates. Then someone notices that customers bought cheaper baskets and used a larger discount. Did the test actually improve the business?
Revenue per visitor and conversion rate answer different questions. Use both, then add contribution when costs differ. A consistent scorecard helps prevent a local improvement in orders from becoming a worse commercial decision.
Define the denominator before comparing anything
A visitor is not the same as a session. One person can return several times before buying. Your analytics platform may also use different identity rules across devices and consent states.
For an experiment assigned by visitor, choose an eligible visitor definition and use it consistently. For a session-based operational dashboard, label the metric accordingly. Do not compare a visitor-based test metric with a session-based benchmark as if they were interchangeable.
Write down which visitors qualify, when they enter the experiment, how long orders are attributed, and how repeat visits are handled. That specification matters more than a polished dashboard label.
Calculate conversion and revenue per visitor
For this article, visitor purchase conversion rate is the share of eligible visitors who place at least one qualifying order:
Purchase conversion rate = purchasing visitors ÷ eligible visitors.
Revenue per visitor = qualifying revenue ÷ eligible visitors.
Average order value = qualifying revenue ÷ qualifying orders.
Use the same currency and revenue definition across groups. If one visitor can place multiple orders, conversion rate multiplied by average order value does not necessarily equal revenue per visitor. That shortcut also needs orders per purchasing visitor. It works directly when each purchasing visitor places exactly one qualifying order.
VWO's ecommerce metrics guide provides background on these metrics. The examples below use original hypothetical data to show why definitions matter.
Work through a higher-conversion, lower-revenue result
Assume each purchasing visitor makes one order, the groups contain equally eligible visitors, and revenue is measured after discounts on the same basis.
| Metric | Control | Variant |
|---|---|---|
| Eligible visitors | 10,000 | 10,000 |
| Purchasing visitors and orders | 300 | 360 |
| Purchase conversion rate | 3.0% | 3.6% |
| Average order value | $100 | $75 |
| Qualifying revenue | $30,000 | $27,000 |
| Revenue per visitor | $3.00 | $2.70 |
The variant has a 20% relative conversion increase: 3.6 divided by 3.0, minus one. Its revenue per visitor is 10% lower. These are descriptive calculations, not evidence of statistical significance.
Possible explanations include a cheaper product mix, a larger discount, or fewer higher-value purchases. The next step is to investigate those mechanisms using the predefined analysis plan.
Add contribution when costs change
Revenue excludes the cost of generating the transaction. A free-shipping offer and a bundle can have very different variable costs even if their sales totals match.
Define contribution consistently: qualifying net sales less included variable costs. Then divide by eligible visitors to calculate contribution per visitor. Document which costs are included and whether returns are observed or estimated.
The discount testing guide includes a worked comparison. If reliable cost data is unavailable, report revenue alongside explicit cost assumptions. Avoid relabeling sales as profit.
Build a compact experiment scorecard
Choose the primary metric based on the decision the experiment is meant to inform. A purchase-area clarity test may prioritize purchase conversion, while an offer-economics test may prioritize contribution per visitor.
| Metric role | Example | Why it is useful |
|---|---|---|
| Primary outcome | Contribution per eligible visitor | Evaluates the offer's commercial objective |
| Supporting outcome | Purchase conversion | Shows whether more visitors buy |
| Diagnostic | Add-to-cart progression | Helps explain where behavior changed |
| Operational guardrail | Discount or payment failures | Identifies a broken transaction |
| Delayed outcome | Returns and cancellations | Checks whether immediate gains persist |
Agree on the primary metric and guardrails before launch. Do not pick whichever metric looks best afterward. Diagnostic improvements can help generate the next idea even when the primary result is inconclusive.
Account for noisy revenue and incomplete outcomes
Order values can vary widely. A few unusually large orders can move the average, so revenue experiments need an appropriate analysis method and enough exposure. Use your experimentation platform's supported approach to uncertainty.
Predefine any outlier handling instead of removing large orders after seeing which group they favor. Record campaign interruptions, stock changes, and tracking failures. A clean-looking chart does not repair missing or inconsistent data.
Returns can arrive after the sale. Set a follow-up date and label the immediate result provisional where costs are still developing. Keep exploratory device or channel cuts distinct from preplanned comparisons.
Distinguish seasonal performance from evergreen performance
A Black Friday result answers a question about a particular offer, audience, and period. The same presentation may behave differently when visitors have lower urgency and prices return to normal.
Keep a record of campaign conditions: traffic sources, eligibility, product availability, promotion terms, and the tested dates. If you want to apply the variant year-round, design an appropriate follow-up rather than assuming the seasonal outcome transfers unchanged.
For the immediate campaign, use the CRO checklist to align the launch dashboard with the experiment scorecard. The two views serve related but different operational needs.
Make the next decision explicit
Write a short result note: what changed, which population was measured, the primary estimate and uncertainty, guardrail findings, and the resulting action. “Conversion went up” is incomplete when revenue or costs moved the other way.
VibeVariant can help build a reviewable presentation variant on a supported storefront page. Use your experimentation and analytics stack for assignment, metric calculation, and analysis. A useful experiment ends with a decision that another teammate can understand and revisit.
