ExperimentsShip the variant
Ship the variant
that won.
Every flag is already an experiment. Pick a metric, split the traffic and let the stats engine tell you when it is safe to ship.
checkout-ctaReady to ship
A · control"Continue"3.12%
B · variant"Pay £42 now"3.25%
- Uplift
- +4.2%
- Chance to beat A
- 97.8%
- Revenue / user
- +£0.31
From hypothesis to shipped
in five steps.
- 01HypothesisWrite what you expect and which metric proves it.
- 02SplitPick a flag and a percentage. No new deploy.
- 03GuardrailsAuto-stop if latency or errors move the wrong way.
- 04ReadSequential stats: peek any time without p-hacking.
- 05ShipOne click moves the winner to 100% and archives the loser.
Last week’s results.
pricing-page-annualWinner
+11.3%annual plan take-rate
onboarding-checklistNo effect
+0.4%activation in 7 days
agent-summary-v2Loser
−6.1%tickets resolved by agent
Metrics you
already trust.
Pull metrics from your warehouse or define them in SQL. Every experiment, flag and agent run can use the same definitions.
- conversion_rate
- revenue_per_user
- p95_latency
- churn_30d
- activation_7d
- tokens_per_session
- support_tickets
- refund_rate
- agent_resolution
“We stopped arguing about which idea was better. Now we test it on Tuesday and ship the winner on Friday.”
- 62
- experiments shipped by Brightline last quarter, up from 9