Holdouts Measure the long game.
Single experiments measure single changes. A holdout keeps a small slice of users away from a whole set of launches, so you can see their combined effect on retention and revenue months later.
How it works
-
Reserve a slice
Set aside a small share of users, such as 2%, who won’t see new features.
-
Ship as usual
Every flag and experiment automatically leaves the holdout out.
-
Compare
At the end of the quarter, compare the holdout with everyone else on the metrics that matter.
The details
- Automatic exclusion
- New flags respect the holdout without anyone remembering to configure it.
- Long-term metrics
- Read retention, revenue and support load, not just clicks.
- Rotate fairly
- Release the holdout at the end of each period and start a new one.
More of the platform
-
Feature flags
Put every change behind a flag and decide who sees it, without a deploy.
Learn more: Feature flags -
Scheduled rollouts
Go from 1% to everyone over days, with automatic pauses when errors rise.
Learn more: Scheduled rollouts -
Experiments
Split traffic between variants and get a clear result, with the statistics done for you.
Learn more: Experiments
Try Holdouts today
Free for small teams. Set up in minutes.