Global Holdout
The global holdout is the system-wide control group for measuring Adaptive Journeys' incremental impact. A small fraction of every workspace's end-users — by default 5% — are assigned to the holdout and receive no marketing sends from any journey, no matter what.
The difference between holdout and exposed groups is the calibrated impact of your journey portfolio.
Why workspace-level (not per-journey)
Per-journey holdouts compose poorly. A user excluded from journey A is still exposed to journey B; the "control" stops being a control once two journeys overlap.
A single workspace-level holdout fixes this:
- Every journey's lift estimate uses the same control group
- Lift estimates are comparable across journeys
- Users have a stable holdout status (no flipping between in/out as journeys come and go)
Determinism
Assignment is computed as a stable FNV-1a hash of (projectKey, endUserId) modulo 100:
bucket = FNV-1a(projectKey + "::" + endUserId) % 100
inHoldout = bucket < holdoutPct
The hash is stable across deploys, retries, and re-evaluations. Same (projectKey, endUserId) pair always produces the same bucket. A user's holdout status never changes.
This determinism is what makes the holdout a valid control. If users could flip in and out, accumulated impact measurements would be invalidated.
What "no sends" means
Holdout users still:
- Receive transactional communications (2FA codes, password resets, receipts) — these aren't subject to holdout
- Show up in your event stream, audiences, and analytics
- Have their preferences honored normally
- Get their data erased per GDPR on request
Holdout users do not receive:
- Any
marketingcommunication from any Adaptive Journey - Any webhook step that would otherwise fire on their behalf
Default 5%
The default is 5% — small enough that you don't sacrifice much volume, large enough that lift detections converge in reasonable time for most workspaces.
For very small workspaces (a few thousand active end-users), 5% may not produce statistically significant lift estimates within a quarter. The auto-grow rule (Phase 1b polish) will adjust the holdout percentage based on the number of monthly active users targeting the configured minimum detectable effect.
You can override the default per platform via the JOURNEY_GLOBAL_HOLDOUT_PCT environment variable; per-workspace UI overrides land alongside the calibrated-impact dashboard.
Tradeoffs
| Higher holdout % | Lower holdout % |
|---|---|
| Faster lift convergence | More users exposed to your journeys |
| Smaller sample for the optimizer | Larger sample for the optimizer |
| Easier to detect small effects | Risk of underpowered impact estimates |
5% is a common-case sweet spot. If you're running journeys you're confident produce a >5% lift, the default is comfortable.
Auditing the holdout
Each journey's detail page in the dashboard includes a Holdout audience panel that lists the subjects currently held out for that journey-version. The list is the auditable view of your control group:
- Every row is a subject who matched the journey's trigger but was suppressed by the workspace holdout
- Records are TTL'd to
triggeredAt + goalEventWindowDays + 24h grace, so the panel reflects the currently-attributable control population, not the all-time history - Email + name are hydrated from the contact store when available; users who exist only as anonymous
endUserIds show as— - A CSV export ships the loaded page so you can hand it to compliance or marketing-ops without an extra round trip
The panel hides itself on journeys that have never published (no version, no firings) so it doesn't show empty chrome. On a published journey with no current holdout records, you'll see a "no one is currently held out" note alongside an explanation of the lifecycle.
Holdout vs. Adaptive Branches
These measure different things:
- An Adaptive Branch measures arm A vs. arm B within users who reached the branch
- The global holdout measures journey-exposed users vs. anyone-not-exposed across the entire workspace
You need both. Adaptive Branches optimize which variant of a journey works best. The holdout tells you whether the journey works at all (vs. doing nothing).
Related
- Calibrated Impact — how the lift number is computed
- Adaptive Journeys — what produces the exposed group
- Beliefs — where the resulting impact estimates flow