Ever wonder if your feature actually moved the needle, or if that metric bump was just seasonal variance?
Most teams struggle with this because they can't isolate the real impact of their changes.
That's where holdouts come in - and why we just shipped them in GrowthBook.
What are holdouts?
Think of them as your experiment's control group that persists even after launch. While 95% of users get your shiny new feature, 5% stay in the original experience. This gives you a clean baseline to measure against.
🪝 Catch delayed effects - Some changes take weeks to show impact
⚖️ Measure cumulative impact - Stack multiple features and still understand individual contribution
🤝 Build trust in your data - Show stakeholders exactly what drove that revenue increase
We built this because our community of 5,000+ engineers kept asking for better ways to prove impact. One customer told us: "Holdouts helped us discover our 'successful' feature was actually causing a 2% revenue decline after 30 days."
The best part? It's all open source, warehouse-native, and transparent - you can audit exactly how we calculate everything.
Want to see the implementation details? Check out our deep dive: https://blog.growthbook.io/holdouts-deep-dive/
GrowthBook