The challenge
Halcyon’s document model improved every week in the lab, but each release to customers was a quarterly event with a war room and a rollback plan.
The approach
Each new model now starts at five percent of traffic. Nine judges score its answers against the current model, and an experiment decides when it moves up.
The result
Halcyon ships a model every week. In twelve months no customer has seen a model that failed its judges.