“Yeah, that's how you get got in this process. Once you stop scrutinizing the model's output, the probability something goes off the rails approaches 1. "Human in the loop" is necessary, but the current process itself makes the loop stultifying, and encourages the human to take themselves out of the loop. That process is straight up dangerous. The temptation to let it rip is always there, and I didn't even have a boss pressuring me to ship code.”

https://taggart-tech.com/reckoning/