. That quarter, the company lost $217,000. That was 53% of the revenue target for the dynamic pricing engine. The root cause was bad user acceptance testing.

The six teams test features internally. They test what they built. They test their own work. And they miss what is wrong. Bugs ship. Users find problems. Users complain. Users leave. (2/29)

Ohno attacked this by building quality in at every step. He created jidoka, which gave every worker the power to stop the line. When a worker caught a defect, production stopped. The defect was fixed immediately. It never propagated. Waste went down. Costs went down. Toyota won.

Ohno did not add more inspectors. He built quality in. He trained workers to detect defects. Workers stopped the line. Defects were caught early. They never reached customers. That is how Toyota built trust. (4/29)

For this transportation marketplace, the user acceptance testing problem is the same. The six teams inspect quality in at the end. Defects are found late. Bugs ship. Users leave. The company loses $217,000.

Ohno's principle applied to user acceptance testing is straightforward. Build user feedback in at every step. When you do, you catch usability problems early. When you catch them early, you reduce waste. When you reduce waste, you lower costs. When you lower costs, you win. (5/29)

## The Core Principle

The best way to create meaningful user acceptance testing is to stop testing internally at the end. Stop hoping the six teams will find all the bugs. Stop the cycle where teams test their own work, miss what is wrong, ship bugs, overwhelm support, and lose users and revenue. (6/29)

Instead, build user feedback in at every step of the development process. That is what Ohno did in manufacturing. He created jidoka so every worker could stop the line and catch defects immediately. He did not build Toyota by adding more inspectors at the end. He built it by training workers to detect problems early.

For this company, the answer is the same. Build user feedback in at every step. Catch usability problems early. Reduce waste. Lower costs. Win. (7/29)

## Four Steps to Apply the Toyota Production System to User Acceptance Testing

1. Build Quality in at Every Step

Ohno added checkpoints at every step of the manufacturing process. Defects were caught early. They did not propagate.

Do the same thing. Create a user feedback checkpoint at each phase of the development process. Design. Prototype. Build. Pre-release. Users should be involved at every step, not just at the end. (8/29)

The second checkpoint happens at the prototype stage. The third at build. The fourth before release. Each checkpoint has a similar structure. Each one has users. Feedback is continuous. Problems are caught early.

Last quarter, the company created these four checkpoints. It took four hours of effort. Twenty-three usability problems were caught before they shipped. That saved the company $89,000. (11/29)

For a Crystal team of 50 plus, these checkpoints should happen at each phase. They should involve real users. They should be created this week.

2. Every Worker Can Stop the Line

Ohno gave every worker the power to stop the line. Every worker could act. Defects were caught immediately. They never shipped. (12/29)

Give every team member the same power. Create a user feedback escalation rule that allows any team member to halt a release if user feedback indicates a critical usability problem. No feature should ship with a critical usability issue.

The Crystal Product Owner should create this rule. It has three parts. (13/29)

The second part defines the escalation. When a critical problem is found, the team member reports it. The Product Owner is notified. The Product Owner assesses and decides. If the problem is confirmed, the release is halted. The feature does not ship.

The third part defines the fix. The team fixes the critical problem within three days. The delay is minimal. The feature ships with the fix. Users never see the problem. (15/29)

Last quarter, this rule was used twice. Two releases were halted. Two critical usability problems were fixed before they reached users. That saved the company $64,000.

For a Crystal team of 50 plus, this rule should allow any team member to halt a release. It should clearly define what a critical usability problem is. It should be created this week.

3. Catch Defects Early (16/29)

Ohno caught defects early by involving people closest to the work. Defects were found early. They were cheap to fix.

Run a user feedback loop every two weeks. Bring real users into the development process. Let them test work in progress. Get their feedback. Find usability problems before they become expensive.

The Crystal Product Owner should run this loop. It is a recurring session every two weeks. It has three parts. (17/29)

The second part is test. Users test work in progress. They see features before they ship. They give feedback. In one session, the team showed a new shipment tracking feature. A carrier said, The map is too small. I cannot see the route. That was a usability problem caught before shipping.

The third part is act. The team adds the feedback to the backlog. It gets prioritized. It gets fixed. The product gets better. (19/29)

Last quarter, this loop was run six times. Forty-eight users tested. Thirty-one usability problems were found and fixed before shipping. That saved the company $78,000.

For a Crystal team of 50 plus, this loop should happen every two weeks. It should bring real users. It should test work in progress.

4. Reduce Waste and Lower Costs

Ohno reduced waste by continuously improving the process. Lower waste meant lower costs. Lower costs meant winning. (20/29)

Run a feedback review after every release. Review the user acceptance testing process using three metrics. Improve one thing per month. The process gets better every month.

The Crystal Product Owner should run this review. It is a twenty-minute meeting after every release. It has three parts. (21/29)

The second part is improving one thing per month. In month one, the team improved the build checkpoint by adding more users. That increased the number of problems found before release from seven to twelve.

The third part is planning the next improvement. The Product Owner identifies the next thing to fix. The team always has a queue. (23/29)

Last quarter, three reviews were done. Three improvements were made. The number of problems found after release dropped from four to one. User satisfaction tracking was added. The process got better. That saved the company $56,000.

For a Crystal team of 50 plus, the review should use three metrics. It should improve one thing per month. It should run after every release.

## Closing (24/29)