Police forces stop using AI to create court statements after invented facts sent people to prison | Paul Walsh posted on the topic | LinkedIn

Microsoft Copilot is inventing facts that police forces across England and Wales are using in sworn statements that send people to prison. Because of this, the new £115 million unit Police.AI has ordered them to stop immediately. Officers were feeding their notes and recorded interviews into Copilot and other AI products and letting the software turn them into finished court statements, before anyone tested whether the tools were safe to use. A court statement is evidence given under oath, and a criminal court can only convict on it if it's sure beyond reasonable doubt. When Copilot adds a detail that never happened, that invented detail is added to the statement and it looks identical to a fact the officer witnessed. The officer signs it, the court trusts it's true, and then the jury convicts on it. This has already happened when West Midlands Police asked Copilot for background before a football match, and Copilot described a game between West Ham United FC and Maccabi Tel Aviv FC that had never been played. That made up match went into an official intelligence dossier, and the dossier was used to justify banning Maccabi Tel Aviv supporters from their fixture at Aston Villa Football Club last year. Real supporters were barred on the strength of a game that never happened. The Chief Constable originally told Parliament that his officers had not used AI. He admitted weeks later that they had, and he quickly retired after the Home Secretary stated she had no confidence in him. Alex Murray, the head of Police.AI, says forces must have policies telling officers to check everything Copilot produces. But that policy can't work. To catch an invented fact, an officer must already know it's false, and an officer who knows every fact wouldn't need the tool. At West Midlands, nobody caught the fake match, and the chief constable didn't even know AI was used until after he'd told MPs it hadn't been. A made-up sentence reads as convincingly as a true one, so an officer skimming for tone will just pass it through. The damage runs both ways. A defence barrister can now ask if AI wrote any part of a police statement. Any statement that can't be cleanly accounted for is worth less to a jury, letting guilty people walk. Furthermore, any conviction already secured on an AI written statement is now open to appeal. The West Midlands situation shows the best case, the one where someone happened to notice. Real football fans got banned and a chief police officer lost his job, over a single made up AI fact. | 47 comments on LinkedIn