"AI tools are making potentially harmful errors in social work records, from bogus warnings of suicidal ideation to simple “gibberish”, frontline workers have said.

Keir Starmer last year championed what he called “incredible” time-saving social work transcription technology. But research across 17 English and Scottish councils shared with the Guardian has now found AI-generated hallucinations are slipping in.

As scores of local authorities begin to use AI note-takers to accelerate recording and summarisation of meetings with adult and child service users, a seven-month study by the Ada Lovelace Institute found “some potentially harmful misrepresentations of people’s experiences are occurring in official care records”.

The independent thinktank found that one social worker who had used an AI transcription tool to create a summary said the technology had incorrectly “indicated that there was suicidal ideation”, but “at no point did the client actually … talk about suicidal ideation or planning, or anything”."

https://www.theguardian.com/education/2026/feb/11/ai-tools-potentially-harmful-errors-social-work

#AI #GenerativeAI #AITranscription #SocialWork #UK #Hallucinations