Social workers are facing a new challenge as AI tools are making potentially harmful errors in their records, from bogus warnings of suicidal ideation to simple "gibberish". While some are excited about the potential of AI transcription technology, research has found that AI-generated hallucinations are slipping in, causing concerns among frontline workers. The Ada Lovelace Institute's eight-month study found that AI note-takers are misrepresenting people's experiences in official care records. One social worker reported that the AI incorrectly indicated suicidal ideation, despite the client never mentioning it. Another noted that the AI's notes might refer to "fishfingers or flies or trees" when a child was actually talking about their parents' fighting. These glitches are particularly worrying as they could cause risky patterns of behavior to be missed. Social workers are also concerned about inaccuracies in transcribed conversations with people who have regional accents, with some describing the AI-generated transcriptions as "gibberish". Despite these concerns, the research also found that AI transcription delivered observable time savings and allowed social workers to focus more on relations with service users. However, when one social worker used an AI tool to redraft care documents, the system inserted words that had not been said. The British Association of Social Workers (BASW) is calling for social work regulators to issue clear guidance on how and when AI tools should be used, as disciplinary action has already been taken for failing to properly check AI outputs. While some social workers are excited about the potential of AI, others are concerned about the risks, including potential bias in report summaries and inaccurate "hallucinations" in transcripts. The BASW strategic lead for England and Wales, Andrew Reece, warns that people are not checking what's been written for them, missing out on the important parts of reflective practice. The company behind Magic Notes, which is one popular AI transcription tool, emphasizes that its outputs are a first draft, not a final record, and highlights its specialist features for social work, including automated hallucination risk checks.
