Craig AbbottNew blog post about my thoughts on integrating AI into screen readers: https://www.craigabbott.co.uk/blog/screen-readers-do-not-need-saved-by-ai/
Eric Eggert@craigabbott Great article. I also wonder how people think the cleanup for the clapping hands example would work in practice, especially for different situation. How do you make sure that a user is aware that the text has been fudged with?
And if screen reader vendors want to change “triangular flag on post” to “red triangular flag” or something, they can already do that by the magic of “programming”. 😂
@yatil yeah, it’s an interesting one for sure. I tried to think logically about how it might work in practice, and I could see it perhaps being useful if it was an optional command.
Like, if it reads out nonsense, then there was a shortcut key to switch to LLM mode or something. I could see that perhaps being better than nothing.
But, it still feels like it’s solving the wrong problem. The content should be accessible. The tool shouldn’t need to fudge it.
@craigabbott Sure, but filtering out emojis or repeated emojis is also something that could be a simple user setting without LLMs.
@yatil yeah defintely.
Alex R@yatil @craigabbott I definitely think there is a role for screen readers to help with things like this. I don’t think telling people ‘don’t use the red flag emoji’ is likely to have successful outcomes for people who use screen readers, which presumably is what we want. People are always going to be creative with their text in new and unforeseen ways. How to account for that is not an easy problem to solve, I guess, but things like the clapping hands and red flags are very much not new at this point.
(Of course, I do think business/gov’t should be ensuring their content is accessible to whatever the current AT baseline is)
@alex @yatil I guess, on that point, how can you guarantee an LLM enhanced screen reader will output the right content? It’s less variable than humans, sure, but probably not by a lot. I’m not necessarily saying the only way to solve this is by telling people not to use the flag emoji, I did suggest perhaps the description for the flag needs updated, as an example of an alternative.
@craigabbott @yatil 100% agree there, requiring people using screen readers to essentially filter everything online though an LLM with nothing more than a chirpy ‘AI may generate incorrect results’ would be a disaster. A few hand-crafted options for programmatically transforming informal, social network-y text might go a really long way, but I’m not sure if any mainstream screen reader provides much of anything here that doesn’t require extensive set up in advance
@alex @craigabbott One problem is that the screen reader market has been largely stagnant for a long time. I‘m sure it would be relatively straightforward for them to implement something useful, but they just don’t.
The idea to add “AI” could lead to resources for features, but if you would funnel all text strings through an LLM, it would make “halucinations” more likely and the whole experience less responsive.
@yatil @craigabbott I suppose this might just amount to a wish on my part that we didn’t have to rely on market forces to let people participate in society equitably
Dave Letorey@craigabbott
Re: “For a sighted person, this looks… fun”
For a dyslexic person this just adds extra anxiety and load.
Amazing post mate!
@dletorey thanks dude! Yeah, I guess this focused on screen readers, but that emoji example I doubt is a good experience for anybody to be fair haha!