@elilla I saw a project doing this for NYC council meetings, and brought up the inevitability of mistranscriptions (and bias in whose testimony would be more accurately transcribed). They pushed back saying it was fine and users could submit corrections. I decided to check whether my hypothesis was correct and mistranscriptions were common. I followed an upthread link to a transcription and started reading through to see how long it would take me to spot an error. Literally the next speaker had serious errors in the transcription.

The kicker: NYC already provides human-taken transcriptions. But these are released "too slowly" and it's "too hard" to even implement a feature to automatically check the machine transcriptions against the human ones when they're available.

The second kicker: in the video with the error, the human scribe interrupted to ask the speaker to speak louder/more clearly so that they could get an accurate transcript.

Anyways I got blocked because the project (which aside from using AI for transcription was pretty cool) was super useful for civic engagement and my objections were intolerable.

@elilla I am continuously arguing the accuracy debt angle. I have made significant strides in some areas of building ai 'bulkheads'.

Low background radiation steel is a good analogy I use. There is before, and there is after.

@elilla transcription / translation is one of the areas where I see a good use for LLMs at the moment. But, only as a first-pass.

I use Speech Note to do a first pass at transcribing audio from talks and such that I will write about. But I also go back and watch the talk and clean up the transcript -- I'm not blindly trusting the output, I'm just trying to speed up the act of typing it out and saving some wear and tear on my hands.

An LLM-generated translation or transcription that is not verified is, IMO, generally a dangerous thing. It might be fine for local use to try to get the gist of something, but no organization should be publishing those types of things without verification.

@elilla I’m a data engineer. I’ve been saying for years if not decades: “Bad data is worse than no data”. And, generally, when people hear that, they agree with me.

When I point out that genAI produces bad data, the turnaround to “oh, but, so useful”, “early days”, etc, is quick and disheartening.

@elilla

Thing is, our myths and literature have been telling us this for millennia!

*All* the oracle stories involve an oracle saying something ambiguous, which the protagonist dangerously misinterprets. It will always be mushy, you'll always choose the wrong interpretation, and it will always be your fault. In that sense, saying "you have to check the AI result" is a threat, meaning the AI is free to make mistakes, but you will be held liable.

This is not positive information; it is almost *negative* information in that we still don't know the truth, but are tempted into dangerous fantasies of misinterpretation.

We've even turned the whole mess into a cautionary tale with the "ibis redibis" story of the oracle at Dodona, a caution heeded nowadays by almost nobody:

https://en.wikipedia.org/wiki/Ibis_redibis_nunquam_per_bella_peribis