Mastodawn

@mia I'm working on this today, in fact: https://github.com/benwbrum/fromthepage/issues/5188

The general principles I'm trying to follow are these:
* For human-visible page transcriptions (i.e. WebAnnotations on IIIF manifets):
* If both human-created and ai-created transcripts exist, only show the human-created one.
* If only the AI created transcript exists, show it with a warning prepended to it in bold
* For API access, provide both versions with stanzas indicating provenance, links to prompts, profiles for models, etc.

@mia A big problem that I'm running into is that since I don't control the client, I'm stuck with whatever implementation you find in e.g. Mirador. So although it's technically possible to use a `generator` attribute in a WebAnnotation that semantically defines the act of using the AI software, Mirador will ignore it (as well as the `creator` encoding the person who ran the AI).

So "doing it right" according to standards means doing it wrong according to best practices for AI transparency.

Mia Feb 9

@benwbrum I wonder if the Mirador community are thinking ahead to that? Or it could be a good place for IIIF and AI4LAM folk to discuss

@mia The Mirador people are lovely, but it seems like the standard answer to "I'd like to show the name of the person who wrote this commenting annotation" is to build/install a plug-in.

It's kind of a downside to having a lean core system that's modular and easy to extend--all good things--that some features people think are essential will be extensions.

Mia Feb 9

@benwbrum I guess as long as it's in the manifest and underlying data store, there's only so much you can do about what others do with it

@mia But some of us have proposed a Birds of a Feather in NL to discuss this issue.

@benwbrum @mia We'd be happy to join as we plan/hope to attend this year. Maybe we could discuss and crowd-write some burgeoning good enough practices?

@DavidH That sounds fantastic. After this Thursday's webinar, I should have some examples ready to test. (Unfortunately none from the Swedish National Archives, since they are only using our field-based transcription and we don't have an AI integration for it yet, but if you'd like to experiment we could make that happen.)

@mia We do, in manifests and ALTOs. Though we can't do it fully as we would like to due to limitations in the ALTO-format.

In the manifests and UV-viewer we show a field Transcription method and Confidence value. Those are custom fields though and won't show up in other IIIF-viewers, I guess.

(We'll also be changing the way we calculate the confidence value this year, paper hopefully forthcoming at ICDAR. )

Mia Feb 9

@DavidH ooh, nice! I don't suppose you could post a comment on my blog post? Or I can copy and paste it

@mia Please do copy and share.

I can add that when we implemented this we couldn't really find any good cases or documented practices so we did what we could.

What we would like have included in every ALTO- file is whether it’s been transcribed or corrected by a human or ATRed. If ATRed then using which models and model version and which software. And the version of the ALTO-file as we expect to re-ATR and re-publish the same pages many times as model output quality progress.

@mia Including multiple variant transcriptions per word and confidence value per word would be good. We haven't found a way to include that in ALTO-files though.

I think for that reason the National Archives of Hungary have created their own transcription output format, in JSON.

Nina Rao Feb 12

@mia For AV captions/transcripts we're using the FADGI guidelines and creating them using https://github.com/ninarao/webvtt_metadata

We're also working on HTR for manuscripts and AI-enhanced description for still images, but still figuring out how to document that, and would be interested in any standards for it.

GitHub - ninarao/webvtt_metadata: Python program that embeds metadata in WebVTT files according to the FADGI Guidelines for Embedding Metadata in WebVTT Files, using a csv template file, header data from an associated parent file, and/or a default metadata set.

Python program that embeds metadata in WebVTT files according to the FADGI Guidelines for Embedding Metadata in WebVTT Files, using a csv template file, header data from an associated parent file, ...

GitHub