Mastodawn

I wonder how adding alt text at a high level could be exploited?

I suppose there are both harassment and security issues to consider if the next people that are using that alt text are implicitly trusting the users description.

David Gerard Dec 7, 2022

@slimepsychic the JPEG comment field *could* have been this, but nooo

@davidgerard There's so much that could be done with that.

@slimepsychic This seems like something we should've used the ImageDescription EXIF field (tag 0x010e) for. Which, granted, would require decoding and processing EXIF, which is its own nightmare, but still, it's right there and we should've used it.

@wordshaper Yes! That is a good level to implement it.

@slimepsychic Maybe it's not too late. Not every image format supports EXIF (which is fair, it's a nightmare) but we could try setting that standard now.

This'd probably involve me having to write code for Mastodon, which is certainly possible.

@wordshaper @slimepsychic

Steven Dec 7, 2022

I thought of Exif at first, too. From reading the Wikipedia page there are technical issues with Exif that result in it being regularly dropped any time a file is edited. A metadata format that is unreliable is ... problematic.

Fortunately, there is also XMP (Extensible Metadata Platform) which is also supported in more file formats. That pulls in metadata from Dublin Core which includes a standard "description" field.

In any case, there are multiple metadata formats which support this. It is just a matter of having end-user software that cares about it.

@yam655 @slimepsychic Yeah, and both EXIF and XMP have the issue where quite a lot of software intentionally drops them for safety reasons. (Which is understandable) And, of course, different file formats use one or the other.

Probably the largest issue, honestly, is many of the image processing libraries you might use drop all these metadata segments because it's so much harder to support than the actual image bits are.

@yam655 @slimepsychic I gave a go at integrating metadata support into the Go image reading libraries and got to dive into XMP, EXIF, and some other random metadata stuff that gets attached to images.

My big takeaway was it was worth nope'ing out from as too much work to do just for fun, but I didn't have anything specific (besides vague grumpy spite at the fact that metadata wasn't supported) to push it to completion.

Smackaroonies💸Dec 7, 2022

@slimepsychic imagine a metadata feature that allows you to bake alt text into the file, and it would allow the site it being uploaded to, to read it and add it automatically.

ngoomie is moving!Dec 7, 2022

@slimepsychic as someone who honestly occasionally struggles a lot to describe images, them being already described for me either by someone else or myself in the past when i'm feeling more able to do so would be super helpful -- it's like it's accessible in two different ways!

@nu Oh wow, future self would be so happy to find all of that. I try to think like that whenever I can, it can make such a difference.

I started doing this process with my images in Apple Photos when I got here in April. Adding the alt text to the comment is useful. I have to copy and paste it but I have already done the work.

Rally Vincent 🏳️‍⚧️Dec 7, 2022

@slimepsychic some formats already allow it, but no software uses it :( been delaying the scan+archival of all old family photos because I want proper image management without relying on google or amazon's deep learning things.

Ted Garrison Dec 7, 2022

@slimepsychic that would be awesome. Add alt-txt once and it's done forever, and don't have to think about it every time you post.

Денис 🇺🇦Dec 7, 2022

@slimepsychic The SVG format for vector images (with bitmap embedding possible) has title elements for accessibility: https://developer.mozilla.org/en-US/docs/Web/SVG/Element/title

<title> — the SVG accessible name element - SVG: Scalable Vector Graphics | MDN

The <title> element provides an accessible, short-text description of any SVG container element or graphics element.

MDN Web Docs

Gerard Cunningham ✒️Dec 7, 2022

@slimepsychic Presumably this could be done through exif?

mathew Dec 7, 2022

@slimepsychic
IPTC specifications include alt text as a metadata item, and IPTC can be embedded in JPEG.
https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata-2022.1.html#alt-text-accessibility

PNG supports tEXt chunks for pieces of text, but sadly doesn't officially support IPTC or EXIF.
https://www.w3.org/TR/png/#11textinfo

Happily, since 2021 XMP has supported alt text as an element -- in multiple languages, even. It can be embedded in PNG, JPEG, HEIF, DNG, and many RAW camera formats.
https://github.com/Exiv2/exiv2/issues/1959

Unfortunately, XMP is an expensive ISO standard now.

IPTC Photo Metadata Standard 2022.1

David K Dec 7, 2022

@slimepsychic there’s very strong use of EXIF / IPTC metadata in press and commercial photography to ensure correct captioning gets carried through, copyright notices, version / document management and also just for ease of management of archives and libraries. No reason similar couldn’t be much more widespread.
Would help ppl organise their photo libraries too and it’s important to be aware of what metadata is stored too - smartphones and some cameras embed a lot of GPS info!