robb4d ago
Marcin Wichary

I’m curious: Did people/apps ever attempt to bundle alt text with the image as metadata? (Like EXIF?)

That feels like the right place to put it, so then it could be read by apps/browsers, and also travel with the image on save etc. Rather than specifying it as out-of-band text like we do today.

Apr 9 at 11:37amWeb
Show threadMarcin Wichary3d ago
Thanks to everyone for responding! I’ve learned a few new things.
Show threadHugo Mills4d ago

@mwichary The answer I've seen before on this is that alt text is very much context-dependent.

For example, an alt text describing the relationships of the people in a 19th-century photo will probably miss everything about important about the historical clothing styles, or about the photographic process and how it affects the resultant image.

Show threadMarcin Wichary4d ago
@darkling Oh, interesting!
Show threadJanne Moren4d ago

@darkling @mwichary
Presumably you can just rewrite the alt text when you reuse the image, just like you would write a new alt text today if you need to.

The benefit here is that you would not lose the context of the earlier text the moment you copied the image.

Show threadDavid Cohen4d ago
@jannem @darkling @mwichary Except if the EXIF is stripped through processing. A lot of online processes do that (either inadvertently or deliberately).
Show threadMarcin Wichary3d ago
@jannem @darkling Yeah, I agree with this.
Show threadDan Burzo4d ago
@darkling Yes, that seems to be the main argument against collocating alt text with the image. Additionally, it sounds impractical when the image is used in a multilingual context. @mwichary
Show threadMarcin Wichary3d ago
@db @darkling Fonts have a capability to specify additional names/metadata per locale. I wonder if something like this could solve this problem.
Show threadshork4d ago
@mwichary EXIF has an ImageDescription field that seems to be specifically for that purpose. I don't know if anything uses it
Show threadshork4d ago
@mwichary I recall seeing images retain alt text information when using the share sheet in Android but I can't find any info on it so take it with a grain of salt
Show threadMarcin Wichary3d ago
@noiob Thanks!
Show threadEthan Marcotte4d ago
@mwichary I think one challenge is that alt text is best when it’s contextually tuned; it’s not static. (The AFB did a great job explaining this, though it’s in the context of a bigger piece on LLMs: https://afb.org/blog/entry/alt-text-age-ai)
Beyond Alt Text: Rethinking Visual Description in the Age of AI | American Foundation for the Blind

The American Foundation for the Blind
Show threadMarcin Wichary3d ago
@beep Thanks! I worry that some of these arguments are letting perfect be enemy of good, though…
Show threadMarcin Wichary3d ago
@beep (I don’t mean the AI stuff, mostly just expecting alt texts to be immaculate.)
Show threadEthan Marcotte3d ago
@mwichary Oh, I don’t think there’s any such thing as immaculate alt text! I thought Eric did a good job outlining some of the potential drawbacks/harms, though: https://ericwbailey.website/published/thoughts-on-embedding-alternative-text-metadata-into-images/
Thoughts on embedding alternative text metadata into images

The idea of “solving” alternate text descriptions by automating them away so that they are not a consideration is a bad frame.

Show threadMarcin Wichary2d ago
@beep Thanks! I think the key difference is this maybe: “Not having the ability to adjust this baked-in content—or be educated that its quality is even a problem in the first place—means it is far more likely that the same low-quality content will be passed around.” In my head, the UI to adjust text exists as today, just the information (alt text) is more resilient.
Show threadMarcin Wichary2d ago
@beep In other words, this feels like a classic challenge that people think in very abstract systemic ways, but do not consider how user experience can factor in and improve the system or the usage.
Show threadMarcin Wichary2d ago
@beep I disagree with a lot of what this author wrote, but appreciate you sharing this!
Show threadScott Kellum2d ago

@beep @mwichary I see this as a “yes, and” kind of feature. I would love to see a “content cascade” where alt text can be in EXIF data, and then overwritten with alt text. Like with CSS, style attributes override the stylesheet; alt text can override EXIF data.

“The process of adding a new category of metadata to even a single image standard is a Herculean undertaking.”
— EXIF is a cross-file-type standard that exists, and adding one field seems very doable to me. Font specs regularly add to their naming tables; I don’t see why image specs can’t.

I see this as another tool to help with some workflows, not a change to how we do things. Adding alt text to EXIF data seems like a useful tool to have in addition to the wonderful tools we already have.

Show threadMarcin Wichary2d ago
@scott @beep Yes, all this is how I feel, too.
Show threadEthan Marcotte2d ago

@scott @mwichary One point I take away from Eric’s post is that the surface area for any given image is much, much more vast than any one medium: overriding EXIF data with alt text isn’t going to help someone who’s, say, sharing an image over text.

(I’m not even going to get into the very good points he raises about abuse and localization, which have basically haunted me since I first read the piece.)

Show threadNorm Tovey-Walsh4d ago
@mwichary That's how I do it. My workflow for publishing an image taken with my phone is: capture, edit, augment EXIF, push to S3, push to my photos site. My downstream weblog and Mastodon posting tools use the EXIF caption (IIRC) field as the alt text.
Show threadMarcin Wichary3d ago
@ndw Yeah! I like this.
Show threadSmoljaguar4d ago
@mwichary https://shkspr.mobi/blog/2023/07/should-you-embed-alt-text-inside-image-metadata/ @Edent has written about this a few times
Should you embed alt text inside image metadata?

Not everyone can see the images you post online. They may have vision problems, they may have a slow connection, or they might be using a text-only browser. How can we let them know what the image shows? The answer is alt text. In HTML we can add a snippet of text to aid accessibility. For example <img src="monalisa.jpg" alt="A painting of the Mona Lisa."> Most social networks will let users…

Terence Eden’s Blog
Show threadMarcin Wichary3d ago
@Smoljaguar Thanks! While I don’t disagree with aspects of this, it feels like a strange reason to say “no.” E.g. saying “yes” if you can still contextually override/change alt text as needed (as today) seems like it would be a net positive to me.
Show threadGeoff 🏴󠁧󠁢󠁳󠁣󠁴󠁿4d ago
@mwichary Main problem is that social media platforms convert images into their own preset formats for storage, losing metadata in the process. Could be done with the MakerNote field in EXIF, but some manufacturers use that for other things. Alternatively, you can zip the alt text and cat it onto the end of the image. Image will work as normal, but change the file extension to .zip and you can recover the alt. Can trigger security issues though, as the filesize won't match image size.
Show threadMarcin Wichary3d ago
@_thegeoff This feels eminently solvable, though?
Show threadGeoff 🏴󠁧󠁢󠁳󠁣󠁴󠁿3d ago

@mwichary Absolutely, just as we have more than enough charitable money in the world to make sure nobody is ever hungry again.

Best option I can see is for the JPEG/W3C lot to introduce a new standard specifically including alt text, with all of the main social media and blogging platforms signing up to it, so it gradually becomes the default.That's easy to suggest though...

Show threadMarcin Wichary3d ago
@_thegeoff :nods:
Show threadAndre Louis4d ago
@mwichary It must be doable because I've (sent and received) photos from my iPhone to other people that can read the data as part of the comment field or something.
I'm a musician, not a photographer so my understanding of metadata is a bit different, but yeah I've seen it done.
Show threadcurche4d ago

@mwichary
There's an open feature request tracking the same

https://github.com/mastodon/mastodon/issues/14903

Pre-polulate the image alt text field from the description field in EXIF metadata · Issue #14903 · mastodon/mastodon

Pitch When a user is uploading an image file, any text from the Description field in the EXIF Metadata could be pulled to pre-populate the alt text field. Motivation As the description field serves...

GitHub
Show threadcurche4d ago
@mwichary
(I subscribed to it when I saw a similar discussion a while back, sorry I don't remember the toot)
Show threadBITNACHT4d ago
@mwichary The Caption/Abstract field in the IPTC-IIM EXIF-header is just what you are looking for. I don‘t think that the contextual argument holds. If anything you can still tailor the EXIF in the UI before sending.
Show threadNaomi Stringfield3d ago
@mwichary In addition to what others have said, SEOs used to stuff keywords into alt text and some probably still do.
WordPress lets you store alt text with an image in their media library but then I'm not sure what it does with it.
Show threadMarcin Wichary3d ago
@naomi People abusing alt text wouldn’t materially change whether alt text is inside or outside the image, though? Unless maybe I’m missing something.
Show threadNaomi Stringfield3d ago
@mwichary Could have been me missing something, but I would think that alt text would depend on what page one were trying to drive traffic to, and thus would vary depending on what HTML it was being used in.
Show threadMarcin Wichary3d ago
@naomi Oh, I understand. Thanks!
Show threadNaomi Stringfield2d ago
@mwichary thanks to YOU for making me feel smart. I assure you I am a novice at web development :)
Show threadEli the Bearded3d ago

@mwichary

I think Flickr does, and I do for my personal files, but its frustratingly not widespread and even finding image management software that supports this is difficult.

(I use F-Stop Gallery (paid) for Android, Zenphoto for self-hosted Linux gallery, and Perl scripts with exiftool for various utility tools.)