gloriouscowSeeing that I have a egui web application that is essentially a giant canvas element, making it an opaque blob to all screen-readers, I started looking into why all the smart people working on web technology never came up with a solution to provide some sort of metadata for what a canvas is displaying.
But people did come up with solutions, like this proposal for Canvas hit testing.
https://www.w3.org/wiki/Canvas_hit_testing?utm_source=chatgpt.com#Change_Proposal
From 2012. That never went anywhere.
I don't blame egui for lack of trying. The irony is that an immediate-mode GUI has every single bit of data needed to emit context about where its windows are and what text is showing and how to interact with it, and it can produce that metadata accurately on every single frame.
There's just nowhere to send it.
Now, I can blame lack of foresight, but perhaps nobody really thought that the canvas element would be much more than a toy, and I doubt anyone would have considered you'd be rendering an entire window manager or booting Windows XP in a browser tab a decade ago.
but there's a whole giant area in between where all the titans of technology and the stewards of the web just basically considered all the possible uses of the canvas element, then just shrugged and said "eh"
Marsh Ray@gloriouscow I don’t know about canvas specifically, but often limitations on the ability to read content trace back to Netscape’s approach to web page security (“there is none”).
In 2012 browsers allowed any random web page (perhaps one modified in transit by your ISP) to load any other (eg your bank’s account summary page) in an iframe and the browser would happily supply your bank login session cookie for authentication. Web apps commonly relied on this ability.
If random web page was allowed to read contents, (perhaps via CSS, WebGL, or who knows what other APIs) that would obviously be a disaster.
Per Vognsen@gloriouscow AFAIK egui nowadays supports AccessKit so it should at least be prepared to support that but I don't know what the current situation is for either web standards or AccessKit's support of them. Is there really no web accessibility API at all?
@pervognsen yeah, AccessKit is only on native targets.
@pervognsen Accessibility on the web just assumes you are going to be using the DOM.
you can find these relics of attempts to add some kind of accessibility to canvas - here's a few proposals from 2010
@gloriouscow I'm now imagining you could sync some truly horrific shadow DOM with just enough content and structure for accessibility interop. AFAIK that's more or less what AccessKit is already doing to offer an immediate-mode API that interoperates with native retained-mode accessibility APIs.
@pervognsen I had a similar train of thought - like if frameworks like React and Angular have made thousands of careers out of dynamically mangling the DOM in realtime then surely you could probably hack something up to slide hidden, tagged divs and spans around on top of your egui windows but I also fully realize that is going to be an absolute horror show if you want it to work reliably in more than exactly one browser.
Curtis Wilcox@pervognsen
That's basically what Google Docs does.
https://workspaceupdates.googleblog.com/2021/05/Google-Docs-Canvas-Based-Rendering-Update.html
@gloriouscow
Interesting, yeah, they're doing something there. I'm not quite sure what they're doing and it may just be the content flow, which is easier for a document than say a UI where spatial orientation is more important.
