Mastodawn

I was helping an open-source app become more #accessible yesterday and we ran into the problem that it looks like #screenReader Orca has a bit of a tendency to crash when exposed to webkitGTK applications. I've not done Desktop #linux for a long time and have no idea what the workarounds for this would be, does anyone know anyone who might know what's happening here? Do we need to set a random environment variable somewhere or update a specific package so this person can do basic screen reader testing on their app? #tech #openSource #screenReaders #linux #GTK #webdev

The consensus seems to be that yes, this is broken, and no, there's not really any way around it for the moment. Also no, nobody's really fixing it.
i want to, #Linux enthusiasts, I really, really do. But it's exactly this kind of thing that make me entirely unable to consider any Linux flavor as an OS I could be productive in, heck ... that I could get light usage out of. When it comes to #accessibility stuff either breaks and stays broken for years or even longer, or never worked well to begin with. I know there's efforts on their way to fix that, and all the respect, gratitude and solidarity with the people doing that no doubt thankless, against-the-current work, but as of right now there's just not enough here for me to warrant the gigantic effort and investment to get anything usable out of it that actually stays usable longterm.

Ghostrunner 5d ago

@zersiax what exactly is wrong and where? You are declaring generalities where specifics are required to help correct.

I have 3 hours after work. I can fix one thing today. What one thing do you want fixed?

@ghostrunner I couldn't really begin to tell you. What I know is that several webkitGTK apps, when ran in conjunction with the Orca screen reader, misbehave, or crash outright. This is particularly prevalent in apps using Tauri as their runtime. I don't know why this happens, I myself don't have debug logs or crash dumps, I just know that users, like myself, were trying to run an app they made only to have it either crash the app, the screen reader, or both.
I am generalizing because in my 25+ years of being a computer user, this kind of thing is not anomalous. We're seeing it in Calamares installer accessibility issues that remained open for almost a decade, we're seeing it in the Wayland adoption where accessibility was clearly an afterthought some very talented people are now saddled with fixing, and that's just the examples I can think of by thinking for 10 seconds.
Like I said, I know people are fixing things, I just think right now, there's more people that aren't

Ghostrunner 5d ago

@zersiax I feel for you. One trick I have seen in the industry is to have docker images of hundreds of installation 'profiles' which are then tested using generalized testing tools like selenium as run through cucumber.

The problem is that with cucumber, you are either verifying a location, which is buggy and hard to maintain, or you are verifying a 'rendered object name' which sometimes isnt even visible. its... better? but not ideal.

Anyway, we could create a generalized accessibility profile that can be applied and extended to various docker profiles of people's installations. it is probably better than nothing. then folks can submit to the test suite to get a badge or some silly gamification method to ensure actual use and adoption.

Peter Vágner 5d ago

@ghostrunner I don't have enough knowledge on what exactly has to be done in order to fix it as a whole, however one long standing webkit issue that would be helpfull to move forward is: bugs.webkit.org/show_bug.cgi?i…

@zersiax It would be nice to get more details about crashes you are experiencing.
Is orca crashing in a way so it's no longer running after performing certain steps?
Or is it running and no longer reporting what's happening?

Could you either file an issue or try to describe on how I might be able to reproduce that orca crashing and file it on your behalf?

Orca issues are best filed at: gitlab.gnome.org/GNOME/orca/-/…

270354 – [AX][ATSPI] Accessible text implementation should not flatten text from child accessible objects

WebKit Bugzilla

@pvagner @ghostrunner Unfortunately I don't have a setup to test it reliably. From what I heard, it's not so much Orca that crashes but rather tauri-based web apps themselves

Peter Vágner 5d ago

@zersiax If you are working with someone else on this, can you please ask them to comment if it's an orca crashing or a different issue?

As I have said I have experienced when keyboard navigation has stuck in some unlabelled control and the only way to get the keyboard focus handling to work again is to restart the app either gnome web aka epiphany or any other tauri app such as deltachat-tauri. However during this orca is still running and switching into different windows it's responsive.

Severe orca issues like a crash are usually addressed in a timely manner so if we can find a reliable reproducer for an orca crash we can ask politely to get it fixed.

@pvagner They're telling me essentially the entire UI of the application disappears and only the application background remains visible when scertain parts of the app are interacted with. I passed on that link you gave me asking if they can comment on their experience

🏳️‍⚧️PepperTheVixen🦯5d ago

@ghostrunner @zersiax This is a deficiency in linux desktops as a whole. It would take months of work to fix. It's not just one problem, it's a whole series of problems split across different packages born out of neglect for accessibility

Ghostrunner 4d ago

@PepperTheVixen @zersiax yeah, im seeing this.

if we built a service folks could submit their builds to which will test the matrix of setups, we could start building bulwarks against the neglect.

definitely not a 'ill fix this in 3 hours' and more a 'ill think about this nastiness for 3 hours and feel real bad'

Kay Ohtie 🔜 FWA 4d ago

@PepperTheVixen @zersiax Even as someone who's sighted and (mostly) without disability, and technical, I was floundering on one distro just to get TTS besides espeak working. Thankfully far easier to solve elsedistro, but yeah, I keep seeing it being a problem and then people treating folks who are expressing broad frustration as "you're not being specific enough".

And I feel like half the problem is the people saying that are driven by problem solving only, and not learning about the entire puzzle to create their own problems within it to solve. Making disabled people do the lifting of clearly defining the technicals of their problems is, to me, a pretty crappy approach when the needs are well published already in numerous places and no one wants to learn them because they're not "technical problems".

Can't solve it yourself? Express empathy by doing what you can to put pressure on others to make it more accessible.

Speaking of, I still need to make that PKGBUILD for AUR so folks can install obs-localvocal to put captions on their stream (embed in video and/or standard subtitle closed captions in stream) more easily. Worked great once I built from source.

Jeffrey D. Stark 5d ago

@zersiax true that

🏳️‍⚧️PepperTheVixen🦯5d ago

@zersiax This is part of the reason that I main Gentoo. I don't want Linux on hard mode, but it's the only way I can make it work how I want it to work

Peter Vágner 5d ago

@zersiax It looks as compared to firefox or chromium webkit gtk has some issues when it comes to screen reader accessibility.
Sometimes the focus is trapped in some unlabelled object, accessible labels from multiline children only take first line into account.
In v 2.50.6 an important issue related to incorrect role types being reported has been fixed.

For example all the tauri based apps are suffering from webkitgtk issues no matter how much great screen reader accessibility has been put into their design.

@pvagner yep, this is a Tauri app. Is specifically that issue currently being worked on, that you know of?

Peter Vágner 5d ago

@zersiax I'm afraid no ongoing accessibility related work for webkitgtk is in in progress currently.

That being said I am not involved I'm just a linux user watching some email lists and people on mastodon and these are my guesses.

@pvagner @zersiax oh boy I really hope that's not true but with how slow Webkit bugs get resolved I wouldn't be surprised. VoiceOver is always the hardest one to rangle into submission for me especially where elements with complex parent-child relationships are involved and specific widget rules it just doesn't honor the same.

@pvagner unfortunately about what I expected, thanks for responding :)

the esoteric programmer 4d ago

@zersiax the screenreader doesn't exactly crash when working with webkit-gtk or qtwebengine, but instead, browse mode doesn't work properly because the accessibility code doesn't have support for the object replacement character, which would tell orca where in a given piece of text it should insert a link

@esoteric_programmer would that also cause the underlying application to crash/stop rendering? Thats one of the things we're seeing

the esoteric programmer 4d ago

@zersiax nope, I've never seen that failure mode. Is this packaged as a flatpak with the gnome runtime or something like that, so I can possibly do debugging? what distro? what orca version? did you change anything from the orca config?

@esoteric_programmer couldn't really say, wasn't the person experiencing it. I know it's a Tauri app, likely default orca settings, not sure about versions

the esoteric programmer 4d ago

@zersiax yeah, tauri has issues on linux because of webkit-gtk, and that's not only accessibility related. Anyways, could the person build with electron for the time being, assuming they don't use a lot of tauri specific functions? also, is that a public app? if not, does orca log anything important while that happens?

@esoteric_programmer I ... sincerely doubt they can, they're pretty invested. The app in question is OpenDeck. I really don't know much about logging etc., this wasn't a programmer tring to make Tauri work with Orca, this was a programmer seeing if they could quickly test a fix they'd made to their app, I don't think they stopped long enough to really figure out tauri internals when things fell over :)

the esoteric programmer 4d ago

@zersiax is it this?
https://github.com/nekename/OpenDeck/
and if I don't have the hardware, can I still test the stuff to see why it broke?

GitHub - nekename/OpenDeck: Linux software for the Stream Deck with support for original Elgato Stream Deck plugins

Linux software for the Stream Deck with support for original Elgato Stream Deck plugins - nekename/OpenDeck

GitHub

@esoteric_programmer ...potentially. Interacting with the action list seemed to break it for them, but that was a newer version that adds fixes to make that part of it accessible, it isn't in stable yet. You could build it from main but you'd need cargo and deno. Worth just checking with stable to see if it falls over though, don't need the hardware for that

the esoteric programmer 4d ago

@zersiax alright! I have cargo because I write rust often, I don't have deno yet :p. But sure, will try stable

the esoteric programmer 4d ago

@zersiax alright, I downloaded the appimage. It works well if one knows what one is doing which I don't really, but an issue I see is that whenever I navigate with tab past the end of the ui, orca freezes and the thing along with it. I wasn't able to see exactly why this is, I bet it's something related to an integer overflow, but otherwise the app works normally, if only I knew what I was doing and how it's supposed to work. The settings page has a lot of unlabeled sliders, buttons and checkboxes, which could either be because of webkit-gtk, or those are actually not labeled. Browse mode is fucked but that's a well-known issue with webkit-gtk for now, and the fix might be simple, might be more complicated, I dk how webkit works well enough to say one way or another. But yeah, the appimage also did some unpleasant things to my system when I ran it, it installed files too big to be configuration related in XDG_CONFIG_DIR, what looks like the content of pluggins in ~/.local/share/opendeck and a launch icon in ~/.local/share/applications, without me asking it to do that, and all those files caused my disk space to depleat. So, for further testing, I'm gonna try to either run it as a flatpak if I can find it, or build from source.

@esoteric_programmer yep, I did hear that tabbing onto a thing caused it to crash, so that might be what you're seeing as well. ANd yes, still a lot of unlabeled controls, I'm working on it ;)

the esoteric programmer 4d ago

@zersiax yeah, it's tabbing past the end of the UI that crashes from what I'm observing here. Worthy of note is that this doesn't happen with other tauri apps I tried

@esoteric_programmer I heard that apparently Dorion, another Tauri app, does this as well . never heard of it but that could be another test

the esoteric programmer 4d ago

@zersiax hmm, will look into that, never heard of it

Mike Gorse 4d ago

@zersiax @pixelate I'm really sorry that you all are running into that.

A couple of questions. Which distribution/version are you running? When you say that Orca is crashing, do you mean that it aborts and needs to be restarted, or is it only failing to read your app but responding to other applications? I think the latter would probably mean that WebKit isn't able to connect to the accessibility bus, so possibly an environment variable isn't being set/transferred properly (I assume it is inside a flatpak?) The former might mean a crash in AT-SPI. https://gitlab.gnome.org/GNOME/at-spi2-core/-/issues/219 is the one crash that I've seen reported recently, and I *think* that I fixed it in at-spi2-core 2.60.0 / 2.58.4 / 2.56.8. If you are, in fact, seeing a crash where Orca is aborting, then I would need to either be able to reproduce it or see a stack trace (if you don't know how to retrieve one, then I'll try to help).

Segmentation Fault Due to Chromium (#219) · Issues · GNOME / at-spi2-core · GitLab

Orca Version Orca version 49.5 (rev d8d6066e7), AT-SPI2 version: 2.58.3, Session: wayland...

GitLab

@MikeGorse @pixelate my initial understanding was actually wrong, it's not Orca falling over, but rather the application it's talking to. Two Tauri-based apps, Dorion and OpenDeck, exhibit crashing behavior (UI stops responding, then vanishes to only show background) when user tabs through interface with Orca running. I don't think Orca itself crashes but I wasn't there when this happened

Mike Gorse 4d ago

@zersiax @pixelate I was looking through the commit log the other day and noticed one accessibility-related crash fix post-2.52.1 [1], but I don't know if it is your crash (I know of others). You could of course try filing a bug on bugs.webkit.org if you can get a stacktrace, but I don't think that the Linux accessibility code has been getting a lot of attention lately, and I have several other things on my list of things to do and don't have a lot of time to spend on it either right now. Sorry for not being more helpful.

[1] https://bugs.webkit.org/show_bug.cgi?id=310936

310936 – AX: AccessibilityRenderObject::boundingBoxRect() is missing a null-check on a Node, causing crashes

WebKit Bugzilla

Peter Vágner 4d ago

@MikeGorse @pixelate @zersiax Although I have no compatible devices, I have built current git main version of opendeck on archlinux with gnome 49 with packages at-spi2-core-git 2.60.0.r5.g814b4d12-1 and orca-git 50.0.r137.gc96d101d4-1, webkit2gtk-4.1 2.50.6-1.

On the opendeck main window I can see these controls in the order of keyboard navigation with the tab key or the object navigation:

Initially focused document body element that is causing orca to report nothing when focused. Like the focus changes into a silence when navigating to this. This is not the main issue and my guess is that this is a webkit issue as I can see the same thing with other apps and websites.
Navigation with two buttons inside
A heading followed by two paragraphs of a text and a Restart button.
An input text entry labelled Search actions
An interactive list of actions labelled Opendeck.

When using tab key to navigate the whole opendeck app freezes when trying to move from the text entry to the interactive list. Most likelly it also changes the visual appearance as described earlier but I can't verify that on my own. When navigating in a reverse direction by pressing shift+tab navigating from the silent document body element to the interactive list is working fine.

I don't think it's related to this list as I have seen interactive lists in a different apps such as deltachat-tauri which don't cause such a behaviour. However I have seen random freezes like this with the deltachat-tauri app as well and I see this occurence of this issue as a positive thing as I can reproduce such a freeze reliably all the times with opendeck app.

Peter Vágner 4d ago

@MikeGorse @pixelate @zersiax Oh I wanted to just cherrypick this fix on the top of the latest release and test if it fixes this particular case but it does not apply, actually there are more changes to accessibility since last release.
I will try if I can build webkit easily and verify this.

Here is history of commits inside the accessibility folder just out of curiosity: github.com/WebKit/WebKit/commi…

Mike Gorse 2d ago

@pvagner @zersiax @pixelate I just installed opendeck and can reproduce the crash. I've either ran into the same crash or a similar one when authenticating a Google account in evolution, but, until now, I hadn't heard of other people running into it, so I thought that it might be somewhat rare. I agree it is bad that some apps are crashing so easily. I've filed https://bugs.webkit.org/show_bug.cgi?id=311427 for it. I don't understand the code very well, but I'll see what I can do.

311427 – AX: Opendeck crashes webkitgtk with Orca running

WebKit Bugzilla