Mastodawn

grumplestiltskin Feb 27

A couple of months ago a family friend had a very serious health issue and he couldn't move or speak much. So I put together a web app with a set of phrases, connected to a game controller, in a way that he could just select phrases from the list to communicate. Luckily this person got better quickly, and this app was no longer needed, but I decided to improve this experiment and publish it as an Open Source project.

So, this is VoxEase. It can be operated with a mouse, a touch screen, a computer keyboard, a game controller using a single hand, or you can set it to scan the list of phrases automatically so you only need to press one button to pick your phrase.
It only requires a modern browser and once downloaded it works offline.
It supports multiple languages and it can also be used by people with sight impairments (it works with screen readers).

Any suggestions on how to make it better are welcome!

Link: https://turisc.github.io/voxease/

kat donegan 🇵🇸✊Feb 25

@turi this is the sort of tech we want & need. thank you for reminding us that it doesn't *have* to be obnoxious. it *can* be for good.

André Polykanine Feb 25

@turi wow, this is… generous, and it's the least I can say. And caring about screen readers is hugely appreciated.

Anthropy Feb 25

@turi this is neat! nice work

playing with it a bit, I was thinking as suggestion that maybe if you touch/click/press the text at the top that displays the last message, you can make it speak the last message again, as that would make it easier to repeat a last typed message or any other message

@anthropy great idea, I'll implement that!

👩‍🦯The Blind Fraggle Feb 26

@ElyseMGrasso @turi What a great idea! Thanks for sharing!

Mx. Luna Corbden 🐸Feb 26

@turi Is there support for deaf people? Does it read the selected text aloud?

@corbden It does! Even your custom typing if you use it on a device that can handle text-to-speech.

Dave Rahardja Feb 26

@turi You’re a good man.

oscarfalcon Feb 26

Brilliant!

See this is what technology is for, and who it is for: we humans!

Love it, Bravo good sir!

@turi no Alt Text on image. Lack of #accessibility in posts may mean members of disability communities are less inclined to engage, no matter how well intended your offering

Misuse Case Feb 26

@Kay @turi It’s a video

@MisuseCase @turi I can see a fuzzy image I can't read. Why no description?

@MisuseCase @Kay I'm slowly catching up with messages in chronological order. Mystery solved! Yes, it's a video. It's less than a minute long and it quickly showcases what the app does.

@Kay forgive me but I'm not sure I understand which image you're referring to. I always take time to add manual alt tags to every single image I post here, but maybe I missed one?

@turi the image in the message I replied to. Someone said it was a video link. I saw enough to see a fuzzy image.

@turi On a mobile device that popover makes the page unusable. Also, the headline probably shouldn‘t block the upper half of the screen.

@turi It could also make sense to resort the predefined messages based on importance. E.g. I don‘t understand why basic needs are somewhere in the middle but greetings and politenesses are easier to access at the top.

@weizenspreu thanks for the feedback. I could make the phrases list sortable. The reason why they are sorted like this is purely because initially I only had the non-emergency one for my friend (he was supervised), and the rest was added when I decided to make the app more generic.

@weizenspreu @turi For me it's literally half the screen. The banner at the bottom is not fully readable (had already closed it in the first screenshot). Firefox on Android and Chrome on Android. 360*780 pixels at a device pixel ratio of 3.

@Shepard @weizenspreu yeah, my bad, I did not explicitlytarget such small screens. It should be fairly easy to add a new breakpoint and lay out things more nicely for smartphones.

@weizenspreu fair points. The disclaimer is something I added last minute (and it shows), I can definitely make it less intrusive. I should also tweak a few breakpoints for better display on very small screens!

Alexandro Lacadena 📷Feb 26

@turi outstanding! Thank you for the work!

Luistxo Fernandez Feb 26

@turi fantastic, congrats. Suggestion: double language settings, interface & output. So someone can go to Germany, type options in Spanish, and the voice speaks German.

It would be easy for the pre-defined basic messages (phase 1), but require more development to work with typed input (phase 2, but surely there are resources to integrate automated translation there).

cc-ing this to Basque voice / translation hackers @urtzai and @xezpeleta to check if our language might be added

@luistxo @turi @urtzai Hi Turi!

Great project, congrats! I’ll share it with my vocational training colleagues, as it could be very helpful for students with speech difficulties.

I’ve also opened a pull request adding Basque support. I noticed big differences in voice quality across browsers and OS (tested on Android + Chrome and Firefox + Linux). Hopefully this will improve as the Web Speech API matures.

Thanks for making it open source!

@xezpeleta @luistxo @urtzai yes, text-to-speech quality varies wildly even within the same backend. With Linux there are multiple options, like Pico TTS, Piper, Mimic3, but language support is uneven.
I'll have a look at your PR, thanks!

mystixa 🎶🎧Feb 26

@turi Good work. From a software development point of view may want to try relying on it for your own communication for a bit. I imagine you would find out real quick that having 'I'm in Pain' and 'I need help' should be toward the top and more quickly accessible than 'please' and 'thank you' which are nice to have but aren't communicating anything urgent.

@mystixa @turi adding the ability to reorder the tiles somewhere in settings might be a help.

@mystixa you're right, I haven't tested the app outside of a very narrow scope. I'm now thinking of making the entries sortable.

My goodness that’s brilliant. It would be very welcome in the ICU and rehab communities I used to work in. I hope it gets the attention it deserves.

Thank you for a great contribution.

Jon Yoder Feb 26

@turi I know naming is hard, but you may want to consider a different name. It's kind of hard to say and may not be easy to communicate to other people in everyday contexts. Just a thought.

@jonyoder the one I went with was very low on my list of candidates - everything else was taken! But I was counting on its tagline ('Communication Aid') to do the descriptive legwork.

MostlyTato Feb 26

@turi
This is brilliant, good job. And good suggestions that I can't really add to much
Except maybe the more statements the better, maybe in a tiered system of need.
👍

Sebastian Lasse Feb 26

First of all, thank you for creating humanism preserving FLOSS.

My suggestions on how to make it better, is simpe but depends on resources and time:

my suggestion would be : make it federated :)
ActivityPub is the protocol we speak here and it could be sets of ActivityPub Questions. Think of the "Poll" in mastodon but any Question can be open or "anyOf" as well.

We do similar things for fedi, described it here in this unfortunately mixed language doc https://codeberg.org/Menschys/fedi-codebase
It will become 2 documents later, menschys is our app for mutual help …

fedi-codebase

meta-repo keeping track of the unified codebase for the ActivityPub Clients menschys. taxiteam and redaktor.

Codeberg.org

DerPumu (he/him)Feb 26

@sl007 @turi how would it benefit from being federated? My understanding is that it's a single user app meant for offline use.

@sl007 I'll have a look at this but I never thought that a static app could make a good federated asset.

@turi can I translate to Greek so that it it added?

@stavpup  Ναι, παρακαλώ! You can send a PR or just send me the strings. Not sure how I'll handle the credits - they might just take the form of comments in the source HTML.

@turi can I have the source file ?

@stavpup here's its repo: https://github.com/TuriSc/VoxEase/

GitHub - TuriSc/voxease: Communication Aid – A web application for augmentative and alternative communication (AAC) helping individuals with speech, sight, or language impairments communicate using pre-defined phrases or custom messages.

Communication Aid – A web application for augmentative and alternative communication (AAC) helping individuals with speech, sight, or language impairments communicate using pre-defined phrases or ...

GitHub

@turi
https://pastebin.com/SMHJvXDr

voxease_greek - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

Pastebin

Jake in the desert Feb 26

@turi this is SO cool - what a great idea!!

Benjamin Feb 26

@turi As sad as needing this is, it's so great to have it! Thank you very much for making, and releasing, this software!

PaulaToThePeople 😷Feb 26

@turi Thanks. I'll try this with my client on her eye-tracking operated device.

@PaulaToThePeople wow let me know how it goes and if there's anything that could be improved. I guess you might want to try hovering mode, but I'm not sure how that device operates.

Jeff McNeill Feb 26

@turi 😍

Rebecca Legowski Feb 26

@turi this is a really cool app. Could prove really useful for people who are struggling to speak for whatever reason.
I tested it out with NVDA and it all works fine. I like the custom messages feature. Just struggling to find where I can download it? There doesn't appear to be a download button or link on the webpage.

@rlegowski1 it's a single, self-contained html file, so if your browser has a "save webpage…" function you can just use that. You can also download it from the link at the end of the disclaimer: https://github.com/TuriSc/voxease

GitHub - TuriSc/voxease: Communication Aid – A web application for augmentative and alternative communication (AAC) helping individuals with speech, sight, or language impairments communicate using pre-defined phrases or custom messages.

Communication Aid – A web application for augmentative and alternative communication (AAC) helping individuals with speech, sight, or language impairments communicate using pre-defined phrases or ...

GitHub

Rebecca Legowski Feb 27

@turi OK cool, thanks.

@turi This is amazing, thank you so much! I can offer Dutch translations, if you'd like?

@jay9 absolutely! Send a PR or send me the translated strings.

Abram Kedge🏴󠁧󠁢󠁳󠁣󠁴󠁿🇨🇦 wryly Feb 26

@turi I was thing about this exact problem this morning! PLEASE add "That would be an ecumenical matter" as one of the options. Endlessly useful.

@turi what a great idea, thanks for sharing.

@turi I'd need to actually use it for a bit to give in-depth suggestions, but a thing that directly came to mind was to perhaps sort the options on collapsible categories? Could be as simple as a summary/details per category.
Things like "I need" (help, water, medicine, etc), "greetings" (hello, goodbye, how are you), "interjections" (yes, no, maybe, why, how, okay, stop, continue).
Perhaps not as necessary atm, but if you were to add more phrases it would soon help find the right response. Though you might end up with duplication of phrases.

Also, I could see a bar with most used / most recently used phrases become useful too if you are a regular user.

@FiXato I'm definitely going to make the items sortable. Good call on categories, I'll see how I can do that without breaking the grid.

@turi best of luck!

penguin42 Feb 26

@turi Very nice! Have you seen the old system called 'Dasher'? I don't think it's maintained any more but it was a very powerful system which let you any phrase with a joystick or similar; it might give some ideas:

https://www.bltt.org/software/dasher/
https://www.inference.org.uk/dasher/DasherSummary.html

Dasher On-Screen Keyboard

@penguin42 I didn't know it but I'll have a look, thanks!

Gentleman Programmer Mar 4

@turi @penguin42
For current work on Dasher see this website https://dasher.at/

I worked on the web version a few years ago. There's one or two proper AAC experts working on it too.

Dasher - An information-efficient text-entry interface

An information-efficient text-entry interface

Acratopège Feb 26

@turi thank you so much