Whenever I explain my #research at Google into mobile text editing, I'm usually met with blank stares or a slightly hostile "Everyone can edit text on their phones, right? What's the problem?"
Text editing on mobile isn't ok. It's actually much worse than you think, an invisible problem no one appreciates. I wrote this post so you can understand why it's so important.
https://jenson.org/text
#UXDesign #UX
this is a neat demo!
i'm procrastinating on reading the full article (:p) so i don't know if the answer might be there, but i have a question:
is the "harder press" for selection and action based on pressure sensitivity, and if so, how does it work without pressure sensitive screens?
edit: it is indeed pressure sensitivity. is that a common feature? without it you could just release after selection and tap the relevant action, but there doesn't seem to be a way for selection
@scottjenson a few thoughts/questions:
- The single extended gesture (tap, hold, flick) seems hard to back out of. What's required at each step to cancel out of any change?
- A general tension I feel is how x and y are treated equivalently even though they are very different in text, where y movement causes sudden large movement and x is fine grained and localized
- This keeps a very character-centric approach, which makes editing fiddly; like deleting a word and leaving two spaces
@scottjenson
I worked on voice-based editing flows, which was very difficult until LLMs came around, at which point it became vastly more feasible (but then at that moment I also stopped working in that area).
The natural language edit descriptions that you can produce with voice aren't reasonable on a keyboard. But... seems like there could be something there too. E.g., write a replacement word instead of targeting first. Or gesture to an area you want to edit imprecisely.
@scottjenson Have you had a look at the MessagEase keyboard?
I've found that swiping left/right on the spacebar to move one character at a time combined with swiping up on the backspace giving me a delete key (inverse of backspace) means that I can be imprecise with my finger taps but still very quickly do edits.
Additionally swipe-and-return on backspace deletes a whole word, which is very nice.
@scottjenson I have found I almost entirely rely on the Android/Gboard spacebar cursor for accurate text selection. I like that it’s where my fingers already are when typing, but its main problem IMHO is discoverability; I don’t think I have ever seen someone using Android know about it before I showed them.
I would love to tackle this in GNOME/GTK. It jives with the direction of making GNOME the best adaptive experience across mobile/touch and traditional computers/mouse/keyboard.
@cassidy @scottjenson Having jailbroken iOS since iOS 5-ish days, SwipeSelection habits are just ingrained in me, and in recent years the 3D Touch and iPadOS versions, but IMO that only solves 1 part of the issue which is cursor placement. On Android I specifically choose third party keyboards that have SwipeSelection built in, like FlorisBoard.
If I want to select text, I still find that I stumble on the iOS “selecting on the text area” sometimes (I double tap the start or end word and start dragging immediately), it’s either my fat finger, I’m too quick in my actions for the OS to respond in time, or iOS itself incorrectly selected text. But I’d say 70-80% of the time it’s alright. Using the iPadOS 2 finger on-keyboard cursor to select text, as well as Android (GrapheneOS) in general, just a massive headache to select text with.
IMO iPadOS does well in the text actions regard, the onscreen keyboard has the undo/redo and paste actions in normal mode, and switches to cut, copy, paste in text selected mode (visual mode? LOL). I just wish it had a “Select All” button too.
The other annoyance is deleting text on both platforms. Whole word deletions with OS native solutions are inefficient in terms of both speed and finger movement required. The most optimal solution I’ve used is Nintype (third party keyboard) on a hardware 3D Touch compatible iPhone, as I can have a normal per-letter backspace on tap, swipe from backspace to fast backspace with pretty precise stopping (this was also a SwipeSelection feature), and 3D Touch backspace to delete a whole word under or just before the cursor. This is the fastest combo of deleting text I’ve used, and I’m pretty happy with it.
Unfortunately, that (and the whole of my Nintype setup for efficient iPhone typing) relies on the hardware 3D Touch (so no new iPhones or any Androids, I’m still on iPhone X), and on Nintype, which 1. hasn’t been updated in years and it’s a lucky gamble that it still mostly works on iOS 16, 2. is buggy and not working well on Android 13 (GrapheneOS) for me to begin with, 3. doesn’t work that well on iPad since it doesn’t have a tablet layout.
On iPadOS, I suck it up with the finger travel and use the “selecting on the text area” then backspace to delete, like I said above it’s mostly reliable. On Android, I either use the swipe from backspace with FlorisBoard, or I suck it up with the bad selecting and then delete while feeling annoyed. I’m guessing this will be the same if and when I upgrade my iPhone to one without hardware 3D Touch.
@scottjenson If you haven’t looked at Nintype with a hardware 3D Touch iPhone yet as part of your study/assessment, I’d recommend trying it out… until I realize as I’m typing that it’s no longer available for purchase on the App Store and only available for redownload for users who already have it. Well, if you ever get the opportunity to, I’d go for it. Could be some valuable lessons there, it does a lot of things.
@cassidy @scottjenson in nemo we had a thinkpad-style trackpad that moved the cursor. Windows Phone also added this later (I like to think they copied our idea, although unlikely)
http://web.archive.org/web/20160406191430/http://play.qwazix.com/grog/?p=626
@scottjenson sure, welcome. I'm not sure if you can get your hands on one since the phone is about 10 yrs old. Here are some videos that I found
https://m.youtube.com/watch?v=RWSdK0D4K_Q
https://m.youtube.com/watch?v=h3PUqX413vo
This is with BB10 OS, however select bubble with granular control and cut-copy-paste has been updated quite a bit in updates till BB10.3 version
I'll send pics here if my older phone still runs (I've to check the batteries)
Scott Jenson
pranabekka
Ian Bicking
Ozzie D, NP-hard 
John Schrag
mhoye
suzietchi
Christopher
bullshitter
Martin
Arindam Basu
Matt Woelk
⁂ Fish Id Wardrobe
kravietz 🦇
Noah Mittman 
Cassidy James 
Tech Tyrant ᴶᴶᴳᵃᵈᶢᵉᵗˢ 
qwazix
Niket Agrawal