Whenever I explain my #research at Google into mobile text editing, I'm usually met with blank stares or a slightly hostile "Everyone can edit text on their phones, right? What's the problem?"
Text editing on mobile isn't ok. It's actually much worse than you think, an invisible problem no one appreciates. I wrote this post so you can understand why it's so important.
https://jenson.org/text
#UXDesign #UX
this is a neat demo!
i'm procrastinating on reading the full article (:p) so i don't know if the answer might be there, but i have a question:
is the "harder press" for selection and action based on pressure sensitivity, and if so, how does it work without pressure sensitive screens?
edit: it is indeed pressure sensitivity. is that a common feature? without it you could just release after selection and tap the relevant action, but there doesn't seem to be a way for selection
@scottjenson a few thoughts/questions:
- The single extended gesture (tap, hold, flick) seems hard to back out of. What's required at each step to cancel out of any change?
- A general tension I feel is how x and y are treated equivalently even though they are very different in text, where y movement causes sudden large movement and x is fine grained and localized
- This keeps a very character-centric approach, which makes editing fiddly; like deleting a word and leaving two spaces
@scottjenson
I worked on voice-based editing flows, which was very difficult until LLMs came around, at which point it became vastly more feasible (but then at that moment I also stopped working in that area).
The natural language edit descriptions that you can produce with voice aren't reasonable on a keyboard. But... seems like there could be something there too. E.g., write a replacement word instead of targeting first. Or gesture to an area you want to edit imprecisely.
@scottjenson Have you had a look at the MessagEase keyboard?
I've found that swiping left/right on the spacebar to move one character at a time combined with swiping up on the backspace giving me a delete key (inverse of backspace) means that I can be imprecise with my finger taps but still very quickly do edits.
Additionally swipe-and-return on backspace deletes a whole word, which is very nice.
@scottjenson I have found I almost entirely rely on the Android/Gboard spacebar cursor for accurate text selection. I like that it’s where my fingers already are when typing, but its main problem IMHO is discoverability; I don’t think I have ever seen someone using Android know about it before I showed them.
I would love to tackle this in GNOME/GTK. It jives with the direction of making GNOME the best adaptive experience across mobile/touch and traditional computers/mouse/keyboard.
@cassidy @scottjenson Having jailbroken iOS since iOS 5-ish days, SwipeSelection habits are just ingrained in me, and in recent years the 3D Touch and iPadOS versions, but IMO that only solves 1 part of the issue which is cursor placement. On Android I specifically choose third party keyboards that have SwipeSelection built in, like FlorisBoard.
If I want to select text, I still find that I stumble on the iOS “selecting on the text area” sometimes (I double tap the start or end word and start dragging immediately), it’s either my fat finger, I’m too quick in my actions for the OS to respond in time, or iOS itself incorrectly selected text. But I’d say 70-80% of the time it’s alright. Using the iPadOS 2 finger on-keyboard cursor to select text, as well as Android (GrapheneOS) in general, just a massive headache to select text with.
IMO iPadOS does well in the text actions regard, the onscreen keyboard has the undo/redo and paste actions in normal mode, and switches to cut, copy, paste in text selected mode (visual mode? LOL). I just wish it had a “Select All” button too.
The other annoyance is deleting text on both platforms. Whole word deletions with OS native solutions are inefficient in terms of both speed and finger movement required. The most optimal solution I’ve used is Nintype (third party keyboard) on a hardware 3D Touch compatible iPhone, as I can have a normal per-letter backspace on tap, swipe from backspace to fast backspace with pretty precise stopping (this was also a SwipeSelection feature), and 3D Touch backspace to delete a whole word under or just before the cursor. This is the fastest combo of deleting text I’ve used, and I’m pretty happy with it.
Unfortunately, that (and the whole of my Nintype setup for efficient iPhone typing) relies on the hardware 3D Touch (so no new iPhones or any Androids, I’m still on iPhone X), and on Nintype, which 1. hasn’t been updated in years and it’s a lucky gamble that it still mostly works on iOS 16, 2. is buggy and not working well on Android 13 (GrapheneOS) for me to begin with, 3. doesn’t work that well on iPad since it doesn’t have a tablet layout.
On iPadOS, I suck it up with the finger travel and use the “selecting on the text area” then backspace to delete, like I said above it’s mostly reliable. On Android, I either use the swipe from backspace with FlorisBoard, or I suck it up with the bad selecting and then delete while feeling annoyed. I’m guessing this will be the same if and when I upgrade my iPhone to one without hardware 3D Touch.
@scottjenson If you haven’t looked at Nintype with a hardware 3D Touch iPhone yet as part of your study/assessment, I’d recommend trying it out… until I realize as I’m typing that it’s no longer available for purchase on the App Store and only available for redownload for users who already have it. Well, if you ever get the opportunity to, I’d go for it. Could be some valuable lessons there, it does a lot of things.
@cassidy @scottjenson in nemo we had a thinkpad-style trackpad that moved the cursor. Windows Phone also added this later (I like to think they copied our idea, although unlikely)
http://web.archive.org/web/20160406191430/http://play.qwazix.com/grog/?p=626
@scottjenson sure, welcome. I'm not sure if you can get your hands on one since the phone is about 10 yrs old. Here are some videos that I found
https://m.youtube.com/watch?v=RWSdK0D4K_Q
https://m.youtube.com/watch?v=h3PUqX413vo
This is with BB10 OS, however select bubble with granular control and cut-copy-paste has been updated quite a bit in updates till BB10.3 version
I'll send pics here if my older phone still runs (I've to check the batteries)
@scottjenson huh, I get intensely frustrated by text editing on mobile, especially on iOS (it's one thing that's actually better on Android, IMO). Thought that feeling was universal.
The hardest thing for me is pasting text. When you tap to bring up the contextual menu with the paste button, the cursor position often moves. Often takes me multiple tries to get it right.
I miss arrows on the iOS keyboard that allow you to move the cursor position left or right through the text!
@scottjenson @Brendanjones My eyes would glaze over, but for the opposite reason: I think it's hopeless. When I see something like "this game was developed on an iPad", I think, "WHY?!"
I want all the screen real estate I can get and touch screens are a tool of the Devil.
All that said, it looks like you're doing good work making the mobile experience less miserable!
Selecting text is also terrible. Trying to move the end markers around with fat meat nubs doesn’t work very well.
The default Android keyboard (Gboard I think it's called?) has an option to let you slide left and right on the space bar to move the cursor, it's a godsend.
Editing text on mobile is awful though.
@Parienve @adaliabooks @Brendanjones @scottjenson
I use Hacker's Keyboard on Android. It has cursor keys.
@adaliabooks @Brendanjones @scottjenson
Wait! What! How! Where!
I've never seen this option. My fingers are so stubby any typing on a mobile device is a nightmare. This sounds like it could be helpful.
But it doesn't seem to work that way on my Gboard, and I can't find any relevant options.
@adaliabooks @Brendanjones @scottjenson
Found it - my problem was I had two different language keyboards selected. As soon as I held the space bar, it gave me the keyboard-selection menu. I deleted one which I don't need, and now the feature works. I think this will be a big help. Thanks!
@adaliabooks @Brendanjones @scottjenson
Now I can adjust the cursor from the spacebar, I want a button that allows me to open the menu without touching the cursor.
A menu option would also be nice for anchoring the selection point so I can move the other end left and right to complete the selection before I cut/whatever. That would be great.
I also wouldn't mind being able to move off the top or bottom of the spacebar to move the cursor up/down.
Or just give me a TrackPoint and buttons.
@dhobern @Brendanjones @scottjenson
So I've just found there is actually a selection tool in GBoard, if you press the menu button (blue icon with four squares in the top left corner) there should be an icon with a capital I and two arrows either side. If you hit that you get an up and down arrow, a select toggle and copy and paste buttons.
Which will make text editing on mobile much easier now that I know it exists.
@adaliabooks @Brendanjones @scottjenson
Thank you! That's fantastic. I expect so little it never occurred to me to look for that.
Short of a TrackPoint on the bottom of the phone and left/right tactile buttons on the sides, this will do me for now.
@adaliabooks @Brendanjones @scottjenson
Even better, I've just found I can put it in the space usually reserved for the microphone icon. Fantastic!
@Brendanjones @scottjenson
For people on Android there is a keyboard that has left and right keys above the letters. Swyping up from the space key shows the functions paste at the current position, select text with left right keys, select all, undo and redo button:
#AnySoftKeyboard https://f-droid.org/packages/com.menny.android.anysoftkeyboard/
I just discovered that when long pressing on paste you can select from a list of the last copied things.
@scottjenson 🧵Text editing #UX
“…most importantly, fixing text editing isn’t seen as important enough in the war between Android and iOS. It’s not the flashy feature that shifts your Net Promoter Scores. What I find ironic is that a fundamental change, like fixing text editing, could make people feel much more at ease using their phones and could be an enormous reason to switch. But it would be a slow burn and take years of steady effort. Android just can’t think this way. Apple just might.”
@scottjenson This is fantastic. I am not surprised at all about the results of the study. Making text edits is brutal on a phone.
If you have any visual impairments, arthritis, low manual dexterity, or are out in the bright sun it is so frustrating.
Thanks for sharing. I just sent it to my teams!
@scottjenson It's why I'll write text on the Obsidian.md app on my phone, and then copy and paste it.
It's a janky workflow.
@scottjenson huh. I can see why any of these things can feel problematic, but to me all of them are like complaining that some tiles on a pedestrian path are misaligned, while there is also sinkhole the size of several cars in it.
Writing text fast and writing text without errors are two entirely separate things for me. And generally things that need copying are somewhere that can't be copied in the first place, making copying and pasting a moot point.
deepak
Scott Jenson
pranabekka
Ian Bicking
Ozzie D, NP-hard 
John Schrag
mhoye
suzietchi
Christopher
bullshitter
Martin
Arindam Basu
Matt Woelk
⁂ Fish Id Wardrobe
kravietz 🦇
Noah Mittman 
Cassidy James 
Tech Tyrant ᴶᴶᴳᵃᵈᶢᵉᵗˢ 
qwazix
Niket Agrawal
Brendan Jones
Peter Drake HAS MOVED
Jonathan Hendry
Adam Cafolla
Parienve
Resuna
mirabilos
Donald Hobern
Matt Hodgkinson
Nina🐀🌻WTAF America?
hambach18
John Gordon
Greg Lloyd
ryaninteractive
Joni Korpi
Damon L. Wakes
Debabrata Bhattacharya || SE
Joana de Castro Arnaud
Sumurai8