Mastodawn

Miguel de Icaza ᯅ🍉

Keeping SSH connections alive on iOS with high-fidelity terminal emulation and without draining your battery: the Story of El Preservador.

https://blog.xibbon.com/solving-background-disconnect-issues-on-ios-with-el/

El Preservador: SSHing Responsibly on iOS

With La Terminal, we strive to offer an SSH client that just works on iOS. With this in mind, I set out to make design decisions on behalf of users that would provide an experience that works out of the box while providing the best terminal emulator experience they deserve.

Xibbon Blog

Josh Rivers Jan 24, 2023

@Migueldeicaza how much of an output buffer can it maintain (if I walk away while tailing a log)? Also, will it just continue if I sign back in tomorrow? Next week?

Miguel de Icaza ᯅ🍉Jan 24, 2023

@joshrivers It used to have a limit, but I changed it recently to stop, as the buffer gave a wrong sense of background continuity.

It stays alive for 24 hours for now, on the fence about whether to make this configurable.

Miguel de Icaza ᯅ🍉Jan 24, 2023

@joshrivers These are good questions, and I should document this, so far they had just been wasting brain storage, but they deserve to be available to everyone

Jan van den Berg Jan 24, 2023

@Migueldeicaza This is mighty clever! Love it.

Miguel de Icaza ᯅ🍉Jan 25, 2023

@jan thank you for the kind words Jan!

Jonathan Moosekian Jan 24, 2023

@Migueldeicaza I love the simplicity of it! Fantastic write up!

Miguel de Icaza ᯅ🍉Jan 25, 2023

@johnnysbug thank you Jonathan!

Ken Crandall Jan 25, 2023

@Migueldeicaza The cranky BOFH in me loved the Xibbon logo homage to DEC!

Miguel de Icaza ᯅ🍉Jan 25, 2023

@kencrandall glad you noticed! :-)

smeger Jan 25, 2023

@Migueldeicaza I appreciate the write up! This does not work on my Synology host - the connection is insta-dropped whenever I navigate away from La Terminal. Can I help diagnose? It’s pretty frustrating!

Miguel de Icaza ᯅ🍉Jan 25, 2023

@smeger yes, this means El Preservador didn’t start. Can you look in the “…” menu for “Connection log”, this should have more details

smeger Jan 25, 2023

@Migueldeicaza Yup, that appears to be the reason. Do you know why it’s unable to start? TMUX is fine - I’m just curious.

Miguel de Icaza ᯅ🍉Jan 25, 2023

@smeger yes, the synology ships without the sftp client for ssh. I believe it is some alternative ssh.

smeger Jan 25, 2023

@Migueldeicaza Ahh, lame. Would scp work? And it might be good to let the user know session preservation won’t be active if El Preservativo didn’t load.

Miguel de Icaza ᯅ🍉Jan 25, 2023

@smeger yes, I have it in my todo list to warn about the condition, before it used to hang, now it falls back to a regular connection. I shall try also to downgrade to tmux in that case.

Let me also look into this scp alternative.

smeger Jan 25, 2023

@Migueldeicaza Great! I adore the app, btw!

Miguel de Icaza ᯅ🍉Jan 25, 2023

@smeger ah thanks for the kind words! Glad you like it!

Miguel de Icaza ᯅ🍉Jan 25, 2023

@smeger I believe synology was lacking a way to upload the agent. In this case, you can try “tmux” for session preservation

James Gilbert Jan 29, 2023

@Migueldeicaza Where do I report bugs? On my 9.7" iPad Pro arrow keys are drawn on top of function keys, and there is a black bar above the keyboard. El Preservador fails to reconnect on High Sierra Mac Mini server.

Miguel de Icaza ᯅ🍉Jan 29, 2023

@jgrg hello! Apologies for these bugs, and thanks for sharing. The main page should have a support link, but also, you can go directly to GitHub.com/xibbon/LaTerminal and post there the bugs

Miguel de Icaza ᯅ🍉Jan 29, 2023

@jgrg one quick diagnostic question, what happens if you log into that machine and type: .laterminal/session-preservation-3-Darwin-x86_64 --version

James Gilbert Jan 29, 2023

@Migueldeicaza Ah ha!

$ .laterminal/session-preservation-3-Darwin-x86_64 --version
dyld: Symbol not found: ___darwin_check_fd_set_overflow
Referenced from: /Users/***/.laterminal/session-preservation-3-Darwin-x86_64 (which was built for Mac OS X 12.0)
Expected in: /usr/lib/libSystem.B.dylib
in /Users/***/.laterminal/session-preservation-3-Darwin-x86_64
Abort trap: 6

Miguel de Icaza ᯅ🍉Jan 29, 2023

@jgrg thank you! Ok let me see if I can figure out the magic incarnation to fix this!

James Gilbert Jan 29, 2023

@Migueldeicaza I just added two bug reports on github.

Interpipes 💙Jan 29, 2023

@Migueldeicaza if I actively exit/logoff, should the preserver not exit completely?

It isn't for me, and it's not being reused when a new connection is made, so if you disconnect cleanly and reconnect each time you end up spawning another preserver....

Miguel de Icaza ᯅ🍉Jan 29, 2023

@interpipes it should terminate if you end the session. If this is happening, I have a bug. What OS is the target, and shell you are using?

Interpipes 💙Jan 29, 2023

@Migueldeicaza Debian 11(.6) x64, currently running 5.10.0-20-amd64, regular plain old bash

Interpipes 💙Jan 29, 2023

@Migueldeicaza given it seems to be dynamic, if it is relevant, libc-2.31(-13+deb11u5)

Miguel de Icaza ᯅ🍉Jan 29, 2023

@interpipes thanks, let me test that configuration. Btw, it will die after 16 hours or 12 idle hours - I can’t remember the exact number

Interpipes 💙Jan 29, 2023

@Migueldeicaza I was worried perhaps it was some weird interaction with docker running on the first host, but it behaves the same on another very plain Deb11 server.

A third Deb11 server (this one is admittedly.... special), the shell never starts, I just get connection log that ends in "The initial connection to the preservator has now closed" and no launched background process. I can't see any evidence of the preservator attempting to "re-ssh" to localhost... how would I generate more debug?

Interpipes 💙Jan 29, 2023

@Migueldeicaza also... I have a question - given it must be used for the preservator to connect to localhost, how is the private key data handled when the preservator is set up?

Is the authentication request just proxied through "bytes in, bytes out" style to the host, handled sshagent style, or is it stored (even if only in memory) at all by the process?

Miguel de Icaza ᯅ🍉Jan 30, 2023

@interpipes two parts, the preservador is really a passthrough, so it doesn’t keep any key data (which is invisible to it anyways, as it is ssh traffic), but it uses a randomly generated password to prevent an attacker from attempting to hijack your session. So on reconnects, the client authenticated with the preservador.

Miguel de Icaza ᯅ🍉Jan 30, 2023

@interpipes for that one, could you run: .laterminal/session.*3 --version I suspect a very old Debian that can’t run the binary perhaps?

Interpipes 💙Jan 30, 2023

@Migueldeicaza it's a Deb11 host like the other two.

The binary itself ran ok;
preservator bb60eb8d1c369f26cb7b1767b40ccf924d6ab977

I only said this one is "special" because it is a legacy OSE which was previously a 32-bit that I did an in-place switch to 64-bit a while ago. I straced the binary and it's pulling in x64 libc, though, so no, it's not related to that.

Then I figured it out, by trying to manually SSH to 127.0.0.1 - this sshd is only listening to specific IPs. 🤦‍♂️

Interpipes 💙Jan 30, 2023

@Migueldeicaza I don't know if it's worth having a fallback attempt to ssh to the same IP the LaTerminal client is connecting to rather than localhost to avoid non-standard ListenAddress configurations, but I guess there's also a non-zero risk that might just end up bouncing off firewall rules anyway.

(plus I guess it is maybe less useful where that IP is dynamic in some way, if it changes then the preservator session will be lost.. I guess if that fallback was used, print a warning?)

Miguel de Icaza ᯅ🍉Jan 30, 2023

@interpipes I definitely should add a warning and improve the error reporting on that code path. I just added a few bits to notify the user of the problem, but I am missing crucial information. I should report “failed to connect to machine:port” to aid in diagnosis