Ok, I'm seeing some weird behavior that I can't explain. If I run:
ssh server sh -c 'cd /var/tmp; pwd'

I get the user's home dir, But if I run:
ssh server sh -c 'echo foo; cd /var/tmp; pwd'

I get a blank line (?!?) followed by /var/tmp. I can replace the echo foo with pwd, and then both pwd's output the values expected.

Hmm, I wonder if this is related to cd, echo and pwd being built-ins, but from the local shell (no ssh), it works fine.

I'm seeing this from MacOS (Sequoia and Tahoe) to FreeBSD (multiple versions), MacOS (Sequioa) to MacOS (Sequioa), MacOS to Ubuntu 22.04.3 and Ubuntu 22.04.3 to Ubuntu 22.04.3.

So, looks like there's something more subtle about this. Since it means that bash has the same problem.

It almost like the first built-in is being ignored, but it's clearly not entirely ignored since echo is printing at least a new line, which makes this even more weird.

@mwl

@encthenet @mwl

does forcing it to allocate a pty via -t change things? Not sure why it should but it might be interesting. Similarly -n.

@uep
Nope, no change on either -n or -t.
@mwl

@encthenet

The quotes are being handled by the local system.

The remote runs 'sh -c /var/tmp' followed by 'pwd'.

The cd command only happens in the subshell.

@mwl

@sab38 @encthenet

Yep.

For anything more than bare-bones single commands, I use a shell script.

@sab38 @encthenet @mwl The quotes are hanled by the local shell, but four args (not counting 0) should be passed to ssh:
1: hostname
2: sh
3: -c
4: cd /var/tmp; pwd

I.e., argv[4] contains semicolon(s) in the middle.

@marshray
The strings get merged by the local ssh then expanded by the remote shell.
@encthenet @mwl
@sab38 @encthenet @mwl Wow, yeah.
Looks like the ssh protocol doesn’t encode the args separately, so the client must combine the arguments into a single “command” string interpreted by the server somehow.
https://www.rfc-editor.org/rfc/rfc4254

@marshray
I would say that it does this incorrectly. And definitely in a surprising manner. It should be requires to quote any args so that they can be handled properly by the remote system.

And yes, this gets into the whole, what if the remote system isn't Unix bs, and doesn't handle escaping the way the local client should. Maybe this should have been fixed to not be a problem by actually do proper passing of arguments instead of combining them.

I feel like I "discovered" this problem 10 years ago and was equally annoyed about it but then forgot about the problem.

@sab38 @mwl

@marshray

So the working one is:
ssh server sh -c "'cd /var/tmp; pwd'"

That is double quote single quote and single quote double quote. Other way works as well.
@sab38 @mwl

@marshray
Thinking about it. ssh should generate an error if any args have spaces in them and require a special arg saying, I've quoted my arguments properly to be handled by the remote system.

Yes, it might break a lot of things, but won't break anything that wasn't already broken (IMO).

@sab38 @mwl

@encthenet @marshray @sab38

IIRC, long discussions about this on the openssh dev list.

@mwl @encthenet @marshray @sab38

ohh, right, this is all sounding eerily familiar. ugh.

@uep @encthenet @marshray @sab38

I wrote the SSH book 8 years ago, but I have vague recollections of digging through a LOT of archives on shell command handling.

@uep Hey man, great to see you again!

@marshray indeed.. I've been here all along, but kinda idle on this account, much more active on one that's on the other side of a fedi partition. So they can't see each other, and there are some people that are only in this timeline that get missed.

I had at one point intended to try and separate the accounts for different purposes, more personal and more infosec professional, dedup follows so that I would check both, etc, maybe find a client that works across instances to dedup views, but.. well, it hasn't happened yet, let's say.

@encthenet @sab38 @mwl Yeah, pretty sure I’ve hit my knee on this before too.

Sure would’ve been nice if unix/posix had defined an unambiguous method of encoding and provided utilities to encode and decode to/from it.

Can’t imagine how many security bugs lurk out there due to this.

Could be worse… Win32.

@sab38
Winner winner!

Thanks, yeah, stupid shell quoting. I would say that's broken of ssh. It shouldn't be parsing the args when they are already broken out for you. Makes passing args safely very difficult.
@mwl