Hacker News Reader

Comment by raimue | Hacker News Reader

The author missed that sshd will always execute the user's shell and pass it the command with arguments as a `-c` argument. This means that the given command string will always be parsed by the remote shell. This is required to restrict users to certain commands with special shells like scponly or rbash.

When you keep in mind that the given command string will be parsed twice, first by your local shell and then again by the remote shell, it becomes clear why a running a remote ssh command behaves like this.

View on HN

Yep! God though, this hits me in the face so often. Trying to add `sh -c` to fix it is a trap, because obviously, you just create yet another layer of escaping.

It really becomes one hell of a puzzle sometimes, especially if you're necessarily nesting another layer of escaping. It feels like you're trying to write a quine.

This works:

    ssh host -- ls "folder\ name"

This also works:

    ssh host -- ls \"folder name\"

This works:

    ssh host -- sh -c \"ls \\\"folder name\\\"\"

OK, so clearly, just throwing more escaping at it fixes it. But even if you figure that out, the real mental gymnastics would be figuring out which of the three shells interpreting your command line in the last case would handle shell expansion.

In this case, it's the host:

    ssh host -- sh -c \"ls \\\"folder nam\\\"*\"

In this case it's the remote:

    ssh host -- sh -c "\"ls \\\"folder nam\\\"*\""

Of course where you put the quotes makes no difference. All it does is prevent your shell from processing it. So this works just as well:

    ssh host -- "sh -c \"ls \\\"folder nam\\\"*\""

If you sit and think each layer through, it usually isn't completely impossible to understand, but the odds that you are going to get something wrong the first time is astonishingly high.

It does make me wonder why ssh handles it the way it does, though. Because with the way SSH handles it, it may as well just automatically escape the spaces. Right now, not putting an SSH command in quotes doesn't make much sense unless you for some reason want local shell expansion for something.

If you can invest a little bit of time in configuration of the remote host, then my "special shell" (also mentioned elsewhere on this page) may be of use, and you will no longer need to hit yourself in the face so often!

mention: https://news.ycombinator.com/item?id=36726111

crate: https://crates.io/crates/arghsh

neat. one little pedantic note, argsh requires all arguments to be valid unicode strings while I think at least Linux allows argv to be arbitrary sequences of non-zero bytes. But then I really hope there are not many cases where that is relevant consideration.

Good point, I'll add that to the README next time I touch it. To warn off those who were thinking about passing jpegs via argv ;-)

From my experience, it's possible to just do

  ssh host 'ls "folder name"'

and practically ignore the `command [arguments]` split in SSHs synopsis. It's all passed to a shell and parsed again, anyway.

For simple tasks, that's how I have always done it.

Alternatively,

   echo ls \"folder name\"|ssh -T host

   cat > 1
   ls "folder name"
   ^D

   ssh -T host < 1

The last example really looks like perfect case for using `here document` feature.

    ssh -T host <<EOF
    ls "folder name"
    EOF

Once I run into nested escape wrangling I start to seriously question how I'm trying to accomplish something.

Once I run into nested escape wrangling I seriously question the sanity of my tools.

If I can't fix it in like 3 tries, I switch to my local shell script having the script to run on the remote end be in a HEREDOC and just scp it it over and then ssh exec it. Inside the HEREDOC, it's a sane environment.

Copying over the commands to run also has another added benefit, you have quite good documentation of what was run and when. Also it allows one to easily run a failing thing again (in case output gets mangled in the executing script)

And idempotency is essential for shell scripts that alter state. Otherwise you have to keep track of where failures happened.

wrap it in base64 then cat it | ssh « base64 -d | bash »

`printf '%q'` is your friend.

> This works:

  ssh host -- ls "folder\ name"

> This also works:

  ssh host -- ls \"folder name\"

Uh, why not

  ssh host -- ls '"folder name"'

? Single quotes are the shell’s ultimate bulk “no touchy” escape, so if you don’t need them in the inner command, it seems easier to use them for everything. (Also when passing programs to sed, awk, jq, xmlstarlet, etc.)

If you do need single quotes in the inner command, you can use the "'" trick:

    ssh host -- ls "'"'folder$name'"'"

$name will never be expanded. Is that what you expected?

Yes, that’s the intent: using single quotes at both levels will prevent either your shell or the remote shell from expanding the name, so you can interact with a file name containing a literal $.

On another note, avoiding having paths with troublesome chars in them on servers does generally make the sailing much more pleasurable.

I believe that also works, but I don't do it often because I often wind up wanting to be able to use variables from the host, especially e.g. in cron jobs and scripts.

Yes, that’s a valid thing to want.

At that point, though, I’d try to write a general shell escaping function, because I don’t trust myself to figure out which host things are OK to include unescaped in such a situation. (Here’s when I start to long for Tcl, despite all the times I’ve had to spell out [string index ...].)

The optimal solution would be to have a separate side channel for passing things to the quoted program, like -v in awk or --arg in jq (or whatever it is in your favourite SQL DBMS binding) but I don’t think SSH will let you do that.

This is more or less baked into the SSH protocol because the exec request only has a single string for the command line.

Also the reason echoing a MOTD from rc files or similar crap breaks tools like rsync or scp which use SSH as a neutral transport. SFTP isn’t affected because, while using the three-pipe as a transport, it’s a separate subsystem with its own SSH request type to initiate the channel (just like X11 or port forwarding).

If you control client and server you can define your own subsystems and invoke them directly, which avoids this whole mess.

I think the confusion comes from the documentation where ssh(1) says that the command "is executed on the remote host instead of a login shell.". Which is true from the perspective of ssh(1) in the sense of the protocol. The client has no control over what the server does with that string.

However, sshd(8) clearly documents that it will always execute the login shell, even when a command has been passed. ("All commands are run under the user's login shell as specified in the system password database.")

Subsystems open secondary channels to communicate separately from stdin/stdout of the remote command. I used the X11 forwarding before to run a remote command with sudo without getting the password prompt interfering with the protocol: https://raimue.blog/2016/09/25/how-to-run-rsync-on-remote-ho...

An extra subtlety is that it is the user’s login shell (in the getpwnam sense) but the command is not run in a login shell (in the sh - / sh -l sense). That’s why proper motds or profile files work ok (requesting a shell runs the user’s login shell as a login shell) and don’t break applications. Also the reason why $PATH often differs between the two modes.

yeah, this exec trace from the article is wrong, it is missing one sh -c from the chain

  $ ssh localhost figlet foobar bar\ baz
  execve("/usr/bin/ssh", ["ssh", "localhost", "figlet", "foobar", "bar baz"], …
  execve("/usr/bin/figlet", ["figlet", "foobar", "bar", "baz"], …

in practice it looks more like this (traced with execsnoop):

    PCOMM            PID     PPID    RET ARGS
    ssh              4255    2058      0 "/usr/bin/ssh" "localhost" "figlet" "foobar" "bar baz"
    sshd             4256    2147      0 "/usr/bin/sshd" "-D" "-R"
    bash             4259    4258      0 "/bin/bash" "-c" "figlet foobar bar baz"
    figlet           4259    4258      0 "/usr/bin/figlet" "foobar" "bar" "baz"

> This is required to restrict users to certain commands with special shells like scponly or rbash.

I don't think this is some specific design goal of OpenSSH, I think it's just a side effect of how shell escaping works.

> When you keep in mind that the given command string will be parsed twice, first by your local shell and then again by the remote shell, it becomes clear why a running a remote ssh command behaves like this.

I get that this behavior may be surprising to new users, but anybody working with ssh regularly will encounter these kinds of escaping issues. SSH isn't even the only place you'll encounter this. Things like docker etc will have the same "problem".

In the case of ssh you can simply write your commands to a file and send them via stdin, or copy a script to the target.

The tone of this blog post rubs me wrong. Yes this is a footgun (in the same way many POSIX shell-related things are), but it's not like it's some "problem" with the design of SSH.

Only tangentially related.. Is there something like rbash that is actually secure and more restrictive? Like a shell that only "sees" certain files and folders and can only execute certain commands in a non privileged manner.

The shell rarely "sees" files and folders, except for expanding a glob like "*".

When the shell executes "cmd folder/file", the "folder/file" is just a string as far as the shell is concerned. It is the command that uses that string with a function like unlink or open.

Okay... you're right. So should I say "the process" and every process forked or exec'd instead of "the shell"? But is it clear what I am looking for?

Reply on news.ycombinator.com