Hacker News Reader

Comment by theon144 | Hacker News Reader

Not me, but a colleague - he wanted to look around the system as the `uwsgi` user, so he ran `sudo -u wsgi -s /bin/bash`.

Except that he typoed, and instead ran `sudo -c wsgi -s /bin/bash`. What that does is instead of launching the (-s)hell as the uwsgi (-u)ser, it interprets the rest as a (-c)ommand. Now, `wsgi` is also a binary, and unfortunately, it does support a `-s` switch. It tries to open a socket at that address - or a filesystem path, as the case may be. Meaning that the command (under root) overwrote /bin/bash with 0 bytes.

Within minutes, jobs started failing, the machine couldn't be SSH'd into, but funnily enough, as /bin/bash was the login shell for all users, not even logging in via a tty through KVM worked.

Perhaps not the best story, but certainly a fun way to blow your foot off on a Monday morning :)

View on HN

That's beautiful. I'm not sure I'd have had a clue what just happened even if it was me making the typo.

`ssh $host /bin/sh` (or another shell) should work?

On a Linux box isn’t that just a link to bash?

It depends on the distro; on RHEL yes it is BASH, but Debian uses dash, and Alpine uses busybox sh.

Logging into another shell would be attempted only if someone knew at the time why the logins were failing.

But thanks, I've just added another technique to my toolbox.

Depending on the distribution, /bin/sh might be a symlink to bash

That won't work because sshd runs the command using the user's login shell. From https://man.openbsd.org/sshd#LOGIN_PROCESS:

> When a user successfully logs in, sshd does the following:

> ...

> 9. Runs user's shell or command. All commands are run under the user's login shell as specified in the system password database.

Now I'm curious how you managed to recover. I only know enough of my way around a shell to be dangerous and I'd be SoL if I ended up in this situation.

Recovery disk, then either copy the disk's copy of bash (if it doesn't depend on a later glibc version), copy another shell to /bin/bash (as the system probably doesn't depend on bash-specific commands to boot), chroot and use the package manager, or use the package manager with an explicit sysroot (e.g. pacman --sysroot). The first two steps are very easy compared to the latter two, but should be followed by a reinstallation of the package that provides bash.

i wonder, wouldn't copy over sftp work? i don't think it depends on shell

You should also usually be able to just reboot and as long as another shell is installed, choose that shell as your startup command in the bootloader.

Does the proc entry for a running process still link to the now-deleted file in that situation? If so, you might be able to save yourself from a running bash shell by doing a “cat /proc/$$/exe > /bin/bash”

Probably not if it was overwritten (": >/bin/bash") rather than removed and recreated ("rm -f /bin/bash; : >/bin/bash"). The former will cause all processes to see the empty file, the latter would leave processes with access to the old contents.

In this case if you noticed and still had a shell, you could just copy another shell over ("cp /bin/sh /bin/bash"), to at least get back to probably able to login, until you could pull a copy from another machine or backups.

In the wsgi case of replacing a regular file with an AF_UNIX socket, the destination would have needed to to be unlinked first - the bind() call will otherwise fail.

On Linux you can't overwrite a binary that is currently being executed (you'll -ETXTBSY when you try to open it for writing, and truncate will also fail).

I was unaware of the first part of this, and decided to find a reference.

This stack overflow answer[0] provides some detail on ETXTBUSY and the fact that writing directly writing to a running executable file will fail. (I had thought it would unlink the file like in the second case, simply because that's what I always see when recompiling something that's already running. This obviously uses the second method though).

[0] https://unix.stackexchange.com/questions/187931/modifying-bi...

Reply on news.ycombinator.com