Hacker News Reader

Comment by bretthoerner | Hacker News Reader

tinyssh is great. One use case for it that people may not know about: using it during Linux boot so you can remotely unlock encrypted drives. I have a headless NAS server that uses dm-crypt/LUKS under ZFS. When I update my kernel/ZFS I remotely reboot the server, wait a few seconds, and then ssh into a tinyssh powered encryption key prompt to unlock the drives. (I am immediately booted from ssh, as tinyssh exits.) I can then ssh again a few seconds later and I'm hitting openssh on a fully booted machine that wasn't able to open the drives without my intervention.

https://github.com/grazzolini/mkinitcpio-tinyssh

View on HN

I use normal opensshd for this.

No reason to support two ssh daemons when you can do it with one.

The difference in size on your init image is minimal and you probably aren't even trying to optimize for space there.

If you don't know the size of your rd off the top of your head then it almost certainly doesn't matter.

All fair, I guess I just landed on mkinitcpio-tinyssh first and it was my introduction to the idea, and only took a few seconds to setup. I'll switch to openssh if I ever have issues, but this has been working fine for many years, so I'm no rush.

Makes sense. Probably more work to go off the beaten path then to maintain two configs

Probably not more popular because (for reasons I do not know) the mkinitcpio hooks Arch Linux provides are only for tinyssh and dropbear:

https://wiki.archlinux.org/title/dm-crypt/Specialties#Remote...

Question: when remotely unlock the boot disk via ssh, how do you make sure the boot has not been compromised and that you are not just sending the password to the bad guys?

At some point I wanted to do something with utrablue [1], to work over network rather than Bluetooth, but then it was in go and I got lazy suddenly :)

[1] https://github.com/ANSSI-FR/ultrablue

> how do you make sure the boot has not been compromised and that you are not just sending the password to the bad guys?

In my case, I can't. This is a NAS in my house and this is mostly to prevent me from having to go to another room and plug in a monitor and keyboard. (Also, I've done this from across the country after a power outage.)

The threat vectors I'm protecting against are I guess mostly theft of the entire machine, or forgetting to wipe the drives when I eventually toss them out. Mostly, it's just fun practice because I'm a nerd and every drive should be encrypted.

For my use-case, the auto-unlock-by-polling-a-specific-LAN-IP linked in this thread would probably be fine, for example.

This is mostly me but the case that's the most common is that a disk can't be wiped because its dead. Gotta do that before hand.

Well you can always drill holes in the platter, or hit them with a strong magnet, or just separate them and toss them in the trash. Unless you're fighting the NSA, you can probably get away with enough physical destruction to make recovery challenging.

That doesn't work if you need to RMA the disk. So best to encrypt before you put anything on the disk

This is what a TPM is designed to prevent, but they’re not very popular in Linux community.

What are the best options to solve this problem? It’s a hard problem imho for most threat models.

If the booting machine has been compromised and i use my usb connected keyboard to enter the full disk encryption key I would run into the exact same issues, no?

If you have physical access, seems way easier to put a keylogger directly into the keyboard.

Theoretically you could use secure boot with custom keys to ensure that your boot chain is not modified and you could use TPM for SSH host keys storage to ensure that it's not possible to copy them.

The ultrablue project I linked to solve exactly this problem, with TPM and a smartphone, but it's targeted at unlocking your laptop and uses Bluetooth to communicate with the smartphone for unlocking - and I don't want to have Bluetooth on my NAS ^^

TPM and checking your physical security boundary hasn't been breached.

Isn’t there a host SSH key involved?

The server itself may have been physically breached, and if so you can’t trust anything. But, if your host key matches, you should be confident that at least you’re logging into the correct machine (there was no IP takeover).

If the server was breached, what stops somebody from copying the host key to their new system.

True, but in that case, I think it would still be difficult to swap in a completely new system into the network (with your IP) with your old private host key.

(Without a physical breach... if that happens, all bets are off).

I have disk encryption on a remote server on cheap low-asssurance hosting, and I do it so I don't have to worry about what happens when the disks are recycled. If I could (easily) automate entering the key on boot, I would (but I'm running FreeBSD, so Linux solutions don't help). But I don't need to reboot often and the hardware is fairly reliable, so eh.

This is for my personal hosting which if someone wants to take over, I guess I'd be more curious than upset.

In theory this should be a perfect match for TPM-powered secure booting: your machine starts every time with a clean TPM, which validates the BIOS into it, which validates the bootloader into it, which validates the kernel into it, which derives an RSA private key from the resulting TPM state. If an attacker compromises it, it'll present the wrong host key and you get a big fat error message.

Mandos tries to mitigate this problem by periodically checking that the rebooting server is still up. Exactly how this is done is configurable, but by default it uses ssh-keyscan to check if the server is up and using the SSH server key from its normal encrypted root file system. If a server is ever down too long (configurable, default 5 minutes), the Mandos server will “disable” that server in its server list, and not provide the password to that server anymore.

The idea is that you should configure the timeout to be long enough to allow for a normal kernel panic and reboot, but hopefully short enough that it would be hard for anyone to compromise the server in that time. It’s not a perfect solution, but it’s the best anyone has come up with as far as I know.

(Disclosure: I am a co-author of Mandos.)

For debian/ubuntu users, there's also dropbear-initramfs package with same functionality (works with any fs luks/ext4/lvm/zfs/etc).

https://packages.debian.org/bookworm/dropbear-initramfs https://packages.ubuntu.com/jammy/dropbear-initramfs

I've used this for several years now. It works well and is relatively easy to set up.

Note: Mandos is also in Debian and Ubuntu.

(Obligatory disclaimer: I am a co-author of Mandos)

Usually I use DropBear for this. Do you know if one is necessarily better than the other? DropBear I think is what RHEL docs recommend for remote boot disk decryption.

Ah, I've never used DropBear. I don't know how one could be better than another for my simple use case, honestly.

This is more or less the RedHat based solution to do this using openssh: https://github.com/gsauthof/dracut-sshd https://copr.fedorainfracloud.org/coprs/gsauthof/dracut-sshd...

I thought that everyone has switched to Clevis + Tang for that?

https://access.redhat.com/documentation/en-us/red_hat_enterp...

It's fully automated and supposed to be much more secure.

Has anyone got experience with it?

> I thought that everyone has switched to Clevis + Tang for that?

Clevis+Tang is good. There's also Keylime which takes a different approach to the same[1].

[1] https://keylime.dev/

IIUC whether that is secure depends on your threat model. For example, how good is automated unlocking compared to unencrypted drives in a homelab setup?

I've seen a bit about Clevis. Is there a major difference between using this, and systemd-cryptenroll?

I guess it depends on your use case. If you rent a bunch of bare-metal servers at a remote location and you want restarts after updates to be fully automated, Clevis seems like a way to do. The whole idea is that once you cancel the server, you just remove it from Tang's list and the next customer who gets those hard drives cannot read them.

AFAICT, systemd-cryptenroll requires that you have a USB key plugged into the machine, so someone with physical access would have to insert them at the start and remove when you're done with the server. With Clevis+Tang everything is software.

Or am I missing something?

That makes sense. I was thinking about the case of using a TPM to unlock full disk encryption as long as secure boot hashes checked out.

There’s a non-interactive solution to rebooting safely with encrypted disks: Mandos <https://www.recompile.se/mandos>

Reboot your server while you sleep!

Disclosure: I am a co-author of Mandos.

This is really cool. I'm going to give this a try!

A tool based on Dropbear that does exactly this, automatically.

https://github.com/ViktorStiskala/cryptsetup-ssh-unlocker

The documentation for Cryptsetup SSH unlocker states “To further limit the attack possibility, you should use monitoring and possibly disable SSH unlocker in the case of unexpected behavior.” Mandos has a built-in feature to deal with this, enabled by default.

(Again, disclosure: I am the co-author of Mandos.)

"tinyssh is great."

Agreed.

A static tinysshd works well for the small userlands I create.

Reply on news.ycombinator.com