Hacker News Reader

Comment by anonymousiam | Hacker News Reader

I wish the process of restoring a non-booting Debian-derived system were easier. After needing to do it on several dual-boot systems, I keep the procedure handy now. Here it is for posterity:

1) Boot Linux from your distro CD or DVD

2) Get a shell

3) Mount your normal Linux partitions. (Make sure you know where they are. The following example assumes / on /dev/sda1 and /boot on /dev/sda2.) E.g. mount /dev/sda1 /mnt Note: If you have a separate partition for /boot, then mount it too: mount /dev/sda2 /mnt/boot

4) Mount the special nodes: mount --bind /dev /mnt/dev && mount --bind /dev/pts /mnt/dev/pts && mount --bind /proc /mnt/proc && mount --bind /sys /mnt/sys

5) Change your shell's root: chroot /mnt

6) Re-install grub: grub-install /dev/sda

7) Update the grub boot menu: update-grub

8) Undo chroot: exit

9) Unmount the special nodes: umount /mnt/dev && umount /mnt/dev/pts && umount /mnt/proc && umount /mnt/sys

10) Remove media and reboot

View on HN

After several Win10 upgrades nuked grub on a dual boot system, I have found a much simpler process. Create a rEFInd [1] USB boot key once and for all. Then whenever grub is hosed, boot on this key: rEFInd will find the Debian partition and allows to run it directly. Then once in Debian just do an "update-grub" to fix things.

[1] https://www.rodsbooks.com/refind/getting.html

You could just install rEFInd and make it the default bootloader with its auto find mode enabled.

win10 install nukes it sometimes as well. its a PITA

Have you found a way to deal with this? Swear I have refind on the non-Windows disk, but Windows updates sometimes replaces either the partition, or the default boot device. Mega frustrating!

https://wiki.archlinux.org/index.php/Unified_Extensible_Firm...

Over two years of dual booting with Windows 8 I never had it screw up my UEFI boot. The MBR I had tried in the past never lasted but with UEFI it just left everything on the ESP alone.

Like boomboomsubban, I have never had this happen. My current laptop shipped with Windows 8 and has had Windows 10 installed since the weekend after the public beta was released. It's also had an Ubuntu dual boot since exactly the same time. rEFInd was installed a few months later when I discovered it was still a thing (I had been a rEFIt user on Macs years ago).

Never once has the Linux boot manager/loader (either the original GRUB or rEFInd) been broken, by a Windows update or otherwise.

Windows of course has been well documented to happily overwrite any MBR bootloaders when you install it, but even that doesn't happen in a UEFI environment. Windows, rEFInd, GRUB, and whatever else you may have installed all have their own folders. It might change the default, but restoring your own choice is literally a matter of copying a single file.

EFI has plenty of problems of its own, but this is one it solved pretty well.

I've seen it do it as well and I'm pretty sure now that it's part of the "automatic repair" process that Windows does when it detects 3 (?) failed boots. What happened pretty often to me is that I'd miss the boot screen and Windows would start booting (it's the default bc it's a shared PC) and in order to not waste time, I'd force a reboot. After a few times, next time Win would actually boot, it would go into rescue mode and start poking around the EFI partition and efivars doing god knows what that ended up in my GRUB not even showing up in the F11 boot menu.

I've had this happen too, but I think it just changed something in the efivars, grub itself was still there and still operational, even though it didn't show up in the available boot options.

The workaround for me was the "boot from file" boot option, where I would be able to walk the directories in the efi partition and start grub. Once this was done, it was back in the boot option list.

Yeah, I've only had it remove the actual files once, when I had to use the recovery USB thing to fix a broken update. Unfortunately, my motherboard is from the very early days of UEFI and doesn't allow me to select files or even manage keys so I have to keep a microSD card with a bunch of bootloaders handy in case this happens (I still haven't gotten around to learning UEFI shell in order to manually boot).

Windows repair likes doing it.

I've had it nuke my bootloader a few times when I used to sill dualboot (now full Linux).

One time the windows installer confused the encrypted LVM disk with a corrupted NTFS partition (because it had an NTFS partition on it at some point and underlying data wasn't wiped properly) and tried to fix it. Suffice to say, that required a back restore, so now all my encrypted disks have a 100MiB NTFS buffer partition for Windows repair to fuck around it (until I removed dualboot, I occassionally found remnants of an attempted repair in them)

Is it sufficient to install Windows to a completely separate disk instead of a separate partition to prevent this? I can't imagine Windows starting to overwrite partitions or boot sectors on a drive it is not even installed to

I always install Windows to a different disk, so yes, Windows will try to recover partitions on other disks, especially when it finds multiple ESPs and the windows one isn't the active one (upon which it sometimes tries to install it's bootloader there, looks if that disk has the windows partition on it and confirms that yes, windows is installed on that disk because it has a windows-like ESP that it just installed, so it goes for a repair)

https://wiki.archlinux.org/index.php/Unified_Extensible_Firm...

This strongly depends on the issue though. In most dual booting cases I've had to deal with recently, a single efibootmgr command was enough.

Windows and some Linux distros like to replace the fallback EFI command and the two can conflict every now and then. By properly configuring the UEFI to pick the right OS at boot (sudo efibootmgr -o 0001,0000 or something like that) should be enough.

If you're still using MBR or you have a shitty motherboard with bad UEFI support you're right that you're still forced to do a full Grub reinstall.

I've had to do recovery on a non-booting Windows machine that broke after a recent update and it was just impossible to recover. Some file on the FS was corrupted, but SFC couldn't recover it and wouldn't report what file was corrupted. Rebooting into recovery takes forever, boot recovery constantly fails and the only saving grace is the "reinstall Windows" that uses a complete reinstall as a fix for a broken OS.

Yes, fixing Linux is difficult, but at least it's doable. On Windows you're basically stuck with the two or three auto-recovery options or an OS + software reinstall.

I can't speak for macOS but I can't imagine it being much easier (especially because the limited variety of macOS hardware makes it more unlikely for the OS to fail catastrophically because that's easier to test for).

It shouldn't be too hard to create a Linux boot ISO that lists installed operating systems and then automounts them for recovery. It's only a matter of parsing GRUB config files + fstab + crypttab and some predefined mount patterns for distros after all.

Mac lets you reinstall the OS from the recovery partition without losing data

https://support.apple.com/en-us/HT204904

If the recovery partition is toast you can do a clean install over wifi - Option-Command-R or Shift-Option-Command-R

(would only work over an open wifi the last time I did it)

It also works over Ethernet!

I tried it via an Ethernet to USB adapter and the laptop didn't see the wired connection

Maybe a proper rj45 would work

Does this include programs and settings? Windows lets you reinstall with data as well, but installing every single program and getting it set up all over again takes forever, that's my main problem with the Windows recovery process.

The install over the internet is a nice touch, though I don't expect that feature to ever make it into normal computers because it would probably force manufacturers to put a minimal Windows installer in their UEFI.

>Does this include programs and settings?

Yes, I believe so - I'm guessing this is due to things being more siloed

I should also mention it won't let you choose an OS version, although it does say the name of the one that will be installed

rEFInd can automatically detect linux, Windows, and mac partitions and boot them. You could just install it to a usb stick and use it whenever necessary.

That's a very neat tool! I'm kind of turned away by this though: > Warning: Your kernel and initramfs must reside on a file system that rEFInd can read.

Can it boot fully-encrypted disks (by chainloading GRUB, for example)?

What you mentioned is rEFInd’s auto detect feature which by design doesn’t work with encrypted disks. rEFInd will automatically detect other .efi binaries so you can just install your other bootloaders alongside rEFInd, make rEFInd the default and chainload into your other bootloaders when you need to boot an encrypted disk.

I have a setup where I moved /boot of the encrypted luks partition to the esp and boot from there using a custom entry in rEFInd

This is essentially the process used for installing the base system in Arch. As usual, the Arch Linux wiki page on this topic [1] is a good resource, even if you're using a different system.

If you want to learn more about how a typical Linux system is organized, but don't have the time or patience to go through the Linux From Scratch book [2], installing and configuring Arch once is pretty insightful and also kind of fun.

[1] https://wiki.archlinux.org/index.php/Installation_guide

[2] http://www.linuxfromscratch.org/lfs/

I mean, it's just a chroot; it's the basic working "run a system without booting from it" that underlies a lot of base system tinkering and fixing.

There are lots of people who have used Linux for years but have never actually gone through this exercise. It's not very fun to have to do it for the first time when an important system fails to boot.

It's great that the modern Linux install process is so easy, but one drawback is that it also makes it easy to gloss over how the system is put together.

I just meant that it can be insightful to put together a toy system from scratch, so that you can learn (at your own pace) how chroots work, learn the conventions around manually recreating your root hierarchy in /mnt, learn what systemd services you actually need because you need to manually turn them on, rather than having a bunch of stuff you don't understand enabled by default, etc.

I was going to say Gentoo, for much the same reasons.

I usually do the bind mounts a little differently:

for ii in dev dev/pts proc sys ; do mount --bind /$ii /target/$ii ; done

Unmount: umount /target/dev/* /target/* /target

This can also be useful on newer systems where systemd systems need to be chrooted.

systemd-nspawn --directory /target --boot -- --unit rescue.target

UEFI makes this problem much easier—even if you still use a bootloader (no need!) there’s not much need to update it after it starts working.

Why is the Debian-derived system non-booting?

Many years ago I had disk error problem and could not boot. I tried a popular, Linux-based "system rescue" CD and it could not boot without accessing the disk. I tried my own "live" NetBSD USB stick and booted up no problem. No disk access needed.

I stayed away from Linux for many years as a result of experiences like that. I have been using Linux lately and I continue to find more stuff like this that would just never happen on BSD.

I do not understand why people use grub, let alone grub2. There are plenty of other bootloaders. What is wrong with syslinux?

GRUB2 is the most flexible bootloader. I use it for a multi-boot flash drive, used for booting 64-bit FreeBSD on a 32-bit-EFI machine (earliest Mac mini + 64-bit CPU)…

But yeah for normal desktop usage… there's rEFInd.

Flexibilty would have been my first guess. Are there any other reasons besides flexibility?

I'd say support in distros. With grub you can throw pretty much any distro onto your machine and it will show up (except Solus) . Also I don't think you can just choose which bootloader you want in most distros except Arch and Debian if I remember correctly. You can do that manually of course, but grub is easy and it works.

In arch you can choose the bootloader as well. The recommendation I hear a lot and that I use is systemd-boot (formerly gummiboot), which strangely doesn't have much to do with systemd.

It can pick up Windows for multiboot and is straightforward to configure (can even be configured from a windows live disk if you mount that FAT partition).

Well, I just had a look at the Arch Wiki article on Boot Loaders [1]. GRUB is by far the most widely supported and also the most widely compatible bootloader.

[1] https://wiki.archlinux.org/index.php/Arch_boot_process#Boot_...

The table is somewhat misleading. The filesystem support for example refers to being able to boot a kernel that is on that FS from disk.

In most gummiboot setups, your kernel will be on the ESP partition along with everything else, so it doesn't matter what FS the root is.

The lack of support for MBR or BIOS doesn't matter much either, systemd-boot requires a 64bit System and most 64-bit systems that are still around and largely used (or actively sold) have a UEFI that supports GPT. If you absolutely need to, systemd-boot supports booting from a GPT that has a MBR wrapper.

So while the table looks like systemd-boot lacks support for a lot of things, the reality is that when you setup systemd-boot a lot of these columns simply don't matter

So does that mean that I can't use those filesystems for my /boot/efi partition and everything else is ok? Then it's really not that bad. I haven't played around with other boot manager as you might be able to tell.

That's pretty much your limitation; your linux kernel + initramfs can't be on filesystems other than VFAT (ie, must be on the ESP but there is some ways you can have it work across disks).

Hence the table being slightly misleading.

For example, it also mentions that EFISTUB means you can't boot on btrfs and friends anymore. But it's the same limitation, initramfs and kernel need to be on the ESP, everything after that is up to the initramfs to bring up.

Obviously you can with Gentoo. That's basically its "thing" (choice).

Oh yeah, I forgot about Gentoo.

This isn't applicable to a UEFI Secure Boot system, where the shim and grubx64 binaries are distribution signed and installed from the package - not via grub2-install, which on UEFI produces an unsigned binary and will not boot under UEFI Secure Boot.

Congratulations, you just learned how to install Gentoo.

This is essentially identical to the procedure I did 15 years ago. What's the issue?

I don't think GP is complaining because the procedure changed in the last 15 years, but precisely because it didn't. Surely, there's room for improvement.

Well, he still doesn't use UEFI. He could, it has been available for more than decade (i.e. better part of those 15 years), so the procedure didn't change because he has chosen so.

Not that it is a bad thing, some people do prefer that kind of stability. But then, why complain about that?

Btw, Intel did plan to remove CSM (legacy BIOS compatibility) with Tiger Lake and require UEFI class 3. We will see if they will follow through. Then the procedure WILL change, and we will hear from people who didn't expect it.

Wouldn't UEFI make it more complex, since you now potentially need to mount / and /boot and /boot/efi all separately before you can fully recover grub?

Not more complex, but different.

It does not have magic sectors on disk, in MBR for boot loader, on somewhere in dummy area of the filesystem for grubenv, everything happens on the EFI partition and with EFI variables (stored in nvram). Creating bootable EFI media means having vfat-formatted filesystem and copying files there. No need for tools like Rufus.

Your firmware would find it at boot time, and allow you to boot from it; most UEFI implementation have boot manager built-in.

I think the issue boils down to the fact that it depends almost completely on exactly how the system is put together. I mean, the process is:

1. Boot into a working-enough live system - only way to improve this is to put a recovery system on the same system, but then you risk making it unbootable as well

2. Mount needed filesystems - depends completely on what filesystems there are, which is extremely variable; my only thought would be to use labels and hope that the installed system labels root/boot/efi the way you expect

3. Mount special filesystems - this is possible to automate; ex. arch-chroot (https://wiki.archlinux.org/index.php/Chroot#Using_arch-chroo...) does it, but that requires that you know what you need, so it's going to be brittle unless you stick with distro-provided tools (hence, arch chroot)

4. Fix the system - again, totally depends on what happened and how it should be set up; yes, in the trivial case you could make a "just-reinstall-grub.sh", but it'll fall apart for non-trivial setups

If every system looked the same, then yes you could make a livecd that automatically booted, set up mounts, chrooted in, fixed grub, and rebooted out, but this is the world of Linux-based systems so even within a single distro there's worlds of difference.

Semi-automating the common cases with a GUI would probably help many of the less sophisticated users. E.g. scan the existing system for partitions and ask them which one they want to repair.

Windows' repair mechanisms also only work in the common case and you have resort to quite similar CLI stepss when they don't.

Man, I’m glad ArchLinux has arch-chroot to do this for you!

Reply on news.ycombinator.com