Bootable EFI isos is pure hell. Any tips? | Hacker News Reader

Bootable EFI isos is pure hell. Any tips?

1 pointBluescreens16 comments

I've been trying to make a bootable EFI iso... But it just flat-out refuses to boot. Any tips for good guides LFS style?

View on HN

I know this doesn't answer your question directly but is there a reason you can't use a USB stick? If you can I can give pointers on how to create a live USB image. Also is there a particular OS or Linux distro you're trying to boot?

Yup, skipped out on that info. Mb.

Basically what I did previously was use Distroshare to create an iso, which could then be burned to USB using Unetbootin. Unetbootin did its own magic, but it worked. Now I have a two partition USB-stick, one with /boot & /EFI, the other holding the e.g. ubuntu-16 iso. The default one from Canonical boots just fine, but not my custom version.

I did it that way because I thought it'd be handy. If you could give pointers on how to create a live USB image, I'd greatly appreciate it ;). Also, it's all Ubuntu (16.04 & 14.04).

Try Yumi https://www.pendrivelinux.com/yumi-multiboot-usb-creator/

Took a looksy, doesn't seem to be doing the job :(. It's a Windows tool, I'm running Ubuntus so that's not making it more reliable I guess..

Did peak into the source-code of the UEFI-beta version. Seems they are also using grub2 the way I am generally. (basically the grub-install --target=x86_64-efi type /boot & /EFI folders).

That's the part that _is_ working for me. It seems that my casper/filesystem.squashfs is missing a few mystery 'things'. You have any cluess regarding that?

It just works for me. As far as I know it uses recompiled linux tools to do its job on windows.

Yumi used to have a linux version, but it seems to be under rewrite.

You could try using wine I guess?

Ok so what I'm reading is that:

1) you can get EFI to load grub but 2) actually loading the live system image isn't working

So given those assumptions my next question is:

Is the kernel and initramfs loading? If you're not sure try:

* holding shift to display the grub boot menu on power up

* hitting 'e' to edit the default boot menu

* removing 'splash' from the kernel command line so you can observe boot messages

* continue booting by hitting ctrl+x

See if you can glean some more info from that

also, if you can pastebin your boot/grub/grub.cfg file that'd be helpful too

menuentry '[loopback]ubuntu-14.04.1-desktop-amd64' { set isofile='/ubuntu-14.04.1-desktop-amd64.iso' loopback loop (hd0,2)$isofile linux (loop)/casper/vmlinuz.efi boot=casper iso-scan/filename=$isofile locale=en_US.UTF-8 initrd (loop)/casper/initrd.lz }

This works.

Ok so one immediate point that I notice here is that if you look closely at this grub configuration, you'll see that the EFI boot Kernel and initrd are not loaded from an EFI partition, but rather taken from the ISO itself:

> loopback loop (hd0,2)$isofile

> linux (loop)/casper/vmlinuz.efi

Whereas above, for Ubuntu 16.04, you appear to be trying to use a separate fat32 EFI partition to load vmlinuz.efi

So, just to test, you should try to do what Ubuntu 14.04 is doing, but keep your EFI parition to load grubx64.efi. Make sense? So your partiton layout on this USB stick will be as follows:

Partlabel: gpt

(hd0,gpt1): FAT32 (type: EFI system)

  - has path/to/grubx64.efi

(hd0,gpt2): ext4 (type: Linux)

  - has boot/grub/grub.cfg

  - has ubuntu-16.04-WHATEVER-REALNAME-HERE.iso

----

And the grub.cfg now has a new menu entry as follows:

menuentry '[loopback]ubuntu-16.04-desktop' { set isofile='/ubuntu-16.04-WHATEVER-REALNAME-HERE.iso' loopback loop (hd0,2)$isofile linux (loop)/casper/vmlinuz.efi boot=casper iso-scan/filename=$isofile locale=en_US.UTF-8 initrd (loop)/casper/initrd.lz }

Let me know if that solution works for you?

[Edit] I think the issue with all of this here is the other scripts/parts of Casper that get loaded normally using the ISO. Its annoying, but casper is the ubuntu way. There are better ways to boot a live system off EFI but this would be the quickest path

[Edit 2] where you put grub/grub.cfg in the above partition layout is also up to you. It could reside on the EFI parition. Its all about how you lay it out in the host system when putting this disk together and then how you run grub-install

I'm... not following a 100%. But getting there. Its frustrating to feel like your 98% there, and just missing the one thing :/.

My setup right now _is_: (hd0,gpt1): ESP

    - /boot/grub/grub.cfg
    - /boot/{i386-pc,locale,x86_64-efi}
    - /EFI/BOOT/BOOTx64.EFI

(hd0,gpt2): ext4

    - ubuntu-16.04.bla.iso
    - custom.iso

I don't see the difference in grub.cfg(appart from 14/16, that was mb, I tried both 14/16 standard isos :D), and indeed its location doesn't matter as this works for the default live cd/it does seem to find the things inside the custom.iso/casper.

------------------------------------

And I totally agree with the part regarding the Casper scripts ;). I tried the following:

    - mount default 16.04 iso
    - take out its vmlinux.efi & initrd.lz
    - put it in the casper/ folder of the custom.iso

This booted! :). And then it hang on the 'not enough loop devices' error from the post below. :(

So I tried grabbing either the vmlinuz-LATEST_VERSION and the vmlinuz-LATEST_VERSION.efi.signed for the vmlinuz.efi in the casper/ (with corresponding initrd.lz), but those crashed even earlier. Just "error: no suitable video mode found. booting in blind mode", which is not the actual critical error, as the default 16.04 proclaims this as well, and then proceeds to boot properly in blind mode. Can't really seem to figure out how far the custom version got...

You happen to know of a listing of sorts regarding the required Casper files? I just get some general stuff. It really seems to indeed hinge around those vmlinuz.efi and initrd.lz files. The naming convention with the .efi and .lz shouldn't even matter that much, as long as the grub entry matches it. But those _are_ the images the host OS is booting from. Do they have some internal pathing/referencing I'm missing?

Something that keeps staring at me though is the /grub and /EFI inside the .iso. The iso basically has:

    - boot/grub
    - casper
    - EFI/BOOT/{BOOTx64.EFI,grubx64.efi}

But I'm not too trusting of the grub.cfg inside the iso...

    menuentry "Boot custom iso" {
	    set gfxpayload=keep
	    linux	/casper/vmlinuz.efi  boot=casper quiet splash --
	    initrd	/casper/initrd.lz
    }

I do think I need it, but does it need more references than this? Should it also have the (hd0,2)etc. prepending? Or is this OK as it is a selfreference within the iso?

Thanks for the help so far, its already comforting to know that someone else suspects the same area for the issues :). I swear this is going to drive me properly mad one of these days.

You're welcome! I definitely think this is an internal pathing issue relative to the ISO -- and I have grappled with this same stuff before many times and gone mad several times over so I get it :)

I just found a nice reference that I remember using when working with Ubuntu ISOs before at https://help.ubuntu.com/community/LiveCDCustomizationFromScr...

It may be outdated and you may have already seen it, but the key point is the section "Install Packages Needed For Live System":

> apt-get install --yes ubuntu-standard casper lupin-casper

> apt-get install --yes discover laptop-detect os-prober

> apt-get install --yes linux-generic

Since they're building the ISO from scratch using debootstrap and a chroot, some scripts in the casper package will probably set up paths, grub, etc, all relative to the current root (the chroot they're working in).

This would also probably explain why using a vmlinuz.efi and initrd.lz from a default 16.04 iso kinda worked for you -- because some paths and versions match up, etc.

Long story short: you should try following that guide to some extent when packing up the custom.iso (unpack, install casper package(s) in a chroot, repack), then find a way to use grub on your ESP to "chainload" [1] the iso from the second partition.

HTH, I'll check back later and see if you had any luck :)

[1] https://wiki.gentoo.org/wiki/GRUB2/Chainloading#ISO_images

Sounds reasonable and looks interesting ^^. Will take a look at it first thing tomorrow ;).

Sorry for the late reply, had to work through a few iterations... But it works now ^^.

Turns out the issue was not really in the grub and or initrd/vmlinuz...

I worked through your LiveCD guide, but did not en up with a working iso in the end :(. However, it did remind me of this guide: https://help.ubuntu.com/community/MakeALiveCD/DVD/BootableFl... on which distroshare is based. Running through the original guide resulted in a bootable iso. After checking, none of my previous homegrown isos were ever bootable on their own :O!. Apparently the level of black-magic in Unetbootin was greater then I had anticipated.

Best part of all, that iso is created on a non-EFI machine, and boots in EFI-mode just fine due to the grub-install --target=x86_64-efi on the bootstick. Everything else is just loaded over the original EFI-modules apparently. So no need to completely revamp all of my old code, just need to take a looksy into the iso generation part (used mkisofs instead of grub-mkrescue in my old scripts because of >4GB fat32 issues).

Thanks for all the help and interest man. It really kept me (mostly) sane ;)!

Np, I'm just glad you got it working! Like I said I wrestle with the same issues almost every day. I think it pays off in the end though because you come away with a deeper understanding in general and start to see the bugs other ppl don't , so keep hacking.

And this whole exchange has made me want to keep a detailed journal of this kind of work because I am doing it and would like to share knowledge more rather than just immediately forget. You should do the same!

Correct. Surprisingly all the grub2 parts seem to be in working order. The issue seems to lie somewhere with the vmlinuz.efi/initrd.lz... I can follow the grub entry to my custom.iso, but the system hangs with "error: no suitable video mode found. booting in blind mode" if I use the vmlinuz-version-generic and/or vmlinuz-version-generic.efi.signed from the host OS. (the regular ubuntu-16.iso also displays the same message, but then proceeds to but correctly. So that shouldn't be the issue here). But it does load when I copy over the /EFI and casper/{vmlinuz.efi,initrd.lz}. However it again eventually stalls with: BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash) Enter 'help' for a list of built-in commands.

    (initramfs) losetup: could not find any free loop devicet go on FAT32 partitions.

[I somehow can´t seem to get the formatting to stick with this next part, sorry :S] After: Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/nfs-top ... done. Begin: Running /scripts/nfs-premount ... done. Begin: Running /scripts/casper-premount ... [ 10.177576] random: nonblocing pool is initialized [10.473862] SGI XFS with ACLs, security attributes, realtime, no debug enabled [10.479729] JFS: nTxBlock = 8192, nTxLock = 65536 [10.627678] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null) [10.776705] FAT-fs (sdb1): error, fat_get_cluster: invalid cluster chain (i_pos 1) [10.776720] FAT-fs (sdb1): Filesystem has been set read-only [11.029557] EXT4-fs (sdb3): mounted filesystem with ordered data mode. Opts: (null) done. Begin: ...waiting for devs... ... done. touch: /dev/.initramfs/lupin-waited-for-devs: No such file or directory stdin: Not a typewriter mount: mounting /dev/sdb1 on /isodevice failed: Input/output error warning: Cannot mount /dev/sdb1 on /isodevice stdin: Not a typewriter [11.689807] FAT-fs (sdc1): Volume was not properly unmounted. Some data may be corrupt. Please run fschk. [11.740150] EXT4-fs (sdc2): mounted filesystem with ordered data mode. Opts: (null) done. ioctl: LOOP_CLR_FD: No such device or adress [12.524128] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) ioctl: LOOP_CLR_FD: No such device or address ioctl: LOOP_CLR_FD: No such device or address ioctl: LOOP_CLR_FD: No such device or address stdin: Not a typewriter [13.247442] FAT-fs (loop0): error, fat_get_cluster: invalid cluster chain (i_pos 1) [13.870988] EXT4-fs (loop1): mounted filesystem with ordered data mode. Opts: (null) ioctl: LOOP_CLR_FD: No such device or address ioctl: LOOP_CLR_FD: No such device or address stdin: Not a typewriter ioctl: LOOP_CLR_FD: No such device or address unable to open '/dev/sdc' ioctl: LOOP_CLR_FD: No such device or address [18.328866] EXT4-fs (loop1): INFO: recovery required on readonly filesystem [18.329151] EXT4-fs (loop1): write access will be enable during recovery [18.356624] EXT4-fs (loop1): recovery complete [18.367799] EXT4-fs (loop1): mounted filesystem with ordered data mode. Opts: (null) ioctl: LOOP_CLR_FD: No such device or address ioctl: LOOP_CLR_FD: No such device or address /init: line7: can't open /dev/sr0: No medium found /dev/sr0: No medium found stdin: Not a typewriter ioctl: LOOP_CLR_FD: No such device or address [27.398618] hidraw: raw HID events driver (C) Jiri Kosina [27.401012] usbcore: registered new interface driver usbhid [27.401366] usbhid: USB HID core driver

    BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
    Enter 'help' for a list of built-in commands.

    (initramfs) losetup: could not find any free loop devicet go on FAT32 partitions.

/dev/sdc is the location of my usb stick when I'm running my regular, hdd booted OS. The errors seem just... weird... It's not even technically out of loops for as far as I know, because those could go all the way up to device loop7.

Reply on news.ycombinator.com