Investigating a vanishing BIOS on the Fujitsu Lifebook AH532
blog.timschumi.net
blog.timschumi.net
Someone clearly thought that was a good idea, and I wouldn't be surprised if they thought the same of the bloated monstrosity that is UEFI. "Let's make the setup an EFI application" sounds like a reasonable argument, but they don't realise that it's a very important application, one which should be accessible under all circumstances short of having the BIOS erased.[1]
We're approaching 10 years since this happened: https://news.ycombinator.com/item?id=5139055
And almost 8 years since this: https://news.ycombinator.com/item?id=11008449 https://news.ycombinator.com/item?id=10999335
[1] Most if not all BIOSes on EEPROM (late 90s onwards) before UEFI had "boot block recovery" which would automatically detect if they were corrupt and attempt to recover by flashing from a specially formatted floppy disk.
Profoundly annoying, but, thankfully not commonplace and those couple times it's happened to me I've been in the return window, so, back they went. I figure that's as big of an FU as I can muster, the OEM dealing with a higher than normal return rate.
Last time it was a HP laptop running a god aweful bloatware infested version of windows 8.1, never again.
These variables are what `efibootmgr` lists and can change on Linux, what `bcdedit /enum firmware` lists on Windows, and what GetSetVariable can manipulate on Windows cf https://github.com/ProSlatisa/GetSetVariable/tree/master/Var...
Ultimately, each are OS-specific solutions, while it would be interesting to make a crossover between efibootmgr and GetSetVariable (and that C program) to create a tool working on both Linux and Windows with cosmopolitan to restore/hack 8BE4DF61-93CA-11D2-AA0D-00E098032B8C variables, because a quick search on that magic shows some people have uploaded their efivar to github for other models so it must be a common issue!
I seems that nobody except for Linux (for some not yet determined reason) is having issues with retrieving EFI variables on this hardware, and one could potentially classify this as a bug in `efibootmgr` as well (due to how it handles creating the new entry in unknown conditions).
In either case, Linux is the only thing affected by this, so in real-world setups Linux is going to be the only boot option that is available while the boot menu is in a broken state.
Especially in laptops, a lot of hardware / firmware issues are simply "solved" by baking a fix into the pre-installed Windows version. It's a solution for 99% of users, so why bother spending time looking into the root cause?
From TFA: "Note: At this point, I checked that Windows and various other UEFI tools are able to read the variables just fine, so Linux’ output is confirmed to be incorrect."
UEFI is a bit complicated and it's well known the EDD3 specifications can cause issues to efibootmgr, for example on Dell https://github.com/rhboot/efibootmgr/issues/86
I just think it'd be nicer to have a multiplatform way to tweak UEFI boot variable, so you can fiddle with your UEFI variables from either Linux or Windows without having to actually go into the UEFI shell or use a PE32 like RU.EFI : https://ruexe.blogspot.com/
> Especially in laptops, a lot of hardware / firmware issues are simply "solved" by baking a fix into the pre-installed Windows version. It's a solution for 99% of users, so why bother spending time looking into the root cause?
The Windows versions I installed for testing were non-OEM versions. They still behaved as expected.
Notably, Windows didn't just know about all the standard UEFI variables, but also about a non-standard one that I added for testing. This means that there definitely is a way to ask for the list of variables so that the UEFI accepts it (sadly, reverse engineering that is a pain), and that the Linux kernel is most likely the place where an actual fix has to happen.
Of course, yes, at the end of the day, the root cause is a specification non-conformity in the UEFI itself.
You did most of the work already, and it's super interesting (or at least, it's the kind of things I find super interesting lol) so you may want to finish fixing the issue?
It's funny how outside RU.EFI, there're no nice tools for such a basic features as tweaking UEFI variables, so if you are into this kind of things, you may also be interested by writing a better efibootmgr: many people (including myself, and now you) are dissatisfied by the issues it can create: https://old.reddit.com/r/archlinux/comments/18j6o7x/rfc_what...
I've never really seen a single example of this. Could you provide some?
As it was advertised as OpenSUSE compatible (and even came with stickers and such) first one or two times they made a fuss about it, but after that they replaced pretty much on the demand.
I wonder if that was also related to the issue article was mentioning. But such design lasting for multiple years?
UEFI was supposed to make this better, I guess, by specifying everything in several hundred (if not thousand) pages, but it added a lot of complexity (esp. with Secure Boot and such). It doesn't help that most end-user visible firmware functions are rarely accessed, so as long as it does boot Windows in the manufacturer supplied hardware configuration, most people won't care or even know of any issues.
PC firmware bugs aren't really anything new. When ACPI first came out in the late 90's/early 00's, initial implementations were buggy - so buggy that I think starting from Vista, it won't boot if the BIOS date is before 2000. Linux source code has numerous BIOS workarounds in it.
El Torito, the standard enabling bootable CDs, also had problems when it first came out.
Another example:
https://askubuntu.com/questions/521293/an-ubuntu-command-bri...
I remember the opposite problem being the case on the T420 BIOS. If you didn't set a newly added entry as NextBoot, it'd just disappear after reboot.
The main issue with fixing it properly is that I'd most likely have to reverse engineer the Windows kernel or the UEFI firmware itself (note to self: I haven't yet checked whether any of the *BSDs can read EFI variables in general and on this hardware in particular) to figure out where the request is going wrong/what Windows is doing different.
It's not impossible, given that one can unpack the UEFI PI firmware image into all the separate modules, but going through them to figure out where variable management is implemented will still take me a few weeks at least (not due to any particular challenge, it's just consuming a lot of time that I don't have right now).
This could happen either through somehow getting logging from the Windows end, or somehow changing the UEFI to be one you control and logging there, or finding a different BIOS/OS that can read the vars and getting it to log its work.
The userspace interface is somewhat documented by third-parties (because it is technically internal). However, the important parts happen kernel side, and I'd rather avoid diving too deep into Windows because some very interesting job postings (understandably) have "No exposure to Microsoft code or reverse-engineering of Microsoft software" in them.
I already tried getting to the service handler implementation via Linux, but memory protections made it weird enough that I was even questioning whether it was returning correct raw data when trying to read it from memory (or I have been looking at the wrong set of headers).
wouldn't that be trivial simply using a VM?
I don't know a thing about BIOS internals, so this might be completely irrelevant
To the untrained layman like me, this sounds like Windows actually is querying via `GetNextVariableName`, because UEFI doesn't seem to offer any other interfaces that aren't "get/set variable by name".
Before doing such complicated things, have you tried with RU.EFI?
Grab the memstick image from here: https://download.freebsd.org/releases/ISO-IMAGES/14.0/
Uncompress it, and dd it to your USB drive. (dd if=FreeBSD-14.0-RELEASE-amd64-memstick.img of=/dev/sdb bs=1m conv=sync, assuming sdb is your usb stick..)
In the old BIOS + ACPI days, the OS carries hardware specific hacks. These hack were buggy and hard to keep to day.
We (the community as a whole) decided it is better leave the hardware specific hacks to the hardware, UEFI was supposed to provide enough abstraction for all we need.
The result is, of course, the hacks with all its bugs are moved to the firmware.
More realistically, we now have them in all the places because vendors. :)
Good news: FreeBSD is able to read the whole variable list.
Bad news: I just wiped my test drive with all the in-progress kernel patches by accident.
I'd love a footnote explaining how you did this.
Had to reflash an SOIC i2c chip before and just soldered wires on. Some pins are easier where you have a via to solder to instead of the pin. Had to carefully lift the VCC pin to avoid turning on the whole board. Not all is lost if you break a pin: can carefully file down a bit of the plastic chip encapsulation to expose some pin to rebridge it once complete.
It's really nice to see how much cheaper ZIF sockets, test clips and programmers have gotten over the last few decades.
A process note. I did not really know what I was doing. and nothing was flashing correctly. Most of my problems ended up being with the cheap soic clip I had bought. After buying a nicer one it flashed first try. If anyone wants a recommendation, the nicer clip was from pomona electronics.
https://www.pomonaelectronics.com/products/test-clips/soic-c...
I can +1 for that clip. Had similar issues pulling data off a flash chip. If `flashrom` could see it at all, each read would come back with a different `sha1` hash.
Spent _way_ too long fighting that before I had the "wonder if this cheap clip is the issue..." thought. The pomona clip is so much better made and holds on to the chip well.
Had there been any more than one possible type of chip and more than three of these similar looking chips on the [accessible part of the] motherboard, I'd probably still be sitting here trying to figure out what to do.
In any case, I'll try and add it to the blog post once I figure out how to do footnotes. :^)
Your programmer needs to operate at the same voltage as your target.
I assume if this programmer is designed as a dev tool, it probably has a jumper somewhere to set what voltage to operate at.
The 25Q32BVSIG operates on 3.3V, so this particular programmer was compatible by default, but for 1.8V one would have to watch out indeed.
The BIOS never stood a chance. Maybe it was self deleting to hide evidence.
Thanks, I'm here all night.
Consider 'I got caught' and 'I got hit'.