Hell Never Ends on x86: The Hyperspace Story, Continued, Sort Of
cohost.org
cohost.org
Some company even sold laptops with two CPUs, one that would run some Windows CE while the main Windows was booting. Even that is just gluing a PDA and a laptop together...
BIOSes "perverting" sleep behavior? Sleep is precisely what the BIOS handles. E.g. way before Windows would support hibernation natively, laptops with hibernation capabilities would have to do it all on firmware. And it wasn't that fragile, since it would work with everything from MS-DOS to Linux (even the progress bar UI which was some VGA text mode). This is how I learn in the 90s that Windows likes to keep unused pages zeroed while Linux does not: hibernation under Linux would take significantly longer. But it'd work.
No one remembers firmware hibernation partitions? I'd guess that many people do, since Intel tried to resurrect that in the 2015s (e.g. https://www.dell.com/support/kbdoc/en-us/000136460/how-to-us... ).
Wait until you learn that MS has hooks for the firmware to download executables that will run during kernel boot time...
Could you elaborate? Sounds very sinister
Also there is an unofficial method of running firmware-initiated VBScript in the all versions Windows starting from Windows 2000. There is a special type of ACPI devices (PNP0C14, "Windows Management Instrumentation Device") which is handled by Windows builtin driver WMIACPI.SYS. This type of ACPI device can export (via ACPI bytecode) various data fields and methods for Windows subsystem WMI/WBEM (think CPU thermal sensors; laptop backlight control; on/off switches for builtin camera/wifi/bluetooth hardware, which can be managed by Windows tray utility supplied by laptop vendor etc), as well as binary MOF which contains WMI metadata for these fields and methods. Besides primary purpose, this binary MOF can be constructed in such way that VBScript code is executed immediately in the MOF parser context (wmiprvse.exe, a builtin Windows service). Windows DDK has various MOF-related tools like compiler, verifier etc; as well as some samples. Binary MOF is compressed, but its format has been reverse engineered (it uses an old compression algorithm from Microsoft DoubleSpace). This method for running firmware-initiated VBScript works for all versions of Windows starting from Windows 2000, including WinPE-based Windows Setup (Vista+).
References:
https://wiki.ubuntu.com/Kernel/Reference/WMI
https://raw.githubusercontent.com/microsoft/Windows-driver-s...
https://www.blackhat.com/docs/us-15/materials/us-15-Graeber-...
I've seen laptop OEMs that rely on a custom kernel driver for this, and then if you want to reimplement their pile of 'overly modular everything control suite' you're stuck loading a generic read/write-memory driver which is blocklisted by every antimalware and anticheat solution in existence.
(In addition, it seems some WMI/CIM/MOF ACPI stuff is supported on Linux fine, too, as long as OEMs try to stick to standard DMTF-spec stuff I guess... so one can also just do 'generic' user-space stuff there if lucky)
I'm not so sure about that. Especially when AFAIK you need to use a rather large toolchain to access the obtusely indirect APIs.
xchg ax. bp
mov dx, 107h
int 21h
ret
db "INT 21 forever!",13,10,"$"Weight that (literally only a couple macros) versus the idea of having to dig out a 16-bit toolchain and/or a DOS extender which also brings its own obtusely indirect APIs with it. In case you decide to do the former, you'll probably have to rewrite big chunks of your program in order to rearchitect it around XMS. In case you decide to do the latter, you'll be able to use a 32-bit toolchain, but you'll have to write your own drivers for high-resolution graphics, sound, and depending on your luck, mouse. All of this UEFI brings for free, and even for recent hardware.
All you need is the function signatures and the constants to lookup the protocols, it's no more than you need to use any other C library. I can tell you from experience that if all you want to do is textual IO you only need a few hundred lines of header files: https://github.com/liampwll/zig-efi-os
Edit: Here's a minimal example using clang from someone else: https://github.com/AndreVallestero/minimal-efi
That's what I mean by "obtusely indirect".
if all you want to do is textual IO you only need a few hundred lines of header files
"only need a few hundred lines"? That's huge in comparison to MS-DOS.
They could've made UEFI almost like MS-DOS in API too, just with a flat address space. Yet they came up with a monstrosity far more complex.
> An OS may choose to execute in a virtual addressing mode and, as a result, may prefer to call firmware services provided by UEFI Runtime Drivers in a virtual addressing mode. A UEFI Runtime Driver must not make any assumptions about the type of operating system to be booted, so the driver must always be able to switch from using physical addresses to using virtual addresses if the operating system calls SetVirtualAddressMap().
Disappointingly, it's just a 'plain' case of SMM (which is also, funnily, the main target for ACPI callouts to more 'complex' methods, and on ARM has the very fun equivalent called 'SMC/HVC callouts to EL3/EL2 firmware' although that's supposedly a bit better-defined than Intel SMM) and nothing too clever either - an ACPI method hijacking something to then write to the framebuffer?
Other than that, the writing style here feels almost.. presumptuous and might've been written for the original audience on the site it's posted on, for example I was left wondering things like 'if the author or their friends have access to a machine with this software on it, why don't they dump the firmware flash storage and decompile the actual images, wherever they might be', though it may be they indeed didn't have access to any such hardware or firmware downloads and are just piecing random puzzle parts together, and the post eventually devolved into a general rant against OEM cleverness that I somewhat skipped.
(I'm also mildly torn between 'wow, clever hack' and 'oh lord this would be terrible to diagnose OS boot issues on as you really can't trust OEMs with side effects or long-term maintenance' in this regard)
[1]: https://edk2-docs.gitbook.io/edk-ii-uefi-driver-writer-s-gui...
1) The PC, loaded with any security contraption you can think of and severely underprovisioned in RAM is deemed user-worthy by IT if it boots in 15 minutes or less. It may take 10 min from desktop visible to can actually move the mouse.
2) Work day start and end are logged in a web application, requiring a running PC.
3) Shutdown takes just at long.
4)Sleep modes don't work. Fresh boot every day, or weird crashes will haunt your whole working day .
So how, as a worker drone, do you avoid adding an hour of working time just waiting to start and end the work day?
Very predictably, once end users found out the firmware based browser, there was no stopping them. And then they found the browser could surf to sites without said security contraptions blocking them.
Most people in the decision making grade have absolutely no clue about tech, be it software or hardware in general.
It also feels like the case of some higher up VP-of-whatever not realizing how atrocious the implementation would be, and just ordered it to be done. Nobody below them in the development or engineering teams dared to chime in and protest. Those who protested probably found themselves without a job, or replaced by some overseas contractors who never object to anything (and are usually cheaper per line of code written too, a win-win, surely!).
I don't see any "rate your experience" popups, nor login/signup prompts, nor ads, nor endless amounts of whitespace and the (few) options hidden behind a catch-all "burger" menu.
This thing was actually built to a much higher UX standard than most software nowadays.
Well, I bought a ZBook from HP in January 2014 because of these Features:
1. All the laptops I worked with (my company's or my own) were HP and they were good. They were of the Compaq lineage, not the consumer specs.
2. They have on site next business day support as long as I pay a warranty. That was less than 100 Euro for the first three years. I don't have to bring the laptop somewhere or leave it there for days or weeks.
3. I can open it and replace basically every single part.
4. I knew it would run Linux reasonably well. That means perfectly as long as I don't care that shutdown is a reboot and I have to long press the power button at the BIOS screen to actually shutdown. Maybe I didn't shutdown once since January this year.
5. It has three physical buttons on the touchpad. I wish it didn't have the numberpad.
6. It could accommodate up to 32 GB, which it has now. Plus 2 SSDs, 3 TB total. It came with a 750 GB HDD.
So, these are the kind of features I care about. Software gimmicks in the BIOS not really, nor weird multimedia buttons.
Until the pandemic hit and we had three days to get more than a thousand people working off-site.
I don't find this stuff negative at all, unlike what the author seems to be implying; instead, it reassures me that there's still plenty of old-school "hacker spirit" left in the industry, if you know where to look.
He told me about a project he had worked on where they collaborated with Microsoft to make something that would make the OS boot significantly faster working with their hard drive. I'm filling in blanks in my memory here, but I think it involved a combination of putting some files in flash memory (hybrid disk) as well as storing files on the spinning disk in the order they'd be accessed.
It worked! They were able to make boot process significantly faster. But consumers would never see it.
The punchline was, it turns out many peripheral devices and drivers rely on the boot process being slow that all kinds of race conditions and intermittent bugs and glitches turned up as soon as they started testing on a wider range of PCs. Even if you solved the dependency and order of scheduling problems on the main thread of the OS loading, devices may have microcontrollers that are slow to come online or drivers initializing independently.
(You might object well doesn't the hardware or driver have a "ready for first query" flag or something like that. Well that depends on hardware manufacturers doing the right thing, thinking of that ahead of time to add it to the interface, and implementing it correctly. This would be something that had never been tested because previously there was always XX seconds before the OS tried to query the device for the first time.)
So the moral of the story is the boot process of x86 PCs is still slow because it has always been slow...
That's a running thread in the article above and its related posts: "the PC boot is slow, let's do weird BIOS and dual boot hacks" only made sense for one weird, brief moment in around 2009 between a dying breed of XP-era hardware and just before/as Windows itself decided to be judge, jury, and executioner of hardware boot up times.
I had a chuckle at the “distinguished technologist” on that slide though.
Update: I had the quicklook/quickweb stuff disabled in the BIOS, I enabled it and lo and behold the computer booted when touching the browser button. Cool.
There's plenty of room for unintrusive capacitive buttons on laptops, why did they stop adding them I wonder
Capacitive buttons are terrible.
"seeing you lay bare this altar to absurdity that HP has laid helps. perhaps what I have built is a little Extra, but I’ll never hijack the windows boot process in ring -1 to show my outlook calendar"
So what will be the next IRONCHEF called? :) DailyDriver?
Of course it's tech equivalent of attaching wings to horses and making them fly
I’d totally befriend a wild Pegasus if they still existed. Imagine having your friend Pegasus fly you everywhere instead of taking the bus for hours. Would have been so much better.
> Windows gets itself all ready for sleep, tucks itself in, then tells the BIOS "too eepy! time for bed, S3~" but instead of the S3 reaching the BIOS, OSM intercepts that call. The OS has halted itself, and is waiting to be told that it's time to wake up, but OSM instead proceeds to alter the universe around it ... the OSM will move all the memory pointers around, then reverse the sleep process, causing the machine to instantly "wake" - from a sleep it never actually entered.
> Windows thinks it's taking a nap, but it wakes up as someone else, leaves the house, and commits murders it has no memory of. In other words: Phoenix figured out how to turn the BIOS into a hypervisor. It's almost, almost beautiful, if it wasn't both incredibly fragile and completely uncalled-for. But christ. Christ almighty, what a hack. One could come to tears over it.
There is a software archaeology epilogue waiting to be written, based on artifact reverse engineering and a trail of public breadcrumbs:
Hyperspace was acquired by HP.
Hyperspace creator left HP and founded Bromium (acquired by HP in 2019).
Bromium productized OSS Type-2 Xen (uXen microvisor), copy-on-write VMs in 150ms.
Microsoft cloned Bromium vSentry as Defender - Application Guard.
Bromium AX ships on HP biz laptops /w nested x86 virt for OS+UEFI virt.
OSS Bareflank hypervisor = attempt at replicating AX (Windows-as-dom0).
OSS Android 13 pKVM-for-Arm is closest OSS in spirit to Bromium AX.
Industry needs an OSS Type-1 L0 hypervisor: https://twitter.com/stacktrust/status/1640145980833005568
Video with virtualization history of Xen (Type 1), uXen (Type 2) and Bromium/HP AX (Type1 L0 for nesting of Xen, KVM, VMware, Hyper-V): https://www.youtube.com/watch?v=bNVe2y34dnM)
pKVM baby step towards dropping SMM/TrustZone: https://twitter.com/stacktrust/status/1595951965699473408
I have not done that in many years, but still!
Most of what's happening here is pretty standard that's-how-a-minimal-hypervisor-works. The Outlook-data-migration part is interestingly janky, but not technically complex.
Generalizing massively, a simple hypervisor runs a small amount of code at boot to setup its own state somewhere, and injects its interrupt handlers into various places in the system, and finally chain boots the real OS. When the real OS tries to communicate with the underlying hardware, the hypervisor interrupt is triggered and it does it's stuff.
With modern virtualization acceleration, the hypervisor can enable and handle interrupts for an even wider range of events that it might need in order to efficiently interpose the OS.
A more full-featured hypervisor may start to look like a mini-OS with its own scheduler, etc. But a simple hypervisor is basically like an old DOS Terminate-and-Stay-Resident executable.
It's a ROM module with the strings "IBM V86" and "Secure OS boot support ROM", neither of which search engines seem to find any results for (perhaps deliberately?); neither now nor the roughly a decade and a half ago that I attempted to research it. This might now be the only reference to such a thing on the increasingly forgetful Internet.
This is the future, after AI breaks down and nobody know how their fancy contraptions work anymore.
tf-a boots a "trusted environment" (a secure OS, e.g. op-tee), which defers back to tf-a after initial setup. tf-a then boots the main OS (e.g. linux). apps can be loaded into the TE at runtime, the main OS can defer execution to the TE, and the main OS's scheduler can coordinate with the TE to allow pseudo-preemption. coupled with hardware modules (an SPU, TZC, and GIC) in the fabric, specific interrupts, memory regions, and hardware peripherals can be marked as "secure" (i.e. accessible only by the TE) or "non-secure" (i.e. accessible by the main OS).
what's unique (i think? i may be misunderstanding.) in this case is that the TE is an OS without its own scheduler. it depends on the main OS's scheduler to defer control whenever the main OS deems appropriate. each OS has directly registered their own interrupts, so the firmware is really only handling one interrupt: when the main OS defers to the TE.
According to user ‘zaarn’ here on HN, this is not feasible for most use cases:
Software has always been built in layers, repurposed for other things. This seems like a great example.
And it did make sense in the context of 2010 because it let HP experiment with non-Windows environments.
In another world maybe one of these would have become super popular.
Haha