hlandau

The TKey is a good design here.

TPMs are a red herring here though, as the TKey is not a plausible replacement for a TPM, which exists to measure a platform boot process. There's not really any way to use a TKey for this, since a) you'd have to load firmware at every boot before the first measurement is taken (i.e., before the BIOS even starts running), which no PC is setup to do, b) you would still be vulnerable to classical MitM of the device, as for any discrete TPM, and unlike modern functional TPMs.

The lack of controlled storage it should be noted does create vulnerability to rollback attacks. It's not really possible to delete data this way.

In any case, with regards to the lack of user-programmable secure elements, it's the industry attitudes here that are the problem. This kind of technology absolutely exists, but it's all under NDA and you can't have it. Smartcards are the most obvious example; you can get nice flash-based programmable smartcards now with 32-bit ARM cores, and no you can't have one. It's ridiculous.

So the TKey is built out of a COTS FPGA, one of the few FPGAs with an open source toolchain (painstakingly reverse engineered). This means it doesn't have any of the silicon hardening that smartcards and other secure element chips have - but there's no choice but to build out of something like this because those chips are all under NDA. The hardware industry doesn't seem to believe in Kerchhoff's principle.

IMO the TKey is basically the best you can do with the publicly available silicon today. In that regard it's pretty good. But a TPM is literally the one application it's least suitable for as a secure element.

You don't see this with curl/wget because they use user agent sniffing. If they don't think you're a browser they _will_ give you a Location header. To see it, capture a request in Firefox developer tools, right click on the request, copy as CURL. (May need to remove the Accept-Encoding tag and add -i to see the headers).

Nothing is ultimately pure since you're running on top of a kernel which can affect computation. It's unclear to me what practical gain you get from trying to obsessively get "maximum purity", which is in any case illusory because you're still running on a kernel from outside the closure.

Making it impossible to write portable code isn't a good design.

This problem seems to be solved commonly in two ways:

- People thoughtlessly and senselessly listing only x86;

- People using the popular flake utility repository.

Having to rely on an unendorsed third party just to write architecture-independent code isn't very appealing to me. This seems like a serious deficiency in the design of Flakes, though it would be tolerable at least if this functionality was at least in Nixpkgs.

It turns out a lot of high-power circuit breakers need the energy from a clockwork mechanism in order to open in the event of a fault. So the thing where you have to pump the handle is actually to charge a clockwork mechanism to ensure there's enough mechanical energy to open in the event of a fault - AIUI. Presumably, the breaker is designed with safety in mind and won't let you push-to-close until you've done this.

My earliest exposure to this concept as a child was watching the film Jurassic Park. As someone fascinated by systems I found the idea of having to bring the whole system back up from scratch pretty interesting.

Today I still find these kinds of bootstrapping processes fascinating - both these megascale processes, but also the boot process that occurs whenever you turn on your computer. The latter is probably one of the most Rube Goldbergian feats of engineering with us today that actually still achieves a useful purpose. In fact it's absurd how Rube Goldbergian it is. And the complexity of the boot processes for modern systems (see [1] for a small glimpse) is extraordinary.

When you turn on your computer, it's like you're re-executing the entire process of a civilization bringing itself into being, gradually developing progressively more sophisticated technologies: at first RAM isn't working, but then you get RAM working and that lets you get progressively more sophisticated parts of the hardware working, etc.

By comparison, humans have no "automatic boot process". We're constructed in the 'on' state via fork(). So this repetition of entire process of, ah, 'abiogenesis' whenever you turn on your computer is kind of insane by comparison. Entire kingdoms of hardware state rise and fall with the press of a power button.

As an aside, I'm fond of the Red Dwarf novels, which are set on a massive mothership-type spaceship. The ship was constructed in space and never designed to enter a planet's atmosphere. In particular, the ship, and its engines, was constructed in the 'on' state by the crew that built it originally. It was never conceived that the ship would ever need to be rebooted, because it was assumed once the engines were initially fired during the commissioning of the ship, they would never be turned off until decommissioning. Thus, the ship has no automatic boot process for the engines, only a manual engine firing procedure which is extraordinarily arduous and long-winded and takes weeks to execute, said procedure having been included in the manual only as a curiosity more than anything else. This idea of a ship built "on" under the assumption it would never once be shut down or "restarted" until decommissioning is interesting, but of course also directly mirrors biological life.

[1] https://www.devever.net/~hl/backstage-cast

https://www.youtube.com/watch?v=4Ir5b0GzfiE

Some favourites:

- https://www.devever.net/~hl/ruthlessness - Computers are an inherently oppressive technology

- https://www.devever.net/~hl/mildlydynamic - The Demise of the Mildly Dynamic Website

- https://www.devever.net/~hl/ortega - Adventures in reverse engineering Broadcom NIC firmware

- https://www.devever.net/~hl/sip-victory - Netheads vs. bellheads redux: the strange victory of SIP over the telephone network

- https://www.devever.net/~hl/power9tags - The Talos II, Blackbird POWER9 systems support tagged memory

- https://www.devever.net/~hl/backstage-cast - Modern CPUs have a backstage cast

Once you have this, this means there now two nodes which need to be coerced to compromise the system - endpoint software provider and communications provider. Well, actually, just one - the endpoint software could just be changed to use a different, compromised communications provider (I think you were envisaging some more subtle compromise of ciphertext above).

This can be fixed, however. A lot of the work Linux distros have done around reproducible builds, and the new focus on release transparency, is informative here. You could absolutely have a FOSS project with a release process that requires multiparty signatures of releases with the release signers in different jurisdictions, and requiring proof of logging to a release transparency log, etc.

I suspect a lot of pushback on this is because the implications are significant, Emperor's New Clothes and all, but fixing this sort of thing is the first step to solving the problem. Pretty much the entire internet community was in a mindset of "but government wouldn't ever do..." complacency before Snowden, despite increasingly blatant warning signs, before people were forced to accept that reality. (The reason is simple; it's a funny thing about humans, when they realise some fact X being true would be very materially inconvenient to them, their first reaction is to argue that X isn't true, as though reality is negotiable...) Having an accurate view of just how impaired our security model is in this area is the first step to making things better IMO.

It is worth noting that if we ever run out of space in 21-bit Unicode, one could conceivably re-extend it back to the original 31 bits by throwing UTF-16 under the bus. The only reason Unicode is limited to 21 bits is because of UTF-16.

See comment by bri3d below for details. It appears they're just sending a command to the ME politely asking it to stop doing things, maybe. Of course, this happens long after the ME has already done a great deal of work bringing up the system.

Of the three options for ME scope reduction, none are good and none actually "disable" the ME, but it seems like they've chosen the least effective/audited option of the three. I should add that if you don't trust the ME not to be owned, there's not really any reason to trust that it will honour a polite request to stop doing anything sent to it, and you can't trust it not to have compromised the boot process anyway, making this rather pointless.

https://github.com/openbmc/openpower-proc-control/blob/maste...

The very first instructions executed by the SBE are from OTPROM (one-time programmable ROM, set at the factory). This is just a few instructions: https://github.com/open-power/sbe/blob/master/src/boot/otpro...

The first SBE code embodied in a mutable nonvolatile storage device executed is here: https://github.com/open-power/sbe/blob/master/src/boot/loade...

Interestingly the SBE code is actually stored on a serial I2C EEPROM on the CPU module itself, not on the host. This is quite unusual from an x86 perspective where there is usually not any mutable nonvolatile state on the CPU itself.

POWER9 is also a little unusual in that just powering on the CPU doesn't cause it to do anything by itself. You have to talk to the CPU over FSI (Flexible Service Interface), a custom IBM clock+data control protocol which is mastered by the BMC and used by the BMC to send a 'Go' signal after powering on the system's power rails. This FSI interface is basically the CPU's "front port" - think of an operator front panel on an old mainframe. It's kind of fascinating how IBM's system designs still reflect this kind of past. Indeed, until very recently (I think this only changed with POWER8, but it may have been POWER7) IBM's own POWER server CPUs weren't designed to boot, in the sense of being brought up in a self-sufficient way - they were externally IPLed by the FSP (IBM's version of a BMC).

Basically, what we think of as boot firmware would actually execute on the FSP - which is a PPC476 running Linux - so you have the boot firmware executing as a userspace Linux program on the FSP, doing register pokes to get the CPU initialised, do memory training, etc. all remotely via the FSI master, rather than it being done on the CPU itself. It would even load the hypervisor, PowerVM. The CPU itself traditionally was essentially held in reset until the service processor had completed all this init.

So the POWER CPUs have traditionally all been initialised externally, rather than via 'bootstrapping' in the literal sense. Even memory training was done this way, all via the CPU's front port. What's really amazing is that when IBM wanted to switch to a self-boot model around the time of POWER8 (I think), they looked at their boot/IPL code, which was designed under the assumption it would be running on an external processor and initialise the CPU by probing it. Their response was to write a C++-level emulation environment for all the internal API calls this firmware uses to access the hardware. Basically, you have hardware initialisation procedures written as though running on a separate machine, and then a C++ framework all of this is hosted in which pretends that this is still the case, but allows all of these hardware initialisation procedures to be reused without having to rewrite them all. Amazingly it works. Though the size of this boot firmware is so large, it even has to have a paging mechanism for its own code while running in CAR mode(!). http://github.com/open-power/hostboot

With POWER8/9 since IBM moved to a self-boot model, the FSI interface is often just used to send the 'Go' signal, and I believe by the BMC to query temperature information after boot. But the CPU won't do anything until you send that signal. It's kind of cute, really. Sending the 'Go' signal starts the SBE running. And yes, this means that all of the hardware initialisation procedures which were written to be used from a separate service processor are now never used that way.

This kind of 'front port' based design to system 'IPL' in which a service processor just pokes the CPU remotely to initialise it is pretty fascinating. It's like the idea of automating what a human operator would have done to initialise a mainframe long, long, ago (makes me think of the autopilot in Airplane). Though of course the amount of stuff you'd have to do flipping switches on an operator panel to initialise one of these CPUs may take a lifetime if done manually...

So this is an example of how IBM's technical history very definitely lives on and is reflected in their systems engineering today.

Worth noting an advantage of this FSI interface is that it's also effectively a hardware debugger interface. In fact I believe the Talos/Blackbird systems ship with the 'pdbg' hardware debugger tool accessible via BMC SSH shell. So these systems effectively have a hardware debugger built in, which is just a Linux userspace tool which pokes the CPU's debug registers over FSI. https://github.com/open-power/pdbg

The idea of 'front ports' seems pretty rare nowadays in most SoCs. Or rather, it actually does exist: it's called JTAG. In that regard IBM's traditional boot-via-FSI model isn't so different from the idea of initialising a chip at boot using JTAG. ...I'm fairly sure I've heard some horror stories in which some systems actually do this.

Moreover Intel is just this week actually finally proposing removing real mode. [2] I'm a bit worried for what this means for emulation of old 16-bit Windows and DOS software under Wine (one of the great ironies that Wine can still run Win16 programs on an x64 host OS when Windows can't) - though I suspect the performance requirements of such software is so low by modern standards that emulating such programs wouldn't pose any challenge.

[1] https://www.devever.net/~hl/ortega [2] https://www.phoronix.com/news/Intel-X86-S-64-bit-Only

Points for mentioning this! But things have come a long way since 2008. You can get Intel ME-less machines from the 2008 era. Not sure if OpenSPARC T2 has any management cores.

>The maintainers of the me_cleaner script likely have the clearest view of what is known.

Yep, absolutely. Much of what we know is thanks to the efforts of researchers like these. See also the talks on finding the 'Red Unlock' mode of modern Intel CPUs.

PCIe in particular is literally a packet-switched computer network - it has a physical layer, data link layer, and a transaction layer which is basically packet switched. There are even proprietary solutions for tunnelling PCIe over Ethernet.

Pretty interesting. When AWS launched Glacier, people assumed tape, but that doesn't seem to quite add up relative to the characteristics of the product, and AWS has never said a word on the technology it uses. Then someone came out and started claiming they were using BDXLs, as I recall. Have to say, I did not conceive of "invent your own optical storage media" as a possibility.

My guess is Microsoft made this information public because AWS is doing something similar and they're the second-mover rather than the first-mover. But that's just my guess.

Video: https://www.youtube.com/watch?v=3RKpA1OiEFE

Also interesting to note how this demonstrates there is a ton of unharnessed potential to optical recording that isn't available in a product available to the general public. In fact, it seems quite telling that every increase in optical media storage capacity available to the public has coincided with a new video format (e.g. 128GB BDXLs for UHD Blu-ray). Since physical medium-based video distribution is dying, that doesn't bode well for the continued availability of optical media archival solutions that can keep up with contemporary capacities...

It's a very natural proclivity of a language designer designing a template language for language X to want to find a way to articulate that template language in that same language X also. I think most engineers can't help but love ideas like that. It's probably the same reason everyone who creates a programming language wants to make a self-hosted compiler for it.

In this case though, XML being a rather verbose and arguably limited semantic markup language for textual documents, it's an extraordinarily unergonomic choice for templating itself (in notable stark contrast to SXML combined with Lisp code, which is an example of homoiconicity between the structure being templated and the templating language works very well).

- "No one has structured HTML creation that's as easy as string templates." -> I think this is usually true, but not always. Lisp allows use of S-expressions to define HTML (via SXML), and I genuinely find that more pleasant and powerful than writing HTML, or HTML with string templates. JSX is also arguably easier than adopting a string templating system since you can just drop into it instantly and write it as a function; it's language-integrated.

I also actually classify languages such as Haml as structural containment, which might be counterintuitive, since they're largely just seen as a syntax sugar on writing HTML. But because these languages are completely different to HTML, they're necessarily defined and parsed as languages in their own right, and the capacity for interpolating variables, etc. is integrated into the definition of that language as part of its syntax. (Consider how you can't provide mis-structured HTML with Haml, for example.) So this is a bit of a surprising result where something which is just seen as sugar on top of HTML actually takes you from string templating to structural containment. Thus using languages like Haml is a simple and easy way to move to structural separation in ecosystems where Haml or similar technologies are available. Since the whole point of Haml is to make writing HTML easier/lower effort, I also think this is another counterexample to "no one has structured HTML creation that's as easy as string templates."

- "One of my fundamental rules of system design is when people keep doing it wrong, the people are right and your system or idea is wrong." -> Yeah. If web developers keep adopting the wrong solution, it does suggest a lack of better leadership in this area in terms of the web development tools and languages they're provided with. I agree with this constructive view of leading via positive example; it's not reasonable to blame people for not using the wheel if they live in a world where nobody has ever seen one.

- It's mentioned that Go came up with html/template rather than come up with a better solution. It seems like the problem here is that in order for a language to be flexible enough to allow articulating HTML in a structural way that is actually ergonomic, it basically has to either a) be as flexible as Lisp, or b) be a DSL or language extension specially designed for the task (JSX, Haml).

While Chris takes a different view here I find his POV interesting, so I've added a link to his page from mine.