Trusting-Trust Attack against an Entire Linux Distribution
arxiv.org
arxiv.org
A utility such as strip sustains the attack whenever a tampered instance participates in the rebuild, even after the compiler seed shrinks to a few hundred bytes.
The question that should have been addressed is “does the tampered instance participate in the build?”, and the whole point of these minimal-byte “seed” bootstraps is that they are just enough bytes of executable to do everything else from the source. So, no, the tampered instance won’t be participating in the build. We might even deploy our own tautology and say that if a pre-existing tainted binary participates in a build, then that build was not a bootstrap build, and therefore all bootstrap builds are in fact perfectly and definitionally immune to the attack.
Honestly, this whole paper kinda reads like an academic version of a slop CVE.
It is precisely about showing that you can still propagate backdoored code if the compromised binary in your seed is NOT the compiler.
Then why have section 7.3 'Bootstrappable Builds', that briefly surveys a type of trusting-trust mitigation, and concludes that their method "sustains the attack [...], even after the compiler seed shrinks to a few hundred bytes"?
>It is precisely about showing that you can still propagate backdoored code if the compromised binary in your seed is NOT the compiler.
Whatever this paper is about, it is certainly not precisely about it. But quibbles aside, if that truly is what the authors set out to show, they should probably re-read the transcript, where Ken says "[For demonstration purposes], I picked on the C compiler. I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode."
GUIX is unconfortable about that, but as they name the current process "full source bootstrap" implicitly consider this unaudited execution environment out-of-scope and more of an academic blemish, rather than a real problem: https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
After all you rebuild it together with the world immediately after compiler bootstrap is done. And the utilities in your execution environment do not process the compiler source, so this dependency cannot be a problem, right?
The paper precisely addresses this. You cannot handwave away your shell or ELF utilities. They can sustain an attack just like the compiler.
> "[For demonstration purposes], I picked on the C compiler. I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode."
This is precisely the problem that most people dealing with OPSEC have.
If you get "paranoid enough", your threat model always collapses into an unactionable mess where everything is just toast and your only solution is to quit computing and start doing woodwork or something.
If everything is potentially compromised then of course the results from the paper become obvious. If your environment is compromised, don't trust its output, duh!
If you want to quantify though, under which assumptions, "how much" your environment can be broken without compromising its output, the paper has value. And it seems like real attempts of bootstrapping binaries actually missed a spot.
But that seems like quibbling about semantics. Thompson says that if a compromised binary generates a binary, you cannot trust the generated binary.
This does not even have to be gcc, it could be vim. Or strip. Or cat. Or dd.
Or an iop to write data to persistent storage if the firmware of your SSD is compromised.
Just write a new OS. It's a weekend project to get enough groundwork that you can bootstrap a clean system from clean source code.
> And wouldn't there be difficulty comparing binaries built from significantly different environments?
Not really. Starting from stage 0, compile the compiler under test (stage 1), then use the compiled compiler to compile the compiler (stage 2), and compare the stage 2 artefacts. Provided that your comparison program is known-good, and the stage 2 build is deterministic (not the case for some real-world programs, but true for things like tcc), this lets you verify that the two compilation procedures work identically.
get ready.
the effort required for a complete infiltration has been lowered a great deal.
simple single bit changes are enough to blast your private keys out to the ether of the public facing internet.
that big fat LLM does know how to make tentacles and eyes.
But they're not going to backdoor an Apple ][e, or a random 80m¢ microcontroller, for basically any value of "they"; so you can just use one of those instead, and save yourself the hassle.
Individual cpu instructions, even of a crude old 8-bit cpu with no embedded minix os like today, are both simple enough for a human to manually understand what they do, and useful enough to build crude versions of useful things like an editor, interpreter, or compiler.
You can write a forth-like language or even a c-like language starting from individual cpu instructions that a human can read, understand, and write totally manually, and then use that to build up rapidly all the way to a full modern desktop.
If you were really paranoid about the very act of the initial typing-in, there are any number of ways to store data in a totally brainless eprom or record it to tape or something, and examine it with nothing but some leds, no cpu at all, to verify the bytes are the bytes you want. And you only need to do that for a pretty small number of initial bytes. After that it's all just regular source code which could be written on paper.
Bootstrapping is only an inconvenience problem, not a real problem.
It's not convenient for most people to assemble some bytes into some storage medium and then verify them without simply using a normal untrust-able computer to do it. But it's no problem really if you had some reason to be that careful.
if you're concerned about this and not on x86, i encourage you to extend this to other platforms! i believe it's possible to generalize this to every linux platform musl itself supports, in time.
[1] https://fransfaase.github.io/Emulator/tdiagram.html
[2] https://fransfaase.github.io/MES-replacement/
[3] https://fransfaase.github.io/MES-replacement/Tdiagram.html
So, your program that combines source files or checks dependencies would be fully specified in its success and failure states. Only combinations of functions leading to a provably-secure state are even allowed. If you can't do that, the feature is too complex to allow. Human pentesters review it from design to algorithms to building it to spot ways attacks might happen.
That's what it takes to build software that usually resists subversion. Most software isn't built that way. It can't be because the priorities of developers and customers work against it. So, we'll continue to see clever attacks that exploit systems not designed to high security standards.
For this topic, I recommend David A. Wheeler's page on Software, Configuration Management Security because it covers many issues with it in mostly-centralized systems.
The most significant such priority may be the costs of paying developers and of time to delivery.
The dramatic reductions in those costs due to LLMs enable us to produce much more quantity and/or quality. Many complain now about quantity, so perhaps we are finally at a stage where we don't need much more software, and can focus on quality. Also, LLM attackers create demand for higher quality.
In other words, LLMs might enable us to some of these things that were impossible before.
The LLM might take longer than otherwise, but still be fast enough. The new technology, like every useful one, changes the tradeoff equation.
https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-...
If you have malware on your CI machine it can infect the artifacts it creates.
In this paper the malware was a strip trojan, but it could have been just as well a normal piece of malware which starts a service and then waits for the final artifact to be generated after which it infects it.
However, anything in the build toolchain that someone uses to build those widely-executed executables necessarily does have permission to write to them!
This is certainly not novel, having been the subject of a Turing Award speech, but it is something that many people haven't thought through.
> Ken Thompson's trusting-trust attack [...] is widely regarded as a threat specific to compilers. We show that it is not
And yet, from Reflections On Trusting Trust:
> In demonstrating the possibility of this kind of attack, I picked on the C compiler. I could have picked on any program-handling program such as an assembler, a loader, or even hardware microcode.
The paper is certainly a nice worked-out example of the attack, which is worthwhile, but it's not novel.
But this one is crippled because there is only one file that can infect other files.
If you know, than you know why RISCV was fast-tracked in some places... =3
It would take at least some knowledge to hack, not just a random script from a forum.
Now, with LLMs, it's the '90s all over again.
If its easy enough to find exploits, its likely similarly easy to scan code for exploits, or use AI-based anti-virus technologies. The only thing holding us back is the cost of compute. We can't all run the latest models against everything.
Ken Thompson's Trusting Trust is an interesting conundrum... there are ways to minimize the need to trust however, if you can express everything in the same language or common runtime: