Hacker News Reader

Comment by titzer | Hacker News Reader

> Since this flaw is not fundamental, that is, not even every currently existing processor exhibits it, it appears to be reasonable to replace the flawed design of the bricks, and not to re-engineer house construction.

Timing leaks are an absolutely fundamental flaw of computers. It may be the case that this specific instance of speculation-based attacks that leak information via timing channels can be fixed by stopping speculation or by complete rollback of speculated state including caches. "It's the chip's fault" is true for Spectre and Meltdown, yes, and it will take some time for Intel and other manufacturers to design new chips that stop leaking information due to speculation.

However, timing attacks are probably everywhere. They are covert channels that subvert language safety. The question isn't whether information leaks, but what and how fast. How are we going to deal with them comprehensively at the language level? I don't know.

View on HN

Time leaks are not fundamental to computing. They can be fundamental to running concurrent adversarial processes on the same core, though.

Indeed, having no speculative execution, or just having caches separated by trust domains, would make Spectre impossible. This would make the whole promise of 'securely virtualized hardware' largely disappear, though. For guaranteed isolation, you'd have to have a physical core (or set of cores) per trust domain.

With trust domains sliced pretty thinly, e.g. a domain per JS-executing web page, it's hard. AFAICT, reseting the caches when switching processes (that is, trust domains) sort of mitigates this attack at a relative low cost, a performance hit instead of having to replace the CPU.

> Time leaks are not fundamental to computing.

Ok, fine, in some abstract sense. But they are fundamental to realizations of computing that involve shared resources, as the rest of your comment is getting at.

We want to share resources for efficiency, which is going to be the constant pressure that keeps leaks coming.

Sure, there's a pressure for performance figures that makes hardware makers cut various corners, including security. Same with software.

Reality is always about compromises and tradeoffs. Now we see how the balance point is being pushed back closer to the pole of "security". A new equilibrium will eventually appear.

(You can see the very same process in e.g. encryption algorithms: tradeoffs between performance and security, and phasing out of solutions that yielded either to finding of logical flaws or increases in available computing power.)

Currently we build CPUs with a handful of very powerful cores and multiplex processes onto them, to optimize for the use case that a process really does need all the power a core offers. We could build processors with, say, only two powerful cores and a hundred simple cores and give most processes their own core.

That's either a big waste of silicon or a very hard problem.

On a related topic, I know someone who worked on CPU/FPGA hybrid architectures. The idea was to move processes between CPUs and FPGAs. Awesome stuff, realtime processes offloaded to the FPGA get to run completely off the CPU, with ridiculously low latencies. The issue is that you really have to design your code for it.

FPGAs are great because they are field-programmable and can adapt the hardware to a task. In all other aspects (density, power draw, cost) they are not that great.

We could consider something like Tilera chips working together with privileged "general compute" chips, the same way as current machines work with separate GPUs as privilege "vector compute" chips.

I still dream of a day where the compiler largely handles this kind of thing for you. I'm starting to suspect that this day may never come, but it feels like a logical extension of the jit. To have code that self modifies not just itself to run faster, but also the hardware it runs on just seems so cool.

Course, in the context of the larger thread, I can't help but wonder how much more attack surface this would tend to create.

I remember something marketed as a “hypercomputer” by Starburst Systems or something like that (long since defunct, apparently... I'm referring back to the late 1990s and early 2000s) that “compiled software to hardware” by substituting parallel CPUs with FPGAs.

This approach has been tried - as it turns out, we're not very good at writing compilers.

Compilers are sort of at the edge of technology, there is always some value in trying again, no matter how many times it has been tried before.

>> That's either a big waste of silicon or a very hard problem.

How about 16 big core and 4096 small cores that don't have speculative execution?

https://content.riscv.org/wp-content/uploads/2017/12/Tue1136...

Perhaps one day we'll talk about how many cores are allocated to a program rather than the other way around. OTOH even their chip will have shared caches. Spectre is achieved via speculation but communicates via side channel of the cache, so it's not clear this is a whole lot better.

There are always things that are shared even between distinct cores: RAM, buses, disk I/O, network I/O, LAN, the Internet, etc.

Agreed. I pointed out caches. Running only a single process per core does prevent sharing of the branch predictor and the lowest level cache.

What nobody wants to say is that the real problem is running untrusted code. They don't want to say it because Apps and JS are so many peoples bread an butter these days.

"Time leaks are not fundamental to computing. They can be fundamental to running concurrent adversarial processes on the same core, though."

Wrong.

Even if you separate different privilege levels onto different physical computers, and they only exchange information by message passing, then they are likely to leak information merely from the message timing.

The message timing may indicate what has been cached or pre-computed on the privileged machine. So it's a matter of sending messages from the malicious machine such that they manipulate what is cached/pre-computed on the privileged machine based on the contents of privileged information.

Manipulating what is cached/pre-computed is generally not too hard for an attacker; the real trick is doing so based on the contents of privileged information. But it's conceivable that you could extract some information from a database by determining which pages of a btree got cached when you try (and fail) to look at something you're not supposed to. It might be hard to extract the 1000th byte of a big text field that way, but you might be able to find out whether "joe" or "sally" is the person you're looking for.

Sounds a bit difficult in practice, but computing (perhaps more than any other field) tends to quickly eliminate the difference between theory and practice when the motivation is strong enough.

Timing attacks for e.g. password cracking are well-known. This is why most of the security-related computations go to great lengths to make computation-time strictly data-independent. Adding an arbitrary / random delay on failure is also know.

The more you trust the code you run alongside other code, the less precautions like that you have to take. Delimiting security domains right is hard, though.

A "security model" like a memory model should be able to tell you the guarantees available to shield a process from another process upon execution of an instruction. I think a memory model already does this to some extent, just not as everyone thought for speculative execution and side channel timing attacks. A quick Google search will show research on static analyzers for side channel timing attacks, as others have said, on the same core. Having clock strata are also not a new thing, it is resource intensive for make available a more accurate clock, and much cheaper to provide an inaccurate clock.

As far as sharing cache goes, there can be 4096 PCIDs/ASIDs AFAICT, maybe there should be more. Tying speculative execution to PCIDs/ASIDs might not be too bad of an idea.. if it isn't done already. Better CPU security models that reason about cache and speculative execution should be enough to solve this problem.

Deal with them on OS/VM/sandbox/"system" level. Treat time measurement as privileged. Make it really hard to get precise enough timing in untrusted processes.

Disable TSC access, lower clock_gettime precision, make clock_gettime return fuzzy (randomized) time. Randomize CPU frequency to make usage of spinning threads for time measurement hard. Insert randomized delays into untrusted binaries, kinda like DTrace inserts probes (i.e. imagine a JavaScript JIT that inserts no-ops all over the compiled code, and randomly changes these to random sleeps). etc.

But those are all just mitigations, not fixes. Such changes don't guarantee that it's impossible to still leak information - just that it will be harder (but not inherently impossible) to exploit.

If you increase the number of bits in a hash, you do not guarantee against collisions, but you can make them so infrequent that searching for them becomes impractical.

If the randomizations have such an effect that data sniffing is still possible but its speed is 1 bit per week, trying to sniff something interesting becomes impractical. (Or not; depends on your threat model.)

The root of all evil is nondeterminism. A program which executes deterministically cannot receive a timing channel! So "all" we have to do to end this entire class of problems is deny all nondeterministic features, including shared memory threading and timers and two way network communication, to all untrusted code.

A practical vision of computing without these things is challenging, though!

Yes exactly, the problem is non determinism. But I think it should be possible to have deterministic execution while still keeping compatibility with existing software, by replacing the real time with a "fake" time that is deterministic upon the stream of instructions executed, which is on average close to real time. This requires support by the CPU, but it is a relatively simple change, compared to changing speculative execution and caching. Fake time could be kept in sync with real time, by periodically running a privileged process that takes a fixed amount of fake time, but variable amount of real time. (I brought this up in a different comment thread that got marked as duplicate.)

... and then you have to make, for example, scheduling of threads based on "fake time".

rr does just this! I think it uses the "instructions retired" performance counter as its "fake time"; that turns out to be deterministic enough for its purposes. Whether it's deterministic enough in a security context, I don't know for sure.

But this approach, though it can run threaded software, will not let untrusted software (which should really mean all software!) use more than one real core or hardware thread.

What's rr?

found it: https://github.com/mozilla/rr

We can also try to avoid concurrently executing untrusted code on a single processor?

At least on the server side, that's realistic. Big companies like Google already run their own trusted code on their own hardware. For everyone else, it's not impossible to move from VPS hosting to "bare metal cloud" like Scaleway offers.

(Vision for the Future™: instead of their tied-to-Linux "someone else's Raspberry Pi" experience, it would be a bunch of tiny single core processors, each connected to their own tiny DRAM chip. You upload unikernel images or CloudABI binaries (that would be then wrapped into unikernels) into the Cloud™, where the load balancer boots up your images on-demand on as many of these tiny computers as needed. RAM is zeroed before boot.)

The other big environment with tons of untrusted code is of course the web browser. This is significantly more ridiculous, but: why not embed a ton of tiny simple in-order ARM cores without any shared cache into desktop processors, and run JS/WASM on them? (One core used per origin per site) :D

> The other big environment with tons of untrusted code is of course the web browser. This is significantly more ridiculous, but: why not embed a ton of tiny simple in-order ARM cores without any shared cache into desktop processors, and run JS/WASM on them? (One core used per origin per site) :D

I definitely chuckled. :) On a more serious note- I'd bet at least a couple bucks it would be possible (albeit super weird and difficult) to channel information through hardware-acceleration mechanisms. Lots of rendering happens on the GPU these days.

> A program which executes deterministically cannot receive a timing channel!

Only if you define 'deterministically' to mean 'determined only by the program's own legitimately accessible context'. These timing side-channel attacks work precisely because the processor performs in a deterministic and detectable manner, but does so based on information which the attacking program should not be able to access.

No, hardware bugs could work like you describe, but Spectre/Meltdown does not make the result of executing a program depend on the results of speculative execution. It just makes timings depend on such results. If you can't time things, which a deterministic program can't (access to any real timer makes a program nondeterministic), you can set up spectre attacks but you can't receive the results.

It won‘t work. The mechanism that hides the timing information from untrusted code will itself leak timing information.

You just need to refine statistics. The underlying program can‘t be solved in a physical universe.

All interactive programs are nondeterministic. If you remove all other timers, you can still tell the user to click something and see how much work you can do before they click it. That's an exceptionally noisy timer, but a timer nonetheless. The only way to completely remove timers is to revert to batch mode programs.

If you buffer all I/O until the program has finished executing, I don't think this type of timer works. (The message to the user is displayed only after the process you are trying to time completes, and then when the user clicks the button you are executing again)

The program is equivalent to a pure function from (state, events) to (state, actions) and the computing device that evaluates this function can't leak any information to the function without evaluating it incorrectly. It can still leak information in how long it takes to evaluate the function, but there is no direct way to get this information back into the function.

You could ask the user what time it is, or how long something took, but that starts to look suspicious fast!

But shared information is not restricted to shared memory in a computer and it’s threads. As soon you interact with some other system you will have this problem, in a database or any other non trivial (non computer) system?

Timing doesn't have to be in seconds. You can spin up a second thread, that increments a variable and use that a your meter stick for elapsed time. Another though is to have the timer on some external server.

This wrecks a lot of useful things and doesn't really address the problem.

Please no! Some of us need to do performance analysis. Surely the proper fix is to not expose the sensitive information, rather than to try to constrain the attacker's ability to see it easily?

Given that "silicon is cheap" why don't we work around these problems by ensuring that processes within different security domains run on separate cores with their own cache and branch prediction cache? At context-switch, flush all that.

We need to do performance analysis, but sandboxed code usually doesn't. The performance measurement can be in the environment.

This is similar to eval(). The operating system always needs a way to load new code, but you don't need it in the language, or least not in a sandboxed language.

It's also more consistent with the abstract programming model that we typically use. A new version of a function is normally considered compatible with the previous one based on the values it returns, not the details of its timing (within reason).

We often depend on this correctness vs. performance separation to be able to make performance improvements easily. It's very convenient to be able to swap in faster code without worrying that you'll break any callers. It's also convenient not to have to write constant-time functions (to avoid information leaks) most of the time.

Yeah, and for developers of sandboxed code (i.e. JS/WASM in webpages) precise timing should be available in a special Developer Mode.

Or just include the timing in your models, and derive from those models a suite of tests that can be run on the hardware to verify that the timing is as it was taken to be in the proven system?

Uh, there are literal piles of papers on static analysis of timing based confidentiality leaks. We've been studying this for years. It is a hard problem, but by no means new or insurmountable.

And there are literally billions of lines of code and billions of devices that have never been analyzed for timing leaks.

Industry reality is that we are still fighting stack overflow bugs 40 years after safe languages were invented.

Timing attacks are going to be around a long time.

Security did not appear a feature worth spending money on in many cases.

As devices become more powerful and more connected, thus increasing the attack surface and the potential losses form them being cracked, the cost-based analysis will start to prioritize security more.

The tools for writing dramatically less brittle code are there, but there's the cost of switching. It needs to become lower than the potential losses. And with the current state of the industry, the losses and lawsuits are surely coming. Remember what it took to make an average e-commerce website become reasonably secure.

It's going to take a LOT of money. Where does this money come from, or on what gets less money get spent, to use instead to pay the very significant cost of security we haven't been paying?

Do you mean buffer overflow? What language protects against stack overflow?

It's not languages but compilers that protect against stack overflow. Rust has contributed code to LLVM that causes stack overflows to abort the process (via guard pages and stack cookies), which means that theoretically you could enable this for C and C++ as well. See https://github.com/rust-lang/rust/issues/16012

If you forbid self-recursive functions, the required depth of stack can be statically analyzed. I believe such tools exist even for (a feature-limited subset of) C.

Not just self-recursion, but all recursion, including transitive mutual recursion. I think it may be possible to do this via strictly lexically-scoped symbols (so imagine C with no forward declarations). Might still be ways to sneak recursion in there though.

That's what fix point combinators are for. The Y combinator won't work in a language like C++, but the Z will work fine.

For example, here's Z in C++14, used to compute 5 factorial: https://godbolt.org/g/se5B8u

Yeah, I specifically had combinators in mind when considering ways to sneak recursion in. :P And it occurs to me you'd have to forbid function pointers too.

Yup, and you would probably not be able to do Object Oriented Programming either (or have to limit it horrendously), cause an object is similar to a closure, and you therefore can use it for unbounded recursion.

Yes, I meant any recursive calls via a chain of calls to other functions.

I think proving that no recursion exists should be relatively simple, just keep following the call graph and keep a set of the nodes (functions) called; you either reach the terminal node(s), or reach an already-visited node again, which means a recursion exists.

What if functions are first class? Then imagine a function gets passed in as an argument and called. You won't know what function you're calling until runtime.

I thought that self recursion was fine so long as the calls were provably "smaller problems" than the function calling them, something like "bounded recursion".

If you can prove that the recursion is bounded, great!

But sometimes it is hard, and I'm afraid it's equivalent to the halting problem in general case.

Without recursion, though, finding the maximal depth of the call graph is definitely solvable, and likely not even very hard.

Really? I thought the problems were equivalent, of proving that e.g. a loop vs its recursive equivalent were bounded.

Side note: an interesting case is the Stable Marriage Problem. It's easy to write the O(n) algorithm and prove its bound, but it's hard to translate into a bounded recursive algorithm that does the same thing.

(At least as far as I understand -- I searched it on Stack Overflow and that's what it said, and I've tried the problem itself. A Coq fanatic insisted that "if you can prove the run time is bounded it's easy to write the bounded recursive equivalent" but somehow that never translated into an actual insight on how to do it.)

Proving that a loop is bounded is also undecidable.

Only in the general case. In many/most loops that humans would find useful in practice you can prove termination.

In general case, yes, of course.

There are still cases where you can prove that a recursion is bounded (I suppose something like dependent types could help here). There are definitely cases where you can prove that a loop is bounded, because the semantics of the language requires them to: a loop with a counter (not otherwise mutated), iteration over a known finite structure, etc.

Well, both C++ and Haskell allow you to use compiler features that are undecidable in general case, but usually work well in practical cases. Also, if e.g. your recursion / loop analysis is taking too long, you can just stop the analyzer. If the code can't be reasonably proved right, it should be considered wrong (with a chance of a false negative).

You might be thinking of Total Functional Programming [0] "a programming paradigm that restricts the range of programs to those that are provably terminating."

[0] https://en.wikipedia.org/wiki/Total_functional_programming

But the comment here at the top was "how are we going to deal with these things at the language level?" and the implication that there are no understood strategies for tackling this problem.

Getting people to use heavy static analysis is an uphill battle. Getting people to use prepared statements to prevent SQL injection is an uphill battle! But nobody should take then lesson that we are woefully unprepared to deal with timing leaks via program analysis and that we've entered some new unfamiliar territory.

Reply on news.ycombinator.com