Hacker News Reader

Comment by tlb | Hacker News Reader

Interesting project. I almost wish I had a concurrency bug to test it on.

> Guest software running in the Antithesis platform still experiences concurrency similar to a multi-core / multi-machine system, thanks to the process scheduling imposed by the guest OS

This might not exercise the full set of race conditions. When two threads are running simultaneously on separate cores (or hyper-threaded on the same core) they can interleave instructions at a much finer granularity than any OS time slicing would cause, even within instructions.

For example, could it find a race condition where two threads are executing INC [addr] on the same memory address, where context switching between instructions doesn't trigger it?

View on HN

You are correct. There's a set of concurrency bugs that require actual SMP setups to trigger (like stuff with atomic operations, memory ordering, etc.). If you're trying to build a very low-level lock-free data structure where this is your primary threat model, Antithesis is not the right tool for you... for now... until we build a CPU simulator...

> For example, could it find a race condition where two threads are executing INC [addr] on the same memory address, where context switching between instructions doesn't trigger it?

I'm not actually familiar with the details of hardware MMUs, but would they not enforce sequential access of the address? Or do MMUs allow parallel reads and writes?

x86 processors have a LOCK instruction prefix, which makes some instructions atomic including increment. Increment is nontrivial to make atomic, because there are two memory accesses: read X and write back X+1. It's a bit slow, because it has to inform every other core in the system, "Hey, I just read this memory address and I'm about to write something back to it, so don't don't use it until I'm done." C++ has functions to do this like std::atomic_fetch_add, but they're clunky to use so people often forget.

Ahh, it's 2 separate memory accesses just encapsulated in one instruction, that explains it. Yeah, unless this hypervisor allows context switching at the level of microcode ops, that seems to be undetectable currently.

The way that's currently implemented is actually just one memory operation these days. L2 (or really wherever coherency is mostly managed) has a tiny ALU so in addition to read or write, atomicop is an operation you can send to L2. It'll gain Modified or Exclusive access to the cache line(s) that op is addressed to and just do the operation right there. That way the normal cache protocol is all you need for atomicity, and really the line is only contended for a single cycle from the L2 controller's (and the rest of the coherency peers) perspective.

That's why they retconned the lock prefix to not be an actual assertion of the #LOCK signal any more.

That's also why TileLink and AMBA include atomic ops like addition and bitwise ops in their coherency protocols rather than just 'claim region'.

That's also why you see newer archs like RISC-V and Arm64 that have both lr/sc style ops, in addition to direct atomic memory ops like amoadd.w, it better matches the primitives of the underlying memory system.

That's not how it works on x86 as far as I know. The atomic ops are simply performed by the ALU, against the L1 cache when the instruction is about to retire. Atomicity is guaranteed by not allowing the line to be stolen by another core between the operation, and memory order is ensured by draining the store buffer before executing the op and other speculation (eg loads can speculatively pass even atomic operations).

If history is any guide, everybody has concurrency bugs.

Reply on news.ycombinator.com