ibraheemdev

Not the runtime per se, but cooperative scheduling has the advantage that tasks do not yield at adverse code points, e.g., right before giving up a lock, or performing an I/O request. Of course the lack of preemption has it's own downsides, but with thread-per-request you tend to run into tail latency issues much earlier than context switching overhead.

> OS threads are expensive: an operating system thread typically reserves a megabyte of stack space

Why is reserving a megabyte of stack space "expensive"?

> and takes roughly a millisecond to create

I'm not sure where this number is from, it seems off by a few orders of magnitude. On Linux, thread creation is closer to 10 microseconds.

It does make a difference of course if you're running fetch_max from multiple threads, adding a load fast-path introduces a race condition.

pip, PDM, and uv already support PEP751 [0] and were involved in the design process.

[0]: https://discuss.python.org/t/community-adoption-of-pylock-to...

I did not claim that x86 provides sequential consistency in general, I made that claim only for RMW operations. Sequentially consistent stores are typically lowered to an XCHG instruction on x86 without an explicit barrier.

From the Intel SDM:

> Synchronization mechanisms in multiple-processor systems may depend upon a strong memory-ordering model. Here, a program can use a locking instruction such as the XCHG instruction or the LOCK prefix to ensure that a read-modify-write operation on memory is carried out atomically. Locking operations typically operate like I/O operations in that they wait for all previous instructions to complete and for all buffered writes to drain to memory (see Section 8.1.2, “Bus Locking”).

Yes, what I meant was that the same instruction is generated by the compiler, regardless if the RMW operation is performed with relaxed or sequentially consistent ordering, because that instruction is strong enough in terms of hardware semantics to enforce C++'s definition of sequential consistency.

There is a pretty clear mapping in terms of C++ atomic operations to hardware instructions, and while the C++ memory model is not defined in terms of instruction reordering, that mapping is still useful to talk about performance. Sequential consistency is also a pretty broadly accepted concept outside of the C++ memory model, I think you're being a little too nitpicky on terminology.

I'm referring to the performance implications of the hardware instruction, not the programming language semantics. Incrementing or decrementing the reference count is going to require an RMW instruction, which is expensive on x86 regardless of the ordering.

> There is no way the shared_ptr<T> is using the expensive sequentially consistent atomic operations.

All RMW operations have sequentially consistent semantics on x86.

It's not exactly a store buffer flush, but any subsequent loads in the pipeline will stall until the store has completed.

> ParkingLot just uses pthread mutex and cond.

That's interesting, I'm more familiar with the Rust parking-lot implementation, which uses futex on Linux [0].

> Sure that uses futex under the hood, but the point is, you use futexes on Linux because that’s just what Linux gives you

It's a little more than that though, using a pthread_mutex or even thread.park() on the slow path is less efficient than using a futex directly. A futex lets you manage the atomic condition yourself, while generic parking utilities encode that state internally. A mutex implementation generally already has a built-in atomic condition with simpler state transitions for each thread in the queue, and so can avoid the additional overhead by making the futex call directly.

[0]: https://github.com/Amanieu/parking_lot/blob/739d370a809878e4...

> And futexes aren’t the only way to get there. Alternatives:

> - thin locks (what JVMs use)

> - ParkingLot (a futex-like primitive that works entirely in userland and doesn’t require that the OS have futexes)

Worth nothing that somewhere under the hood, any modern lock is going to be using a futex (if supported). futex is the most efficient way to park on Linux, so you even want to be using it on the slow path. Your language's thread.park() primitive is almost certainly using a futex.

> The message has some weird mentions in (alloc565), but the actual useful information is there: a pointer is dangling.

The allocation ID is actually very useful for debugging. You can actually use the flags `-Zmiri-track-alloc-id=alloc565 -Zmiri-track-alloc-accesses` to track the allocation, deallocation, and any reads/writes to/from this location.

> Every single Future you look at will look like this,

That's not true. A Future is supposed to schedule itself to be woken up again when it's ready. This Future schedules it to be woken immediately. Most runtimes, like Tokio, will put a Future that acts like this at the end of the run queue, so in practice it's not as egregious. However, it's unquestionably a spin lock, equivalent to back off with thread::yield.

It's quite common for concurrent algorithms to only implement a subset of operations. For example forgoing, removal or iteration. It's also common to put limitations on the data structure, such as limiting keys and values to 64-bits. Papaya being feature-complete means that it does not have any of these limitations when compared to std::collections::HashMap.

Looks very interesting, but seems to serve a pretty different use case:

> This is an ordered data structure, and supports very high throughput iteration over lexicographically sorted ranges of values. If you are looking for simple point operation performance, you may find a better option among one of the many concurrent hashmap implementations that are floating around. Pay for what you actually use :)

Java atomics are actually sequentially consistent. C# relaxes this to acquire/release. Though the general concept of happens-before is still immensely useful for learning atomics as sequential consistency is a superset of acquire/release.

Formatting the if onto a single line gets you most of the way there.

    if err != nil { return err }

Yeah you're right regarding signal handlers (though I'm not sure the windows equivalent), and I was planning on adding a note about the alternative. The main thing I wanted to point out was that async/await as a model is built such that expressing complex control flow with arbitrary inputs becomes very simple. Nowadays thread-per-request will work well performance wise for most, but async/await became the default in various ecosystems because it's a more expressive model, especially useful in server-level code, which more or less forces it to be used by everyone. The advantage becomes apparent when you need to compose more complex logic within a system, which async makes seamless (though in Rust specifically things are harder because of the interaction between async and the borrow checker).

Except pgx recommends using their native interface, not database/sql, for performance and extra features [0], so it's not that simple in practice.

[0]: https://github.com/jackc/pgx#choosing-between-the-pgx-and-da...

> But we are deleting this bs. I only learned about it now! Will be gone by tomorrow.

https://twitter.com/elonmusk/status/1641908130274525187?t=5t...

https://asahilinux.org/ is looking very promising

The post introducing Ristretto, a similar Go cache is also a great technical read: https://dgraph.io/blog/post/introducing-ristretto-high-perf-...

Works for me.

Is allocating a large array on startup to avoid gc cycles better?

The new `runtime.SetMemoryLimit` is actually a pretty big deal and solves a lot of long standing issues regarding the garbage collector not being very configurable, I believe including the one described by the viral "Go memory ballast" article [0].

[0]: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...

This is a consequence of runtimes relying on global variables that their core future types are dependent on. Creating abstractions to solve this problem is one of the main goals of the the async working group [0].

[0]: https://github.com/rust-lang/wg-async

Blocking I/O+threads can actually scale very well now, and with block_on you get the worst of both worlds, but yeah, I agree that most people are probably fine with it.

> I'd instinctively worry about overhead there

Yeah, one of the nice things about blocking I/O is that you can perform it with a single syscall. With block_on(async_io), you're now dealing with registration with a reactor, polling epoll, and extra syscalls for each I/O operation. Not to mention the overhead of running the state-machine as opposed to line by line.

It uses the linux futex api directly, not a user space parking lot as used by the parking-lot crate.

> - Noisy neighbor problems from other threads messing with your TLB and L1 cache

Switching between threads within the same process doesn't require a TLB or L1 cache flush. Not sure if you were implying this, just wanted to point that out.

> - High cost of context switches

Userspace schedulers (like rust's tokio) do make context switching cheaper, however, most of the context switching in the case of a web server is due to blocking I/O and the most expensive part of the switch, entering the kernel, is already accounted for by the I/O request. Kernel context switching is unlikely to be your bottleneck.

> Unpredictable scheduling/priority inversion in the scheduler

This can definitely be an issue at scale, but a general purpose async scheduler like most use is unlikely to be any better.

> More precisely, OS threads, because of their scarcity, introduce an artificial bound on throughput that's lower than what the hardware can support, and usermode threads remove that bound.

Why are OS threads scarce? The OS allocates thread stacks lazily. Given a kernel stack of ~8kb (two pages) and a user stack of ~4kb, one could spawn 100k threads with just over 1GB. A userspace runtime will allow you to bring that number down, but if you're at the scale of concurrency it is unlikely to matter much.