Hyperbridge: Fast multi-producer, multi-consumer unbounded channel in Rust | Hacker News Reader

Hyperbridge: Fast multi-producer, multi-consumer unbounded channel in Rust

github.com

53 pointstosh15 comments

The repository seems abandoned; or maybe complete?

At work we use flume, which is another capable multi-producer, multi-consumer async-capable channel [1]. It's great for shuffling data between threads, as well as between async tasks, and between threads and async tasks. Basically any time you want to pieces of code to exchange data or signals without pesky shared state.

1: https://github.com/zesterer/flume

I use flume as well, and it works very well. In particular, I often use it in contexts where one end of the channel is in an async task and the other end is in a sync thread.

Highly recommended.

It is common for small rust crates to only receive unfrequent updates because they are finished.

Not a common thing in web development.

it’s also common for small rust crates to get abandoned

Why use this or Flume over the crossbeam multi-producer multi-consumer channel [1]? I thought crossbeam was well-regarded and pretty much the unofficial standard library for this sort of thing.

https://github.com/crossbeam-rs/crossbeam/tree/master/crossb...

Crossbeam isn't async[0]. It can multiplex with itself (via the `select!` macro), but not with anything else.

[0]: https://github.com/crossbeam-rs/crossbeam/issues/896

Gotcha, thanks.

Why would it need to be async for an already low latency mpmc queue?

I imagine for working with non-low latency producers or consumers, such as network requests?

The queue itself may be fast, but that doesn't mean that the other end of your pipe is.

Because queues have blocking operations, like getting the next item when it's available, or putting in the next item as soon as there's space in a bounded queue. But if your code is async you can't block the whole thread, you want your queue to block in an async way (well, you could wrap all blocking calls in tokio::task::spawn_blocking, but that's not ergonomic)

I'm curious what the motivation was for linking this now. This repo hasn't been updated in two years. Meanwhile, there are at least five other Rust MPMC queues in use that have been recently updated.

Is there something unique about its algorithm? I'm afraid the repo is low on documentation.

Interesting. Thanks for sharing.

I am in early stages of writing a low latency barrier which provides low latency lock free communication between thread pairs ( 1 thread bidirectionally with 1 other thread ) but wiring up to other barriers is proving difficult.

The number of sent messages doesn't match the number of received messages I think I need a mutex unfortunately

So it's interesting that there is options in this space in Rust.

Would be great if it had benchmarks

The benchmarks are there, just run `cargo bench`

Reply on news.ycombinator.com