Hacker News Reader

Comment by staticassertion | Hacker News Reader

It feels like the fundamental issue here is shared mutable state, and much of the solution is to just not have that.

I don't believe this requires a change to async-await though. It just means you want to wrap up your async-await in its own synchronization primitive, or just avoid the need for it.

This is easy to do with a channel system, or an actor system, where the queue is the synchronizing primitive. An actor's message handler can internally be asynchronous, so long as it processes messages one at a time (but it can yield during processing to another actor).

We can get actors in rust pretty trivially imo. It's just a task with state and a message interface.

View on HN

Interesting, could you elaborate on what you mean by "shared mutable state"? In the parse_line example (first one in the post), what state is shared and with whom is it shared?

Not GP, but perhaps he's thinking of the TcpStream, which is a sort of "shared state" between executions of parse_line?

Canceling an async task feels a lot like aborting a partially executed DB transaction without rolling back any changes.

&TcpStream is a shared reference with interior mutability, and its state can change when you await and let other futures take a turn at accessing the same TcpStream. I prefer how CondVar encodes this property into its types: it gives you a MutexGuard (providing exclusive access), but requires you explicitly release it when you let other threads have a turn at accessing the shared data: https://doc.rust-lang.org/std/sync/struct.Condvar.html#metho...

Is there a reason the function in the article couldn't be written to accept a &mut TcpStream, which doesn't let others interact with the same TcpStream while the function is running or awaiting?

I imagine it's &TcpStream because the TcpStream is used by both the reading and writing halves. There are helpers to split this [1] but maybe carlleche avoided them in this example for simplicity. [edit: oh, I guess the real TcpStream is actually like this. [2] The syscalls don't require &mut of course because that state's in the kernel, and I guess they didn't choose to add a &mut to enforce reasoning about the state. But in real code you'd wrap it in a buffer, and that would require &mut anyway.]

I don't think making it &mut TcpStream would solve the problem. It's not that something else is reading between a parse_line() call's read_u32() and read_exact() but that the parse_line() future is going away entirely between those and the read_exact() never happens (or never completes, as it's probably not atomic either). When that happens, it leaves the TcpStream in an unexpected state (halfway through a line) and the next parse_line() is misaligned.

But I do think having one task which owns reading from the stream to completion would solve the problem. If it dies, the whole TcpStream does too. It could talk with other tasks via channels with atomic messages. (Although I've been trying to minimize my use of channels right now because neither the tokio crates nor the futures crates offer an unbuffered/rendezvous channel, and I don't like sticking extra buffers in the middle of everything.)

[1] https://docs.rs/tokio/1.7.0/tokio/io/fn.split.html

[2] https://docs.rs/tokio/1.7.0/tokio/net/struct.TcpStream.html

Yep, you nailed it.

> I imagine it's &TcpStream because the TcpStream is used by both the reading and writing halves.

Yes, and in fact in older versions of tokio reading and writing required &mut, even for things like UdpSocket, which was a bit of a pain when you wanted to write to a socket concurrently as you couldn't just use an Arc.

> It's not that something else is reading between a parse_line() call's read_u32() and read_exact() but that the parse_line() future is going away entirely between those and the read_exact() never happens (or never completes, as it's probably not atomic either). When that happens, it leaves the TcpStream in an unexpected state (halfway through a line) and the next parse_line() is misaligned.

Oops. I stand corrected.

The mutable state isn't explicit here (it's internal mutability), but it's the TcpStream.

In an actor system the TcpStream would be local to an actor. You'd say "parse_line" by sending it a message. That message would be processed to-completion before another message could be processed - even if there is yielding internally.

You could also wrap TcpStream with some sort of wrapper that doesn't expose its internal mutability - that would let the type system ensure there isn't shared mutability across tasks I think...

Reply on news.ycombinator.com