Comment by scottlamb | Hacker News Reader

Comment by scottlamb | Hacker News Reader

Regarding the quote:

> The Original Sin of Rust async programming is making it multi-threaded by default. If premature optimization is the root of all evil, this is the mother of all premature optimizations, and it curses all your code with the unholy Send + 'static, or worse yet Send + Sync + 'static, which just kills all the joy of actually writing Rust.

Agree about the melodramatic tone. I also don't think removing the Send + Sync really makes that big a difference. It's the 'static that bothers me the most, and that's not there because of work stealing. I want scoped concurrency. Something like <https://github.com/tokio-rs/tokio/issues/2596>.

Another thing I really hate about Rust async right now is the poor instrumentation. I'm having a production problem at work right now in which some tasks just get stuck. I wish I could do the equivalent of `gdb; thread apply all bt`. Looking forward to <https://github.com/tokio-rs/tokio/issues/5638> landing at least. It exists right now but is experimental and in my experience sometimes panics. I'm actually writing a PR today to at least use the experimental version on SIGTERM to see what's going on, on the theory that if it crashes oh well, we're shutting down anyway.

Neither of these complaints would be addressed by taking away work stealing. In fact, I could keep doing down my list, and taking away work stealing wouldn't really help with much of anything.

> I'm having a production problem at work right now in which some tasks just get stuck. I wish I could do the equivalent of `gdb; thread apply all bt`

For all the hate that Java gets, this is something that has Just Worked(tm) for like 25 years, and it's enormously helpful for troubleshooting. You don't even need a debugger; you can just send the JVM a SIGQUIT, and it'll dump a stack trace of every thread to stderr (including which locks each thread is holding and/or waiting on) and keep running.

I miss this feature in every other language I've worked with. You can even use it for ad-hoc profiling in production: just take a bunch of snapshots, and use grep/sed/sort/uniq to look for hotspots.

Go does this well, too. iirc the std library has a package for serving <http://blah/debug/pprof/goroutines> so you don't even need to ssh in to the server in question.

There are libraries for doing the same for any language that just uses kernel threads. It's when you throw in async that you really need to reinvent this kind of observability, and Rust isn't there yet unfortunately.

i’m slowly coming around to the idea that in most cases (1) big runtimes are a good thing, and that (2) compile-once-run-many was a bad idea. i think our programming languages should create and run software in a highly introspective and interruptible environment.

I don't know what you have against compile-once-run-many, but as a Rust user I agree that most software doesn't need C or Rust. I think there could probably be a Rust-like and a Java/Go-like, two general-purpose languages that cover 99% of software.

compile-once-run-many makes decisions, especially optimization decisions, too early ie way before the software got a chance to see real input. vm-based programs that are able to do just-in-time optimization can learn on the job and make adjustments as necessary. making optimization decisions late, taking input into consideration, makes sense for long-running applications.

Ah, that's fair. I doubt most applications need that level of optimization, and AOT still seems to be more reliable overall for performance. A language with GC and/or VM does have great properties, though, so for most apps it's not a loss to go with the managed approach. Poorly designed software seems to be the bigger issue in general.

> I'm having a production problem at work right now in which some tasks just get stuck.

To mitigate this kind of problem, at my company we use a library [1] that allows regularly logging which tasks are running and what file/line numbers each task is currently at. It requires manually sprinkling our code with `r.set_location(file!(), line!());` before every await point, but it has helped us so many times to explain why our systems seem to be stuck.

[1] https://github.com/antialize/tokio-tasks/blob/main/src/run_t... has set_location(), and task.rs has list_tasks()

Yeah, I can see how that'd be helpful. In my case, I suspect this is happening inside a third-party library I'd rather not have to vendor/patch extensively. So that method could confirm my suspicion but probably wouldn't easily allow me to drill down as far as I'd like.

That said, I think the newest version of the third-party library might have some middleware hooks and/or tracing spans. With the right middleware impl / tracing subscriber, maybe I could accomplish something similar.

This code also should be following the general distributed systems practice of setting deadlines/timeouts at the top level of each incoming request, propagating through to all dependent requests, and also setting timeouts on background ops. It's not. Fixing that is also on my list and might be enlightening...

Reply on news.ycombinator.com