Hacker News Reader

Comment by zbobet2012 | Hacker News Reader

zbobet2012

Golang allocates a _goroutine_ per socket. Those goroutines are actually scheduled using epoll (https://golang.org/src/runtime/netpoll_epoll.go). Goroutines are _very_ light (8KB stack size) and don't incur nearly the same cost as a thread in a "context" switch. While for some (honestly very small) subset of problems they are still too large most people will never need to worry about them.

I have production systems which run several 100k goroutines at once with very little overhead.

View on HN

Wait, the only way to do concurrent network programming in Rust is to allocate an OS thread to every connection? That doesn't sound right.

Rust Core team member here. Out of the box, the Rust standard library only provides very basic networking support [1]. If you want to process requests in parallel, then yes you need to spawn an OS thread to process the request. We went with this model because we wanted to have the standard library be portable across all our supported platforms, and there isn't really a great standard for doing portable IO.

Instead, we're layering platform-specific code in external libraries. At the raw C-level bindings, we have libc [2] and winapi [3]. For higher level APIs, we've got Mio [4] which abstracts the system APIs, and now Tokio [5], which uses futures to simplify async operations.

We're just working our way up the stack. All of these are being developed by members of the various Rust teams, so they're as "official" as everything else we're doing.

[1]: https://doc.rust-lang.org/std/net/ [2]: https://crates.io/crates/libc [3]: https://crates.io/crates/winapi [4]: https://github.com/carllerche/mio [5]: https://tokio.rs/

And it's not right. Rust programs can use epoll directly or through abstractions like mio and tokio.

Thread-per-connection is the simplest way to do concurrent networking in Rust using only libstd, but it's not the way that most Rust programmers would actually use (at least, not for a production server handling a large number of connections).

IIRC that's the only option in the standard library, but event loop and coroutine implementations are available from the crate ecosystem.

No, there's tokio, but it is not in the standard library.

> While for some (honestly very small) subset of problems they are still too large most people will never need to worry about them.

This is also true about OS threads!

    Goroutines are _very_ light (8KB stack size)

2kb, actually.

That's curious, as I've always considered 2 pages (1 unallocated) the absolute lower bound for a thread stack size (1 page of actual, allocated memory for stack, 1 page unallocated as a guard page to detect overflow.)

If a goroutine calls into C, and the C code overflows or otherwise writes ‼Fun‼ onto the stack¹ … can it clobber an adjacent stack of another goroutine or other memory?

C code runs on a larger special stack (that's part of the reason calls to C are slow). I don't know whether or not guard pages are added.

There are no added guard pages.

Reply on news.ycombinator.com