Hacker News Reader

Comment by dawkins | Hacker News Reader

"Iron averages 2500-3000+ requests per second for hello world"

For a hello world in my laptop I can get typically 30k requests with Go, and with mono/httpListener ~10k.

I expected it to be much faster.

The benchmarks are a bit lower than we wanted. I think this is mostly due to an immature ecosystem for doing basic http - working off of c bindings to a proven, fast framework would probably speed this up by a few times.

It does sound odd that you compare your framework to NodeJS framework. Comparing to Java frameworks (a fast language which can do both parallelism and concurrency) would be much more relevant.

To be honest, it's mostly because that's what we had experience with.

The comparison with Go is really interesting because I don't think its http server does anything particularly fancy (like calling across FFI to non-Go code) and I don't see any obvious bottlenecks in rust-http that would cause an order of magnitude less throughput. Would be interesting to see some profiling.

As far as I can tell (and have been told), rust-http starts a new task (thread, here, really) for every request.

Edit: but see https://news.ycombinator.com/item?id=7978299

Maybe redesign it to start some thread early, and put them in a pool.. then dispatch to them.. i think i've saw a thread pool already backed in rust std or in servo.. dont recall right.. i think servo even have a fancy work stealing thread pool..

Also the design of Nginx is sophisticated, because it uses the event io + a thread dispatch design.. (at least it was like this last time i've hacked it) and worth being copyied

By the way, something is up with the server serving the static doc. The link to the docs is wrong, eg https://github.com/iron/router should link to http://docs.ironframework.io/router/, instead of http://docs.ironframework.io/core/router. To add insult to injury, http://docs.ironframework.io/core/router returns 200 with an empty body instead of 404.

That said, the framework itself looks interesting, though the 'iron/alloy/etc' terminology is a little bit of an annoyance.

Thanks for the heads up. We reworked our folder structure after writing the READMEs, and it looks like we forgot to update. I'll get working on that.

The 200 is being served as the default of `rust-http`. Static will never serve a 404 - it will only defer to the next middleware, and the docs server is overly simple, so it just goes to default.

This should be fixed. Thanks again.

In our benchmarks our raw TCP performance has generally been best-of-breed. I suspect the issue is rust-http, which is not an official package.

I suspect the issue is Iron, though rust-http certainly isn't helping much. Here are some graphs, generated from data made by rust-http's benchmarking script (https://github.com/chris-morgan/rust-http/tree/master/compar...), on my box.

A spreadsheet: https://docs.google.com/spreadsheets/d/1-5_4wPPKNBYZ6yuw-rRl... The raw data: https://gist.github.com/cmr/3648bcc6365c1dba83f1

The charts themselves: http://imgur.com/gw1mp40,Hn3929i,h2OQiTh

Software versions:

rustc 0.11.0-pre (44ec28cfac9fa3f738e0e77ccca1d804125fd1dd 2014-07-01 21:26:33 +0000)

go version go1.3 linux/amd64

node v0.10.29

Apache 2.4.9

wrk 3.1.0

Arch Linux, x86_64, CPU: https://gist.github.com/cmr/7cc6df806caff2253667

It's worth mentioning that I ran this under -c 100 or 200, not 1-8, which may change things significantly.

I just re-ran benchmarks using wrk instead of ab and I'm getting 16k req/s instead of 2k. Very strange.

Did you try changing the runtime to libgreen? It uses a thread pool and dispatches jobs to them instead of spawning a new thread for each job.

I am not surprised. Most of the machinery necessary for such things is still developing in Rust.

When I was testing my now-discontinued framework (widmann) using a very early version of rust-http, I felt lucky when it didn't drop or break on half of the requests. Reliability comes before performance in this regard and Rust is a bit from done. I expect that to develop quickly once the general shape of code is known and people start profiling and optimizing.

At that time, Rust’s TCP support was flaky and easy to crash; it was the issue normally, not rust-http.

That wasn't a complaint, just an illustration on how things are quickly improving. Yes, it was a TCP issue, but presented to me through rust-http.

Since the tests are relative they should include a script that tests the different frameworks/languages. It shouldn't be too complicated.

Same here, without even looking into what is slow, Clojure with Compojure does 7500 req/s. If Rust would like to be the next close to bare metal language we need some serious performance improvements.

Reply on news.ycombinator.com