Hacker News Reader

Comment by meta_AU | Hacker News Reader

5ms average load latency. But the max looked like it was more than 5 seconds.

The JMeter load tests were conducted to test the throughput of the CDN and backend setup (browser caching was disabled). The load generating servers where hosted in Frankfurt (Germany) where the CDN (Fastly) also has an edge location. This lead to average response time of 5ms. We experienced some hickups within JMeter causing single requests to stall up to 5 seconds. For the 10 million requests we got an 99.9th percentile of below 10ms. We couldn't quite figure out why JMeter had these hiccups, maybe those were GC pauses, maybe something else, but it was consistent across different setups and servers.

One wonders about the irony of carefully avoiding bad stuff (like GC) when serving pages, and then using a test rig that has potential GC delays to test performance.

You have a point there. I found jMeter, however, really easy to use. I could simply let it monitor my browser (via a proxy) while i clicked through the website and the checkout process to record the requests of an average user. Then I configured the checkout process to only be executed in 20% of the cases to simulate the conversion rate. Even executing the test distributed over 20 servers wasn't that hard.

Which tools would you use to generate this amount of traffic?

In a previous life we started with ApacheBench, and then wrote our own async perl benchmarker because we wanted to generate steady hits-per-second instead of ApacheBench's N concurrent requests at a time. By now there's probably a tool out there that does what our custom tool did.

wrk2: https://github.com/giltene/wrk2

That looks pretty good -- not only does it have a "requests per second" generator, which I wanted, but it also presents 50/90/99/99.9 percentile results, which are a must. (Website latencies are not a normal distribution so it's inappropriate to compute standard deviation.)

locust.io is pretty sweet, performs well and allows you to do custom scripting in python

Reply on news.ycombinator.com