Ask HN: Which load testing tool do you use? Any recommendations? | Hacker News Reader

Ask HN: Which load testing tool do you use? Any recommendations?

51 pointssymbolepro21 comments

I like to use https://gatling.io/ for my load testing. It's super easy and you get pretty charts you can hand off to management.

Gatling is great, and the best thing to use if you want performance and fine grained analysis. It's easy to use, great html output with graphs. You can also output to a bunch of other formats, and easily break out into scala. Don't let the scala put you off though, its mostly a really simple and clean dsl. The docs are good too!

+1 for Gatling. It can also output junit reports if that’s something you need too

Another +1 for Gatling. Super easy to use, and maybe more importantly, super easy to read the code later and grok exactly what it does. Scales well, too.

These are the ones I've used recently. Primarily a Windows/.Net shop, so there's a lean towards tools that are scriptable, and don't require Java. ;)

Bombardier - fast cross-platform, written in Go. https://github.com/codesenberg/bombardier

WCAT - quite a manual process to craft requests, but great to orchestrate several clients to hammer your server if needed. https://www.iis.net/downloads/community/2007/05/wcat-63-x64

West Wind WebSurge - Excellent tool, Very GUI focused (but has a command-line option) very easy to copy/create requests from fiddler. Approx $900USD site license. https://websurge.west-wind.com/

Netling - Very nice open source project with gui/console. https://github.com/hallatore/Netling

For quick bursts the free plan of https://loader.io/ is pleasant to work with. For more complex setups have a look at https://locust.io/

Locust has a very nice UX for developers, but it is very slow and produces inaccurate response time measurements

I like to use: - https://en.wikipedia.org/wiki/Tsung - https://k6.io.

Tsung definitely! Harder to use at first, but amazingly powerful

Tsung is very good - fast and w lots of features. Its main drawback, IMO, is the XML-based config/user scenarios.

Thanks for recommending k6! Disclaimer: I'm involved in the k6 project, so quite biased :) I think k6 is the best tool available if you're a developer who wants to automate load testing. But I also actually wrote a review article comparing 11 different open source load testing tools - http://blog.loadimpact.com/open-source-load-testing-tool-rev...

I like the Apache Benchmarking tool: https://httpd.apache.org/docs/2.4/programs/ab.html

I like ab as well. Simple and tells me if my app can handle a load. Exactly what I want to know. I do wish it was easier to see what the failed requests looked like though.

I'll second this.

If you need quick and dirty this is THE tool to use. I can think of 1/2 a dozen times where this has literally saved the day.

I use Vegeta https://github.com/tsenart/vegeta whenever I need to quickly test out limits of API endpoints.

Besides that, artillery also looks promising https://github.com/shoreditch-ops/artillery

Here is my take on the most common open source tools:

- Gatling: https://gatling.io/ - Great tool. Fast, full-featured, flexible, well documented. Main drawback is that you script it in Scala... UX for automation purposes could be better also.

- Tsung: http://tsung.erlang-projects.org/ - Also great. Very fast, scalable, many features. Main drawback is the XML-based DSL (Domain Specific Language) that is somewhat of a pain to use.

- Vegeta: https://github.com/tsenart/vegeta - Good for simple testing of API end points with a fixed RPS rate. No scripting capability.

- Apachebench: https://httpd.apache.org/docs/2.4/programs/ab.html - Single-threaded but very, very fast so will outperform many tools that can use multiple CPU cores. Best tool around if all you want to do is hit one single, static URL and get results printed on screen. Lacks scripting.

- Jmeter: http://jmeter.apache.org/ - Very fast, tons of features, but painful UX (especially for an automation workflow). Biggest community with the most plugins/extensions/whatnot.

- Siege: https://www.joedog.org/siege-home/ - Moderately fast but doesn't scale. Buggy and will crash regularly. Large measurement error. Inconsistent UI. Only reason to use is that it is an alternative to Apachebench if you want to hit a list of URLs, rather than just a single URL.

- Artillery: https://artillery.io/ - Great UX for automation. However, it lacks scripting capabilities, is slow and single-threaded and introduces a large measurement error.

- Locust: https://locust.io/ - Great scripting capabilities (in pure Python), with a very nice API. However, it is the slowest tool I have seen and introduces the most measurement error of any tool also.

- Wrk: https://github.com/wg/wrk - Fastest tool in the universe. About 25x faster than Locust. 3x faster than Jmeter. Scriptable in Lua. Drawbacks are limited output options/reporting and a scripting API that is callback-based, so painful to use for scripting user scenario flows.

- The Grinder: http://grinder.sourceforge.net/ - Fast, scriptable in Jython (Python dialect) with a nice API. Main drawback is that it is an almost dead open source project - very few updates in recent years.

- k6: https://k6.io/ (bias warning: I am involved in the k6 project) - The B3ST tool!! ;) Anyway... it is fast, scriptable in Javascript (ES6), has a very nice UX and scripting API and works for both functional and performance testing (and is excellent for automation). Good docs too (https://docs.k6.io).

General advice:

- If you're a Java-centric shop, take a look at Jmeter or Gatling, or perhaps Siege (all are Java apps) and you'll feel at home. You should probably start with Gatling, because it is more modern than Jmeter, with better UX, and Siege is kind of on its way out.

- If you want to automate your load tests, want to test an API, and prefer to write test cases in code rather than some limited DSL, then I think k6 is the tool you should look at. Only reason not to, is if you don't want to use Javascript.

- If you want to run complex, manual load tests of a web site (e.g. record user interactions and then simulate the same behaviour in a load test) you should look at Gatling or Jmeter primarily. Perhaps Tsung.

As well as using a modern parallel back-end language (Go) and a widely used scripting language (ES6), k6 supports HTTP/2, cipher suite versioning and will include inbuilt orchestration. All under the GPL. K6 can be easily hooked up to InfluxDB with a readymade Grafana template and there is work being done on a Prometheus collector.

I’d add wrk2 to the list. It’s wrk but for latency analysis.

For large scale tests we run https://goad.io on lambdas . We used this a lot and can confirm that most APIs can be well stressed with it.

For quick tests we use loadimpact.com

If you are interested in reusing your functional test libraries to run load tests, drop me a line at hello@browserup.com. We have a solution and are looking to do pilots.

For web applications try TruLoad, this is a codeless load testing tool that executes user-like workflows for automated testing https://cloudqa.io/website-load-test/

Reply on news.ycombinator.com