Locust – Open-source Load Testing Tool
locust.io
locust.io
It’s main advantage over simpler tools like ab or httperf is being able to simulate most any kind of user interaction. Whereas other tools tend to just spam static predefined HTTP requests at a server (which is great when that’s all you need), Locust is great at simulating complex user behavior, such as users clicking through different pages of your site, or clients that make common sequences of API requests, perhaps to a variety of servers at different urls.
In my case, I was load testing a RESTful DNS management api over http and, as part of the same performance test, simulating DNS traffic to nameservers managed by that API. Locust is horizontally scalable and incredibly flexible since you can write arbitrary Python code to make it do whatever you want.
This sounds really interesting and like good engineering (tm), would you mind sharing which company this was for? Maintaining good blackbox e2e tests and the infrastructure is one thing, finding time (and space in the roadmap) to do that and load test on top of that is a great engineering indicator IMO.
I wonder if it would be possible to translate regular recorded user activity (assuming the API was instrumented) for it and find some way to randomize & generate it (a la quickcheck) and re-feed it into the system. There are some sticky points like pre-existing state (if you take the recording of any given day), but you could probably discover/evaluate that from API requirements (if you saw a `GET /resources/1` which succeeded, then obviously a `POST /resources` must have occurred before hand).
You’re really hearing about a unique case. The engineering org there had basically no consistency across teams/projects. Every product team was independent and responsible for their own services. So different teams deployed their services to different infrastructure with different programming languages, different frameworks, different databases, different monitoring and logging solutions, and so on.
I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test.
But, very few other teams were getting to the point of doing any non-trivial load testing. I tried evangelizing Locust a little bit, but there wasn’t enough demand for it without a turnkey solution - something a bit hard to make without much consistency across teams. Not to mention the culture and processes weren’t there to consolidate on a single solution anyhow.
So, grass is always greener. That company’s engineering org has since been decimated by layoffs and outsourcing unfortunately.
> I was in the QA org and was embedded as an SDET on a particularly well-functioning product team and had a lot of independence to solve problems as I saw fit with support from great forward-thinking devs. I got their e2e/integration tests to a point where it was super easy for devs to add at least some new tests themselves, which freed up some of my time to slowly build out a performance test.
is hard to find sustainably happening in the wild.
Since our platform (Appian[0]) has a pretty heavy/complex client, we built a thin abstraction library [1] to empower both our quality engineers as well as some of our more advanced customers to write simple automated performance tests.
[0] https://www.appian.com [1] https://pypi.org/project/appian-locust
[1] https://github.com/tag1consulting/goose
[2] https://www.tag1consulting.com/blog/real-life-goose-load-tes...
As for Locust, I've been testing and reviewing a whole slew of load testing tools, and blogging about them, and Locust is definitely a favourite of mine. It used to perform badly a couple of years ago, but with the new FastHTTPLocust class its performance is quite OK and it's the only tool that allows you to write test code in regular Python (there is another tool - the Grinder - that also lets you write in Python, but it is Java-Jython which means no pypi libraries). If you're into Python you should definitely try out Locust.
It might be worth linking here.
Does someone have a comparison between the two?