Show HN: K6 – developer-centric load testing using Go and JavaScript
k6.io
k6.io
One request: please allow the output of either a) a CSV log of every request and it's timing or b) a configurable histogram of request timing, not just some percentiles and averages. That info is pretty much required for doing any more advanced analysis beyond "the service is fast enough". https://github.com/tsenart/vegeta does this really well (but doesn't have the scripting abilities of this tool).
Is JSON useful to you, or do you prefer CSV?
Once I get a chance to test this out, I'll definitely report any issues through GH.
If you should happen to try out k6 in the meantime, most of the JSON contents should be recognizable/understandable just by looking at the output, if you know what metrics k6 collects (which is documented on docs.k6.io). You can try running with e.g. 2 VUs for 2 iterations, to get a tiny amount of output data but something that looks like a real load test: k6 run --vus 2 --iterations 2 --output json=output.json script.js
Had anyone used both to point out the main differences from k6?
The single biggest difference is that k6 lets you write test cases in real code (JavaScript), while Vegeta will only let you supply a static list of URLs to hit. Vegeta has some things k6 does not have, to be fair (like constant RPS-rate load), but k6 was created to address what we saw as missing in Vegeta and most of the other existing tools. We think developers want to write code to test things, so we created a tool that lets you use JavaScript + a scripting API designed for load testing and automation.
But you can actually aggregate results already, if you use InfluxDB as results storage: you can have multiple k6 instances push results data to the same InfluxDB instance. Results are just time series samples, so there are basically no bad effects if you do this - you get all the results in a single db, with correct time stamps. The only difficulty lies in orchestrating the multiple k6 instances so they're in sync and you get the traffic levels you want (i.e. they ramp up/down at the same time). But for a simple ramp configuration, or a config where you run at a stable traffic level, it shouldn't be a problem.
So - HTTP request timeouts can not be changed at the moment, but this is something I think would be very simple to implement. If you open an issue about it I'll see what we can do to get it implemented quickly.
As of now, k6 can do standard http, https (and it also supports HTTP/2)
We'd really appreciate any comments/feedback you may have on exactly how it should be implemented.
But I am very, very irritated by Golang while I am working with it on a daily basis.
It's entire design is informed by assuming that no developer should understand the codebase, but should instead read it over and over. Symmetrically, no insight can be offered in the code, only a rote transcription of an inductive design.
The net result is a language hostile to both those writing and those reading the code. No shortcuts are offered or taken. All error handling must be done manually. All data structures must be reimplemented over and over with different test suites for each one.