> All tests were run on AWS from an m3.medium EC2 instance
so you're comparing running times measured on a shared host? This is generally not considered a best practice if you want meaningful numbers.
so you're comparing running times measured on a shared host? This is generally not considered a best practice if you want meaningful numbers.
1: http://www.techempower.com/benchmarks/#section=data-r9&hw=ec...
(For reference, I have a team that does nothing but maintain a performance benchmarking harness and test lab, so i know these things pretty cold :P)
I also don't see that the benchmarks were even run multiple times. I tried to follow the ruby-benchmark-suite all the way down the rabbit hole, but don't see anywhere it runs benchmarks multiple times, let alone gives you the variability, etc.