Hacker News Reader

m3.medium is shared in the sense that it is a virtualized instance on top of a hypervisor, however unlike the t2 instance range it provides set resources not burstable ones. Perhaps not the best practice, but it's not an uncommon way of doing benchmarks. TechEmpower Framework Benchmarks[1] are run on AWS for instance.

1: http://www.techempower.com/benchmarks/#section=data-r9&hw=ec...

View on HN

While not uncommon, the performance variation of most of these virtualized instances is so high that your numbers are probably within the range of acceptable error.

(For reference, I have a team that does nothing but maintain a performance benchmarking harness and test lab, so i know these things pretty cold :P)

I also don't see that the benchmarks were even run multiple times. I tried to follow the ruby-benchmark-suite all the way down the rabbit hole, but don't see anywhere it runs benchmarks multiple times, let alone gives you the variability, etc.

Reply on news.ycombinator.com