Anyone know whats going on there?
1. https://www.techempower.com/benchmarks/#section=data-r17&hw=...
This would be a more apt comparison:
https://www.techempower.com/benchmarks/#section=data-r17&hw=...
- better configuration (tweaked to this benchmark / hardware). Bigger communities will try harder to tweak their benchmark config. The Phoenix one is probably just the default one or slightly tweaked. Is the SQL connection pool size ideal? Are all default Phoenix-added Plugs ("middlewares") needed here? Is the BEAM virtual machine config / flags ideal? Would using OTP releases be beneficial?
- Elixir ecosystem is constantly improving the performance. The VM, language and packages versions should be updated. Elixir 1.6.5, Phoenix 1.3.2, and Ecto 2.1 are not recent.
- Abstractions used (and amount of them, so amount of work done at run-time). Compare the Fortunes "handler" implementation in vertx-postgres [1] to the Phoenix one [2]. Raw SQL vs generated query, almost bare DB connection pool vs Repository abstraction, and it's just the "handler". But you wouldn't want to maintain a lot of low level code.
- Typical functional language overhead. Copying (copying the conn when modifying it, Enum.sort on a list etc) is more expensive than modifying in-place. Again, functional code will be easier to maintain.
But overall I think in the particular benchmark you linked, Phoenix isn't that bad. Latency is pretty low and consistent (0.5 ms min and 9.9 ms max (!)) thanks to the BEAM pre-emptive scheduler. And BEAM will scale well vertically.
1. https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast... 2. https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
At the same time, Elixir (and Erlang) are not meant for raw speed. It is best used for real time communication, lots of users & handling errors. At least that is what I have read.
Yup, many things.
It's worth reading this very long thread about exactly this, from 2016. Look for Sasa Juric and Chris McCord's comments in particular. [1]
tl;dr benchmarks are hard, not all benchmarks are implemented well, Elixir folks haven't heard back from Tech Empower re: details of errors rates etc. It's an unfair analysis, and not just for Elixir.