Benchmarking Phoenix vs. Rails vs. Sinatra vs. Express vs. Martini | Hacker News Reader

Benchmarking Phoenix vs. Rails vs. Sinatra vs. Express vs. Martini

github.com

79 pointsrebelidealist30 comments

Elixir is Erlang with some Ruby-like syntactical sugar that vanishes when you look very close at it but makes the transition very easy. It's not like moving to Erlang or Haskell or OCaml coming from procedural languages.

I have experience with Sinatra and Rails and I did a small project with Elixir and Phoenix in September. My bet before looking at the benchmarks was Phoenix >> Sinatra > Ruby, and that's the way it turned out to be. I was curious about how Martini and Node would compare.

That said, I enjoyed developing with Phoenix and it's really similar to Rails in its structure. The differences are about having views+templates (decorator pattern) instead of Rails' plain views (which would be Phoenix's templates) and having first class handlers for websockets. The developers of Phoenix answered quickly to all my questions and to the issues I opened on GitHub (it's still 0.x software in heavy development).

Anyway I still can't use it to replace Rails in the projects I do for my customers. There many reasons for that, some about Phoenix and some about me.

1) There aren't enough developers and it won't be wise for a customer to become too dependent from me (or any other developer)

2) There isn't an ecosystem comparable to the Rails one yet. Example: it can take forever to develop a replacement for all the basic functionality a Rails developer gets for free from a gem like devise (everything that revolves around authentication).

3) There might be many useful tools from the Erlang world but it takes time to learn them and become decently productive with Erlang and Elixir. How to make decent money while going back so many years productivity-wise? Obviously this applies to any transition to any ecosystem.

4) Performances usually don't matter at all: most servers have to sustain only low loads and Rails is good enough for that. In 20 years I probably had to work with only two web applications were speed really mattered. One was done with Java, the other one was PHP+Java and it was constrained more by the db than by the frontend. However I would consider Elixir right now for any heavily concurrent server, let's say a game server with thousands of players.

> Latency (ms): 4.54 Consistency (σ ms): 1.75

That is not to be underestimated. Many frameworks will provide better throughput but consistent low latency is harder to do and Erlang's BEAM VM does it. It is really a marvel of engineering and has been battle tested for decades.

Yep. This is a very important metric to keep in mind. It gives you a sense of what you might expect under heavy load.

For the ones surprised by Go's performance, remember that Martini in known to be slow as it uses heavily reflexion.

Gin, a Go web framework similar to Martini, would probably performs 10 times better. Since the only purpose of using Martini is because it has a similar style than Phoenix, I would suggest changing the framework to Gin or another Go framework since Martini is not the only framework available.

I would love to see openresty in there too.

I feel like these benchmarks always put rails in a tough spot. Let's choose bare bones frameworks and compare it to a fully loaded framework. Oh wow, the bare bones ones are faster?! I'd be concerned for them if they weren't.

Elixir's goal is _not_ to be a micro framework. Perhaps it does less than Rails now but its philosophy seems to be more similar to Rails than Sinatra, see talk from the creator http://www.confreaks.com/videos/4132-elixirconf2014-rise-of-....

And we're already seeing people mixing up Elixir with Phoenix...just like Ruby and Rails. Good sign. :)

reminds me of http://www.techempower.com/benchmarks/

Martini is slow, try again with Negroni which is the new lib by the author of Martini.

Aren't they quite different creatures, though? It's clear that language is not the only determining factor, since Sinatra is so much faster than Rails, and the Node example is so much faster in a cluster configuration. If the experience of programming for Sinatra, Phoenix, and Martini are very similar, it's fair to compare them (though maybe benchmarks may not be super useful, given that the database layer is likely to be the bottleneck in most large site deployments rather than simple routing and other stuff being compared here). If Negroni is very different (I don't know that it is, but a brief perusal of the site for Martini and the github for Negroni indicates very different experiences) then it's possibly no more useful a comparison than Rails to Phoenix.

Plus isn't Negroni made to specifically be more about middleware than a fully-featured framework?

If you say use Negroni, you might as well say use net/http (which is fair enough, since it's a low-level framework in itself).

So, profiled the Martini bench and discovered it was doing a LOT of file operations. I looked in the code and found that it does that in dev mode. The benchmark sets the mode to "production". D'oh! Ok, so I profiled it again and guess what? Martini is STILL doing a lot of file stuff. It's in net/http.Dir.Open and I believe it has to do with how the middleware is setup. You can see a TON of time spent in runtime.cgocall which I believe is it waiting for a syscall to come back. Don't have time to look into it further today but I'd imagine clearing that up could possibly double the performance.

I wonder how koa would fare.

Probably a bit slower than the express example as ES6 generators cannot be optimised the same way than callbacks can.

Probably the same if it uses EJS. That's really what is being benchmarked here, which is confusing, as EJS is one of the slower templating languages.

Martini is not a good reflection on Go's performance (no pun intended). If you want a Go benchmark of a minimalistic framework, use Gorilla mux or even Beego

Also check out Gin which is very lightweight and performant with a great API.

https://github.com/gin-gonic/gin

Is elixir just a different syntax for erlang, or are the semantic differences? Their website doesn't seem to make it very clear.

Elixir makes a lot of the syntax of Erlang (derived from Prolog) simpler and more familiar to people working from, say, a Ruby background.

It adds better support for UTF8, better metaprogramming tools for declaring custom constructs, a standard library for things like collections and streams, proper default setup for some use of OTP patterns, and various other things. It adds a few sane default conventions for things like documentation, testing, and private/public visibility in modules.

It in turn transpiles to Erlang, so has all the same performance characteristics.

Is it therefore possible to use Erlang libraries in an Elixir application, or to "drop to Erlang" in places where it could be advantageous?

Erlang and Elixir can call each other with no overhead.

Yep!

It's baller as hell. :)

Being able to use all of erlang's inbuilt modules and functions are an advantage. But there's way more to Elixir that just syntax.

1.) Elixir ships with a build tool called Mix. Falls back to use rebar for erlang projects. Mix is awesome and IMHO way more stable that rebar (Had problems with erlang libs shipping diff versions of rebar binaries in the git repo itself).

2.) There's a really good package manager - Hex.pm (developed and maintained by Elixir's co-author Eric)

3.) OTP compliant config.

4.) Pipes ~! This for me has been a very useful feature to decongest my code. I'm not sure what project had pipes first (should we credit F# or Unix shell itself?). Having that feature in Elixir is a boon for me.

Finally about Phoenix:

* More mature that most people think. It has a few features that even rails doesn't have (or even cannot handle in real world scenarios). There's support for "transports", which allows people to use polling or websockets and such if your apps need it.

* Phoenix not having an ORM doesn't make a diff. Elixir's core-team develops and maintains Ecto, which is easy to drop it and start using. Ecto still has a long way to go, but for most apps, it is sufficient.

* Authentication? Very simple to write a plug for that and share it across all Elixir frameworks. AFAIK someone wrote plug for OAuth.

>I'm not sure what project had pipes first (should we credit F# or Unix shell itself?)

You'd have to go with unix out of those options, F#'s pipe operator is just flipped function application, and is primarily to make up for the poor type inference F# has. Every functional language has or can easily have a function application operator.

They are pretty much the same. This is what Erlang's creator wrote about Elixir 1.5 years ago http://joearms.github.io/2013/05/31/a-week-with-elixir.html

tl;dr there are good innovations and bad things, not a surprise :-)

I don't have any experience with it, but your question reminded of this article[0] that came up on HN a while ago arguing it's more than just syntax differences. Basically that it's more a Clojure/Scala than a Coffeescript.

[0] http://devintorr.es/blog/2013/06/11/elixir-its-not-about-syn...

Elixir is to Erlang what Scala is to Java. They leverage the Erlang VM in the same way that Scala leverages the JVM.

Actually it is not a good comparison. scalac compiles directly to byte code that runs on the JVM. Scala has pretty different semantics from Java. Whereas Elixir is compiled to the Erlang AST - a pretty high-level data structure - and then the Erlang compiler is invoked on this structure to compile byte code for BEAM. So whereas Scala can do things Java can't do (such as closures on Java 6), there really is nothing Elixir can do that Erlang can't.

The fact that Elixir has the same semantics is really a good thing, because it makes it much easier to learn both languages at the same time and that is still pretty much essential in my opinion, if you want to become productive in Elixir.

Reply on news.ycombinator.com