I used to dislike Rails, since it seemed to be too much abstraction that would require time to learn. I now have the same feeling about Phoenix. I could put up a simple web app quickly with Erlang and Cowboy. With Phoenix, I needed to read documents and the source code to figure out what all the imports are doing.
As someone with a Phoenix app running in production, I can tell you that this framework hits the sweet spot of providing lots of value without requiring the user to learn too much. The abstractions it provides, especially the router and rendering layers, are very welcome. And I'm saying this from the perspective of someone who has built a few smaller vanilla plug applications.
Let's not throw the baby out with the bath water here just because rails went a little too far with the magic.
Because if the latter, it feels like a web framework isn't for you. But you should definitely consider Plug (http://github.com/elixir-lang/plug), which has all the pieces and you just need to put them together (a simple router, a bunch of "middleware", etc).