I don't think so at all. I see elixir gaining a groundswell of momentum very much like the early days of rails and node. Of course it's the beginning, but I see a whole different crowd of people interested in elixir - the "gets things done" crowd rather than the "this is cool/elite" crowd I see playing with factor or haskell (sorry, but it's true).
Elixir does not compete with Python, Ruby or JS - it solves a whole different problem, which is the "massively multiplayer" requirement the apps world is obviously moving towards. No-one I know likes Go or Scala. Clojure is a genuine competitor, you are right about that, but it lacks the ruby community spirit showing itself in elixir. Say what you want about ruby but its community moves mountains.
It's early, yes, but I know a lot of good rails devs who are embracing elixir and they're all "smart and gets things done" types. A lot of torque is building behind taking elixir mainstream, and there's excitement in the air. Time will tell, but it's a bet I'm increasingly ready to make.