I've been an Elixir developer for years now, loved it immediately and still loving it after all this time.
About refactoring: I've been working as a Java developer for the past couple of years and I must admit thaat Intellij does a pretty impressive job at helping you refactoring code and their code sense is top notch.
But, there's a but.
I've never found myself having to make nuclear refactorings in Elixir, not because I don't refactor my code, but because modules are self contained and not tied with anything else, most of the times it is faster to abstract something over what you already have than to refactor all the things.
For example, if you decide that you need a struct instead of the old map, you advd a new function that pattern matches on the struct and pass it down.
If the struct has a different structure, you process it and pass it down.
The "treat data as something that is constantly transformed" mentality can take you a long way.
When I can code in java in a way similar to what I do in Elixir, things get easier in Java as well.
I miss symbol renaming in Elixir too, but it's never been a big stopper in the 6 years I've used it as a primary language.
edit: biggest strength of Elixir is Jose Valim himself. If you watch his videos you can see him finding a bug while doing something, actually explaining why that's a bug in Elixir (or some library) open an issue and ask for PRs
All while showing you complex software patterns that I though I could never face before using Elixir.