Having said all of that, once I got everything working, I felt that using Scala was worth the pain. Programmatic layouts using scaloid gave me a vastly better experience than coding them in XML, and implicits worked really well for passing around the Context object. The tooling just wasn't, by any means, reliable, so I wouldn't recommend it for any apps your career depends on. :)
I haven't tried it, but I've heard that Kotlin is a nice compromise if you're looking for a language nicer than Java with more reliable tooling than Scala.
It's unclear to me whether your pains were specific to Scala on Android or with Scala tooling in general. I'm already familiar with server-side Scala development using IDEA and I'm relatively comfortable with it, so that's not a problem. Is IDEA awful for Android development?
I would say that if you're already familiar with Scala and aren't afraid of getting into the weeds a little bit, you should be fine in the short-term. My personal fear is simply that, especially with Scala 2.12 dropping support for Java 6, Scala development on Android doesn't have a clear future, so as Google changes their build stack, you may find it increasingly harder over time to keep all the tools working.
There is no sense of hurry as 2.11 won't go away and will have a long life.