What about the JVM is such a downer?
I think there's some work going on to address the former. The latter issue isn't really that bad unless you need to use trampoline, which I don't care for because it means the functions have to be written to know they'll be used in this way.
Regardless, Clojure is one of my favorite languages.
Also, the slower startup time of the JVM affects clojure apps and makes simple command line scripts in clojure less attractive
~$ time clj test.clj
Hello world!
real 0m1.337s
user 0m1.309s
sys 0m0.172s
~$ time java Test
Hello world!
real 0m0.142s
user 0m0.116s
sys 0m0.030s
You may want to look at the -XX:+TieredCompilation -Xbootclasspath flags. They reduced the clojure startup significantly for me.