1) Java isn't amenable to pre-installed hosting like PHP is. You usually deal with machine images that you have to prepare yourself. Jetty 7 + Nginx is the going coin, it's quick and easy
2) Java is outright the best performing language of the ones currently used for web development with any degree of frequency (PHP, Python and Ruby being the other ones I'd mention). There's 10+ years of optimization and garbage collection research in those VM's, and language is statically typed, which increases verbosity yes, but also performance
3) If you extend the view beyond start-ups, Java isn't just getting used - the world runs on it. If it's a web app of any meaningful size, there's a 90%+ it's written or getting written in Java. The only other language that has this wide an acceptance level is PHP, which is a different animal entirely. Not to disrespect RoR or Django, but their overall share of the web development market is negligible
Keeping in mind the bias that comes from 12+ years of dealing in Java, here's a quick pro-con for what it's worth:
pro => - Java is fast. Very fast - Tool support is unrivaled. It's hard to appreciate what an IDE can do to speed up your work until you've gotten your first couple dozen IntelliJ keystrokes in the muscle memory. RoR is not faster to write than Java (a frequent claim) - Library support is unrivaled as well. Whatever it is, there's already a library for it, usually quite well-documented, and most questions that come up are googlable on the spot (something I sorely miss when I have to work Rails on occasion) - with the entry of Play, speed and ease of web development is approaching dynamic languages: write, hit reload, be done - knowledge and people to hire are available widely - the JVM is also home to Scala and Clojure. Starting in Java and moving to Scala is a pattern these days
con => - it's wordy and, compared to, say, Ruby or Scala, downright ugly. Not a big deal as far as start-up success is concerned, mostly a hacker bragging right. Nothing wrong with bragging as long as it doesn't get in the way of getting stuff done - generics and collection syntax is unwieldy. IDE support takes care of some of that, but not all - there's arcane stuff one needs to get used to - a learning curve of sorts. Again, mitigated by the widely available documentation and Google support
hope this helps.
- z