Edit: Real Hotswapping will requires profound changes to the memory model. It would introduce too much complexity. I rather restart my application than to suffer from a fragile and leaking platform.
Unfortunately, this will require externalization of state data, but if you can't be down for 5 minutes, that's what you need to do.
You probably only need one large server running the instances, rather than a cluster. Java apps don't seem to do well -- longer GC pauses -- when they have over 2 GB of memory, anyway, so run multiple instances of "small" apps on a server with say 12 cores. A single server with a large pool of RAM shared between a large number of cores can do pretty amazing things with a collection of "small" programs on it (DB, web service apps, message queue server, web servers)
Of course, I'm assuming said server isn't Windows...
The webapp I'm working on currently has a startup time of 50s on my macbook. If I restart the webapp 100 times a day, thats a total waiting time of 30+ minutes. Our team is 6 developers, so we loose 3h per day.
Real hotswapping may be extremely complex but IMO it would be absolutely worth the effort.
Edit: Demo of it in action here, notch is rapidly developing a game in Eclipse by changing the code while it's running: https://www.youtube.com/watch?v=rhN35bGvM8c
edit: Wait, that is liverebel, not jrebel
He's modifying the Swing object tree from the main thread! Don't do that, Swing isn't thread safe ;-)
https://www.youtube.com/watch?v=rhN35bGvM8c#t=1454
http://docs.oracle.com/javase/7/docs/api/javax/swing/package...