Clearly there's the Java approach, but are there any other good options?
The server doesn't have to be cross platform, but the client does.
Security in c++ is hard, memory leaks are hard, crashing on high load is hard to debug. What I learnt from this was... servers are hard and takes a long time to debug. Going with python/perl/C# would have saved us time on memory leaks and some of the crashing (no buffer overruns etc). The downside would be that the server may be slower. BUT the cost to develop the C++ server was about the cost of 40 servers so we could have written a slower app that scaled and bought more servers and rackspace. I estimate choosing c++ added about 6-8 months to the task. Even as I left, while relatively stable the code is brittle and there is only one engineer left who knws the code and he is too chicken to change anything. Launch pressures meant there was 'not time' for unit tests (plenty of time to dig a large hole though)
Think hard about your requirements. Most severs spend most of their time waiting. Either the filesystem, database or on high load reading from memory/swap. So the CPU is often under utilised. Interpreted languages while slower can give a similar performance profile to c/c++ but with more cpu utilisation. When you have a LOT of sockets open you can spend a lot of time waiting for the kernel to poke you (even with kqueue or io completion ports).
Either way, don't write the underlying server code yourself there are frameworks in ALL languages that will get you 90% of the way. Remember scalability is a 'nice' problem to have as it means you have customers. Concentrate more on scaling across more machines as they are a LOT cheaper then engineering times.
good luck
If your app is db-bound then use your favorite language. If it requires a lot of CPU time, you'll probably want a language that compiles to native code. Lots of choices there so look for the most productive one given your performance constraints.
Is your application highly distributed, or in need of fault tolerance outside of the database? Erlang might be an ideal fit. Can you build the entire thing out of Java libraries? Then Java's probably your best bet, but...
If you're building something and expecting tons of of concurrent users then you'll probably have to write most of the code from scratch. So keep that in mind when picking a language.
Client Side
If your client is very GUI-based then most cross-platform toolkits may not feel native enough. You might want to consider writing writing separate clients for Windows, Mac OS X, and Linux, If a purely native feel isn't necessary, you could use your favorite scripting language and GTK+, or wxWindows, or QT. Or Java and Swing or SWT.
If you have a unique, or web-like, GUI then maybe Mozilla's XULRunner would be a good choice.
Take advantage of epoll under Linux and you can probably use any language you want. Brad Fitzpatrick wrote a Jabber server (DJabberd) in Perl that can handle 300k connections in 1GB of memory. It depends what you're doing, but it's highly likely you don't need to choose a painful language for the sake of efficiency.
Depending on your application, you may find that other languages are more suitable -- on the server-side, for example, you might want a language which does more to protect you from foot-shooting -- but I don't think you'll be able to find anything more portable than C.
I'm looking at something that will (ideally) have tens of thousands of concurrent users each using a good amount of bandwidth and server resources. I'm thinking separate login and application servers, as well as load balancing for the back end.
Is anything in particular well suited for this? The two languages I was initially planning on choosing between were C++ and Java, although obviously I wouldn't be bound to one language for the entire project. Or is there some kind of framework designed with something like that in mind?
High speed, highly-engineered portable code for lots of people? I'd go with the C suggestion. Fewer people and wanting to work at a higher level of abstraction? Pick one of the 4GLs or 5GLs. Different languages and platforms are good at different things. For instance, there's been a lot of bad-mouthing of relational databases lately, but for some things you just can't beat 'em. You wouldn't want to use a screwdriver to pound in nails. Games don't scale up like business apps do, and collaboration plays differently than e-commerce.
Or, I could just say you should build the server in Common Lisp and the client in Python with wxwidgets because that would probably be my first thought.