There's great deal of optimization going into the JVM as a server-side VM, but unfortunately, there's less work going into the JVM as a client VM. Lot of the advanced options that exist on the server version of HotSpot aren't available on the client version. Azul's recently open sourced tweaks are also aimed at the server side market.
I fully understand the reason for this: Microsoft dominates the "thick client" market, the money for Java is on the server side (with Javascript client code running in the browser). The few Java desktop apps that I use are development related: Yourkit profiler, JVisualVM, IntelliJ, Processing-based Arduino IDE and Eclipse CDT (for the excellent AVR plugin). They're actually excellent, but (with the exception of light-weight Processing/Arduino IDEs) they're aimed at developers who are familiar with the Java platform and don't mind occasional rough edges: I had to tune IntelliJ's GC settings (enabling CompressedOops on 64-bit Linux, using a 32-bit JVM and CMS collector on OS X, adjusting sizes of various generations etc...) to get it to stop locking up on me. Given I work on memory intensive Java and Scala applications, that's not a problem for me. Imagine, however, a word processor application that required this ("hey mom, it really is called `CompressedOops', one word, capital C and capital O!").
On the other hand Qt provides many utilities for C++ beyond the UI and has bindings for Ruby. It's cross platform between OS X, Linux and Windows. Qt and C++ would be the route I'd take were I to build a cross platform application I am skilled enough with valgrind and honestly don't find lack of a GC to be a major problem for me (especially for desktop programming, which doesn't frequently involve complex parallel algorithms that are difficult to implement with manual memory management). Qt's provides a great deal of what one expects from a high-level platform like the JVM: concurrency libraries beyond primitives, simplifications for building event/callback driven applications, additional collections and even an IoC container.
If I didn't mind tying myself to Microsoft's standards, I'd also take a serious look at Mono: while Mono's GC is primitive compared to HotSpot when it comes to server side applications, I've yet to hear of people having to tune it in order to run Mono-based desktop apps such as (excellent) F-Spot; garbage collection is a long-ago solved problem, I find it hard to believe that client VMs couldn't come with sane defaults out of the box. In this way it reminds me of Linux in mid-90s/early 00s (before distributions like Ubuntu and extensive driver support): lot of work going into scaling to thousands of processors and new architectures, while getting support for desktop sound and video cards required recompiling a kernel.
Finally, the author really strikes a point with lack of a POSIX API built in to the JVM. What's even more striking is that this can't even be explained by "focus on enterprise server market" theme. Server-side, Linux is almost a mono-culture with occasional use of Solaris or OS X (the latter frequently as development environment). The few non-POSIX platforms (Windows, mainframes) aren't really a large part of the Java market and have POSIX compatibility layers available for them (bonus point: why not support both, like Perl does -- having builtins for most system calls, but also an excellent Win32 module).
POSIX-JNA is available and (from what I hear) is an excellent library, but its use of GPL makes it incompatible with Apache-licensed projects (ASL 2.0 being de-facto standard in the Java world). A minimal interface to POSIX, coupled with a "systemcall()" method (allowing easy use of Linux-specific extensions) should be the standard part of the JDK: Python and Perl offer this without sacrificing portability and safety, why can't Java?