Although most often I make initial choices that are based, partly, on aesthetics.
Poorly organized user interfaces are the chief culprit here. Bad typography is another non-favorite; especially with apps that present a lot of type that needs to be read during use.
Since I do a lot of design and development of and with graphics applications, I'm pretty particular about the appearance of things in general. I had a drawing application once that did a horrible job of rendering lines, especially curves. Bunches and knots galore. While the output looked okay at high enough resolutions, the onscreen display was a real impediment to its use. And things flickered terribly while moving elements around.
And a certain large graphics application company switched to a dark grey color scheme for a lot of their apps that's really an affront to my older, squinting eyes. The fact that their UI's organization went to hell in a hand basket was just icing on the cake.
The worse use of color that I can think of was a database editor from 25 or so years ago. I've forgotten the name, but if you got to using it too fast, changing screens, making large selections, etc., it could induce an epileptic fit faster than a Pokemon episode played at high speed.
If you don't think the stock UI works well for your application, then you either have special needs, or are wrong. Special needs can be implemented however you like, since they're by definition not handled with standard widgets. Re-implementations of standard widgets, however, immediately stand out to me, and make using an application similar to listening to out-of-tune musicians.
I'm one of those who stayed away from Linux for UI (read: ugliness) reasons. It took me a long time before I became comfortable with it. I finally stumbled on Crunchbang Limix which gave me all the great things I loved about Debian in a light system with OpenBox as the window manager. I mention this because my Linux machine still has an ugly UI but I like it. Why?
I think it's because it embraces its ugliness. It's meant to be light and fast and is all about function over form. So now my Lonux machine is used almost exclusively for getting things done. I mean really freaking working hard. I definitely get things done on my Mac but I use it for pleasure (maybe leisure) whereas the Limux is For Work Only.
I think a UI can put you in a certain mentality that affects how you work. In a world where you've got two UI's that both have equally great UX (because the two don't always exist together) the subjectively prettier UI will probably lend itself better to play time whereas the "ugly" one will get you more focused on the task at hand.
Probably the best example for me is mp3 players. I've been using WinAmp for over a decade now, and love it. It's stylized as a widget which is exactly how I view a music player - a compact but powerful side tool. At the suggestion of friends I've made honest attempts at other players like iTunes, WMP (yes), FooBar2000, but the featureset and style of WinAmp keeps drawing me back in.
Sidenote: Are there any physical mp3 players that offer queuing for songs? Being able to hand pick the next couple songs and then let the randomizer take over when they're done is probably the greatest feature ever to me.
And it goes both ways. I would have never thought I needed an app like Path, but I gave it a try when they released Path 2.0 purely because I loved the design. Now it has become one of my most used apps, super easy to keep my GF and brother/sister updated on the more personal things going on day-to-day.
If you consider that your time is a scarce resource, why would anyone spend time using apps that didn't look/feel great? Unless they were objected to for their job -- but even then, there almost has to be some alternative.
As an example, I just switched from Colloquy to Lime chat as my IRC client. Colloquy matched my needs perfectly, but so does Lime... So I switched based on the UI.
Example: It's possible that altering the aesthetic of an app in some way will lead to an x% increase in the rate of uninstallation, yet none of the uninstallers would point to "aesthetics" as the reason for their uninstallation or even recognize it as one of the contributing factors.
Personally, I recently switched from one app to another that had less features than the first solely because of the UI. UI can actually be a feature in itself. In my case the UI was important enough for me to forego an app with several extra,museful features that I replaced with other apps in a workflow instead of an all-in-one package. I'm wondering if many people say they're all about function over form (especially developers of any kind) because it's looked at as a superficial reason by peers. Kind of like turning down a girl for a date because she's on the chubby side despite her amazing personality. You know you shouldn't be shallow so you gotta make sure you give a good excuse. Something like "I'm not quite ready to date after my last relationship" will do just fine as a face-saving reason.
My only concern with software is how usable it is. If it looks pretty, but I have to do a whole bunch of operations to get something done, it has a poor UI and I'll pick an alternative. If it doesn't have the flexibility to do exactly what I want, I'll look for an alternative (which is why I choose a commander-style file manager instead of the OS default). If the UI is misaligned or has some weird color scheme or ugly pictures, I really don't care so long as I can get things done quickly. I don't care enough to get nit picky about looks when it comes to software.
It's not about shallowness; it's about priorities.
UX however, is another story. Many well-designed apps have spent time on their user experience and it's a component of their design. Many times the "uglier" apps will have design issues that impair usability, rather than just not look as beautiful.
But... I still have Illustrator and Photoshop on my mac.