Not saying you can't employ a UI designer who does this full-time, but the final software product and the entire experience around it is still the developer's full responsibility. With this in mind, perhaps we'd like the developers to be more involved with UI design meetings?
All of this gets me back to the front-end/back-end meme that seems to have emerged over the last decade. Same sort of problem. All you do with this "not my problem" game is make your own life harder the next iteration.
So to be clear - I am not saying you have to do the whole product yourself, but I am saying that you need to care enough about it or it's going to turn out like shit. Writing software that people use is still decidedly an art form. Imagine paying to go to an art gallery filled with pieces created by artists who absolutely hate to make art.