*Being specific on where exactly it's deficient would be useful. "It's clunky" really isn't actionable criticism.
My own comments are sparse largely as I've found other elements (mostly the readers) so abysmal that I make little use of Calibre.
I don't think this is a matter of calibre de-prioritizing fixing the UI. I'm pretty sure it's a conscious choice. They know their UI backward and forward and it works for them, therefore anyone who complains about the UI Just Doesn't Get It. That unicycle is BEAUTIFUL, dammit.
I'd like to see a few more jet-powered unicycles rides. They sound fun. Functionality, especially in niche software, is sometimes more important than some kind of idealised smooth workflow.
The nearest physical analogue I can conjure up is to contrast a 'sound system' as used in dances in church halls and community centres round here (wardrobe sized speaker cabinets with 18" paper cone speakers, horn tweeters, all patched into a rack based preamplifier that only the constructor and a few close associates know how to navigate) and a domestic hifi by (say) Denon. Both have their place.
I buy this argument for professional GIS software, database management software, etc. But come one, Calibre basically has 3 purposes: keep a library of ebooks, convert them between formats, transfer to devices. It shouldn't be so complicated. Every time I go open it I have to relearn how it works.
Then you must be doing something wrong. I use Calibre very infrequently, and yet I've never had any trouble remembering how to convert a book and load it onto my Kindle.
There are GUI conventions which at this point have been with us for three decades, and no matter what operating system you're using the vast majority of programs you use follow them. I would never argue that it's impossible for a program to be improved by deliberately flouting those conventions -- but I'd argue that you need a defensible rationale for doing so. A sound system for a large space has different tradeoffs and design considerations than a home theatre system does, but it's actually following a different set of well-considered conventions, right? I don't think calibre's UX design is well-considered at all; I think it was put together by someone who doesn't know much about UX design, isn't willing to learn, and whose reaction to even gentle criticism is to dig in his heels.
Calibre has done a few conversions, but given the typical use-case I've found is for trying to get PDFs into anything else vaguely usable, it's almost always a complete wash.
Going to Markdown or LaTeX and from that to the format(s) I want is almost always the preferable route.
I use calibre for converting web pages I want to read offline into epub, but then I use the file manager to transfer the epub onto my device - it seems to be the easiest way.
Also, on every single conversion I have to tell it again that I don't want a huge margin. Why would I want to make my e-reader's screen ever smaller?
Maybe it's best to build a whole new app out of the low-level libraries and tools that make calibre. A simple app with a simple UI.
I found calibre less clunky and very clean once I hid icons, menus and submenus I don't use.
Maybe there should be some kind of toggle mode (easy, god, expert, etc.).
I think the lack of emphasis on creating polished user interfaces is a real ideological stumbling point for the open source movement.
If the project is hostile to merging patches (which Calibre's developer had demonstrated), that is relevant when discussing the possibility of a user providing contributions for improving the UI.