That's just patently untrue: either GTk or Qt would work perfectly. I don't see why the desktop environment would matter--the program doesn't have to (and probably shouldn't) care whether I'm on Gnome or KDE or Unity.

Moreover, all the popular environments support both GTk and Qt fairly well, and both are cross-platform, so you would actually be supporting more users than with a Mac OS-only program.

Really, fragmentation like this is not nearly the issue people make it out to be. Basically, everybody can use everybody else's programs on Linux, with the exception of very odd or customized systems. And the sort of people who have very odd and customized systems can fix problems themselves.

EDIT: Also, you could use wxWidgets. That would give you a native look on all the platforms you support.

I would also like to clarify that I understand an OS X user wanting to write an OS X-only program. The real issue is dissuading others from supporting Linux because of non-existent fragmentation issues. They simply do not affect people using GTk or Qt, in practice.