Gtk was a pain to develop with, sure, but only due to the awkwardness of GObject's attempts to build an OO system on top of C.
From a user perspective, I never saw the problems you're describing. Even having built Xfce out-of-tree all the time, I never recall having to do a full rebuild due to a Gtk update.
I've since moved on; it's a shame to hear what the OP is saying about 3.x, but 2.x most certainly didn't have these problems that you describe.