I can write UI code faster than any developer I've ever met who uses IB, and that's doubly true for class hierarchies I have subclassed for multiple apps, and triply true when it comes time to change those subclassed hierarchies. You note that IB is for simple UI layouts, but I would argue that for layouts that simple, the code is simple too.
Maybe it's a matter of how you think, but I can "see" the changes when I adjust the frames, colors, padding, resizing masks, and that's basically all IB will do for you.
IB will also happily confuse your localization effort, and introduce bugs when you ship apps with unreferenced XIBs.