Swizzling is not the only thing that would break. You'd break categories in dynamically loaded frameworks or bundles, and yes, you'd still break KVO because it replaces -dealloc and -class on the newly-created classes.

You'd also break dynamically adding methods to classes.