When you say "all these misc CDNs" you're talking about just one, CDNjs, right?
I suppose the CDN could host other assets if it made sense. But I can't think of any other assets with the type of commonality that JavaScript libs have.
I would find this somewhat surprising given the increased use of hosted DNS providers for both authoritative and recursive. Can you point to any data that corroborates this (not saying it isn't true, just looking for hard data)?
http://199.27.135.101/ajax/libs/prototype/1.7.0.0/prototype....
CloudFlare doesn't seem to support direct access based on IP, but there's no reason it couldn't, right? Do browsers support this?
While it may be possible to create a IP-based CDN, my guess is the tradeoffs in terms of not being able to thoroughly defend yourself against attacks would not be worth the few milliseconds saved on the initial DNS lookup.