Yes that isn't too many, but right now any device can just hop on a network and start communicating (after DHCP). Not only that, but they can start communicating with any public IP address.

If people decided to take this away by adding some security directly into the IP layer (i.e. such that communicating without it is impossible, such as mandatory IPsec), I don't think the tradeoff would be worth it. Now you would have to manage all the normal stuff that comes with keys (e.g. expiration and renewal), and you may find your device gets wedged if you don't do the delicate key expiry dance correctly (i.e. you can't even connect to the site to get updated keys).

It's very easy to say "those DARPA morons not designing security was a big mistake!", but I am not convinced that the tradeoffs of solving it at the internet level (i.e. L4 and down) are worth the bootstrapping / flexibility hits.