The way that instances connect to each other is O(n^2), which is not to say it's exactly n^2 but the curve is of that shape. This isn't strictly a problem with ActivityPub as with how it's used, especially in Mastodon. It's why you keep hearing about super-long queues chewing up RAM, and causing long delays - hours or even days - before content appears. It's why Mastodon instances tend to be clusters of larger machines at a scale that you would think can be served by a single smaller machine.

The usual solution for this set of problems is connection and/or request aggregation via a proxy layer. Relays (in fediverse-speak) do exist, but seem very lightly used and do little or no caching because ActivityPub relies heavily on POST (which IMO was probably a bad decision). Any caching that's done has to be ActivityPub-specific. I wrote about this a while ago and almost certainly got some stuff wrong, but it might be interesting nonetheless. At least it drew some interesting comments at the time.

https://gist.github.com/jdarcy/60107fe4e653819138396257df302...

A certain degree of overhead just has to be there, a distributed system needs to exchange information between nodes after all. And both the protocol and many implementations leave room for optimization at least. I think tackling that is one of the long-term challenges for the ecosystem, but I wouldn't say it is categorically "not scalable".

EDIT: As an example, right now in Mastodon federation traffic is handled by sidekiq queues running Ruby. Someone makes a post -> tasks to send that information to all their followers instances get queued, which then send individual HTTP requests to all those instances. This to me feels like something that could be moved into some smaller and more-efficiently-written component that batches requests, makes sure to reuse connections, ... at least for paths that see a lot of traffic.