Gossip networks indeed don't scale very well when not all participants want to communicate with each other (perhaps simplified as "don't already have a social graph connection"). But nothing in the example of kernel development necessitates to involve the entirety of the world.
Gossiping can be done over a medium that inherently reaches only an interested subset of people, either by self-selection or moderation. Trivially, you could start a verifiable gossip network on top of the kernel development mailing list, where most interested people have subscribed themselves.
SSB instead implements pub servers, which can be designated to contain gossip with a common topic. If and when gossiping becomes inefficient, you simply shouldn't continue to gossip indiscriminately; centralize and federate a bit, but preferably don't become reliant on a central logistical point.