“ Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation. “
We will always want to do the discovery ourselves, and I can see why fighting that instinct is a challenge for those in the field.
Clearly someone felt that there'd be a better inductive bias and attempted something else, and now CNNs are what's used "in the long run".
One could look at the move from linear models to non-linear models or the use of ConvNets (yes I know ViTs exist, to my knowledge the base layers are still convolution layers) as 'leveraging human knowledge'. Only after those shifts were made did the leveraging of computation help. It would seem to me that the naive reading of that quote only rings true between breakthroughs.