Theoretically, a single pixel camera that scanned fast enough and held the result in memory could create an image with many pixels, so it seems like this barrier isn't necessarily absolute.
But how well is that going to work in dim light? You're going to hit a limit to how fast you can scan given the time it takes to gather enough photons. When you start to have to be too clever is when you've started to hit limits.
That's about two decades ago, then.
None of that was too clever, obviously.
There is still a limit on lens size that is very hard to work around.. i think that by scanning, a camera like this could function as a pinhole lens, but the effective lens size would only be the size of the single pixel.