What’s the size of a photon? You seem to be suggesting that the limiting factor is the physical size of the sensor, which is totally absurd. Photography optics are not even close to approaching the limits of physics.
The relationship between aperture sizes, pixel sizes, and diffraction sizes certainly do effect the quality of images:
* https://www.cambridgeincolour.com/tutorials/diffraction-phot...
* https://fstoppers.com/science/closer-look-lens-diffraction-5...
* https://www.edmundoptics.ca/knowledge-center/application-not...
Now, the following point is not well-known, but the most important thing that determines image quality is simply the frontmost piece of glass in a camera system. Not the sensor size, not the ISO. The front glass determines how many photons you receive per second, how much the depth of field is, and how noisy the image is.
For example, shoot with a certain lens on a full-frame sensor. Now take that lens, put a "speed booster" behind it, and shoot with a crop sensor. Both images will come out essentially the same in terms of field of view, depth of field, and noise. Again, the front glass is what matters, and everything behind it in the optical system are just technical details.
Also as a reminder, shot noise is proportional to 1/sqrt(n), where is n is the number of photons. There are a very finite number of photos you can collect per pixel per aperture per second in real-world shooting situations, and that number is something like 10000. https://en.wikipedia.org/wiki/Shot_noise
QE is a ratio as described by your Wikipedia link. There is a massive difference between a ratio of 0.5 and 1.0!