Wrong. Silicon image sensors are benchmarked by their "quantum efficiency" - the number of electrons generated by a photon. That number is something like 0.5, which isn't far from 1.0. There is very finite room for improvement in sensors.
https://en.wikipedia.org/wiki/Quantum_efficiencyNow, the following point is not well-known, but the most important thing that determines image quality is simply the frontmost piece of glass in a camera system. Not the sensor size, not the ISO. The front glass determines how many photons you receive per second, how much the depth of field is, and how noisy the image is.
For example, shoot with a certain lens on a full-frame sensor. Now take that lens, put a "speed booster" behind it, and shoot with a crop sensor. Both images will come out essentially the same in terms of field of view, depth of field, and noise. Again, the front glass is what matters, and everything behind it in the optical system are just technical details.
Also as a reminder, shot noise is proportional to 1/sqrt(n), where is n is the number of photons. There are a very finite number of photos you can collect per pixel per aperture per second in real-world shooting situations, and that number is something like 10000. https://en.wikipedia.org/wiki/Shot_noise