Given the choice between 24 MP with very high sensitivity or 2 MP with ridiculously high sensitivity, I'm not sure the industry is wrong.
With the quality of lens on the average compact camera for instance, I struggle to see any real noticable improvement above around 5 MP.
There are also physical limits. The angular resolution depends on the lens aperture size. Smaller cameras have smaller lenses with smaller apertures, which must have lower angular resolution just because of the laws of physics. At the same time, engineering trade-offs mean that smaller lenses usually have larger apertures proportional to their focal length, so this difference is not quite proportional.
But these small cameras do not have unreasonable resolutions. I just looked up the specs for a tiny point-and-shoot, the Nikon Coolpix AW100. From the specs (5-25mm lens, 28-140mm equivalent) we can figure out that the sensor is probably about 6.2mm across (36mm * 5mm / 28mm). The lens is f/3.9, which gives a resolution of about 400 lp/mm in visible light, or about 5000 pixels across 6.2mm. The sensor is 16 MP with about 4600 pixels across it. So the sensor is close to the limits of the optical system.
If you're having trouble noticing improvement above 5 MP, it is probably due to technique rather than equipment.
* You need to use a tripod. (Or a flash. This should be apparent.)
* You must not open the lens fully, this will blur the image from lens imperfections. (Look at the MTF charts of your favorite lens. They're way worse when the lens is fully open.)
* You must not close the lens more than a couple stops, this will blur the image from diffraction of light through the lens aperture. (Rayleigh limit. You can do the math yourself.)
* You need to be very careful with focus. (In my experience, the "hyperfocal distance" is total bunk when you're doing critical work, which makes sense because the hyperfocal formula is based on the assumption that diffraction is irrelevant. When you want maximum sharpness, you'll be shooting near the diffraction limit so it will be relevant, and therefore the hyperfocal formula is wrong. If I remember right, the adjusted formula is based on lens travel distance from farthest to closest subject point, but I've been using the same lens for so long I use intuition.)
Assuming that you've got a perfect lens. As you say yourself "You must not open the lens fully", in which case the f/3.9 is irrelevant.
And even stopped-down, the glass (of which there's several elements in the average compact's zoom) is rarely of the highest quality. All of this makes for an effective resolution some way short of the figures you're quoting.
The quality of the lens can make a huge difference. On my main DSLR, the difference in quality between, say, the 50mm f1.2L and and an identical shot taken with a bog standard 17-85 f/4, even at the same f-stop is vast. If the cheaper lenses on DLSRs aren't able to really give enough quality to fully take advantage of higher MP, I'm not quite sure why you'd think that the ones on a compact would be.
You might be surprised at the quality of some of the P&S lenses. There have been excellent lenses on cheap fixed-lens P&S cameras for decades: a modern example is the Sony Cyber-shot DSC-RX100, an older example is the Yahsica Electro series from the 1960s. From resolution tests you can find modern P&S cameras with soft lenses and you can also find ones with real, honest 10+ MP resolutions. You don't need to trust my math, you can go check out test shots online.
> As you say yourself "You must not open the lens fully", in which case the f/3.9 is irrelevant.
I also wasn't really explaining things fully with the maxim. Opening the lens reduces diffraction and increases aberrations. A few good lenses will open all the way to, say, f/2 and still get sharper as you open them up. Some terrible lenses aren't sharp at f/8. However, the trade-off is different at different system sizes. For a P&S, wide open might be the best way to go. For MF and LF, you'll probably want f/11 or f/22 to start with, as ridiculous as that sounds. Of course, this assumes you want maximum resolution.
As I have owned several P&S cameras, and done plenty of test shots with them (including in controlled conditions, on tripods in good light etc - despite this not being how the average user is taking the vast majority of their photos), I would not be surprised by the quality of most of them at all. They simply aren't good enough to take a great deal of advantage of much about about 5MP. Yes, you can get ones with pretty good lenses on. But that's far from the norm - and the norm has for years been to fit high megapixel sensors on cameras than can simply not do them justice. Which was the point of my comment.
Of course, if you have more resolution, you can reduce the resolution in post-processing, and that will also reduce the noise. But then you might as well have used larger pixels to start with.
So I think the right strategy is, figure out how much resolution you need and how large a sensor you can afford, and then make the pixels as large as possible at that resolution. This appears to be what Canon has done.
Otherwise you're just saving noise.
Probably if you did a study from people's online photo albums where they just upload originals without thinking, you'd discover that something like 90% of space is wasted on pure noise. Things like the sky or a smooth constant color area that can not be compressed by the jpg algorithm since it is filled with high frequency noise. (Often of the color variety no less.)
There are limits to sensor efficiency. Photon noise statistics gets worse with smaller absolute aperture lenses (focal length times f-number) and smaller field of view (zoom).