Presumably because between statistical error and the variability of human biology, a problem that affected a large-ish subpopulation would likely not be detected with such a small study group.
They always cherry pick who is included in clinical trials and I don't hear the same complaints about that.
You aren't listening, then, because the fact that clinical trials seek people maximally healthy but for the target conditions rather than a representative sample of those with the target condition is a very frequent complaint.
Where are you seeing this? In every HN thread about some medical trial it is the same complaint about small sample size. It is like a generic complaint thrown by people who don't realize that they are almost all like that...
The agency has to approve your selection criteria and those criteria affect what you can write on your label (i.e. who the drug can be used for). Good example: the Gardasil vaccine is only approved for those under 27 as it was only tested in that age range.