I think it's a little more complicated than that. A pool of 1000 workers won't necessarily earn less than a pool of 100 workers. With higher specialization, they may well earn far more.
However, this assumes that employment is fluid, and that people are allowed to choose a profession or trade that aligns with market signals and their own personal interests and preferences. In short, every citizen who considers becoming a developer int he valley is also free to become a plumber in Duluth or a lawyer in New York City. Or open a scented candle shop, or a surfboard shaping business.
The point of the H1B visa is to create a system where visa restrictions and sunk costs create a pool of workers who aren't allowed to make those choices, and are limited to a few fields where Silicon Valley CEOs have declared a "shortage".
And, astonishingly, many people talk about this shortage without discussing wages, work conditions, career stability, and the other options available to highly skilled people who can choose other professions (you know, those pesky free folk who just won't become developers in the valley at the rates tech CEOs have decided they should).
The US has a really grim history of creating non-free worker classes in order to undermine free labor markets. This is another installment of it, though the H1-B abuses, clearly documented and really bad, are nonetheless minor compared to some of these historical horrors.