The caps are set so that the right number of companies fit. Hence ‘500’ in this case. And change to ensure it all works with their goals.
There are many different types of strategies that you’re talking about - momentum, ‘value’, etc. that all set various rules, and are applied in similar ways too.
The analysis I’m talking about isn’t that different, and is still analysis based on fundamentals. It’s just used a bit differently and intentionally trying to be coarse and relatively transparent/mechanical. Private funds keep it opaque and ‘secret sauce’, and have the freedom to do whatever (within the limits of their charter).
Other indexes have more in depth rules that cover exactly the other elements you’re discussing (sector specific indexes, ‘green’ indexes, ‘good governance’ indexes, etc.).
They still use the same analysis tools though.