This huge increase in pilgrims over the last century is because of new technology like air travel, and the peace the world has experienced over the past few decades that allows people to move freely across the world. But it is due to the efforts of the Saudis to expand the masjid, build camps at Mina, allow hotels to be constructed, modernize the facilities, etc. that every person is able to actually perform the rites of hajj at the proper times. Hajj consists of a number of rites that have to be performed in certain places at certain times on certain days. Think about what it takes to organize millions of people trying to do this. It's very different than organizing thousands, or tens of thousands. You have to accommodate all their sleeping arrangements. You have to provide enough space for marketplaces so they can get something to eat. You have to widen the roads, air condition the facilities, make sure each group leaves at their own time, etc.
It's true that the Saudis aren't perfect, and they make mistakes, but my point is that it's easy to get the impression from the massive construction projects that they are just trying to make another Dubai or Olympics. But really, there is a spiritual purpose behind it, not connected to money, and there is also a lot going on behind the scenes that you can't see in the pictures. It would be easier for the Saudis to just restrict the visas to a tiny fraction and not go through with all of the construction and organizing, but they are making the effort as you can see. I know this is not going to be a popular comment here but I'm just trying to offset all the negative comments with some positive.