It's mostly seen as a bad thing in the lens of ethnic nationalist ideology that low fertility rates will change the demographics of a country (see Great Replacement [0]).
After all, if Musk (and other capitalists) needed, they can get labor from other countries south of America's borders, but there is a push to not make these workers Americans! This isn't an issue only in America either.