That is why the supporters of this amendment (and similar approaches like the NPVIC) should be prepared to settle for a compromise which preserves the disproportionate power of the lower-population states.
In fact, such a compromise was proposed in 1950, with the bipartisan Lodge-Gossett Amendment[0]. The idea behind it was that the electors in each state would have to vote proportionally to the popular vote within that state while not changing the number of electors granted to each state.
Because of the small number of electors in some states, rounding issues can be quite significant in some situations, and I believe the proposal required fractional electoral voters, representing a tenth of a percent of the state popular vote.