The problem is not the chargers, for example in my country, we would need to double our energy production, transport capacity, to phase out fossil fuels. The costs would be enormous, double the infra work in the last 70 years.
Transport is generally put at - 15-25% increase. Due to both needing less refinement and transport, and just how efficient BEVs are.
Electrification generally reduces energy use, while increasing electricity use, due to much higher efficiency.
So if you are correct it will only be because you don't produce the energy used in ICE but instead import it after buying it on the global market.