The other issue is that even a fast charger takes considerably longer than a gas pump to refill your car. This, coupled with the relatively low availability of them overall, means that you are more likely to have to wait in line for your turn and you spend even more time not on the road traveling to your destination.
With all this being said, I think an EV is very compelling as a secondary vehicle for 90%+ of use cases so long as you also have an ICE vehicle and a sufficient way to charge your EV at home.