You're missing the global nature of the market. If every oil producer agreed to reduce supply it might work. But less US oil means others step in and make more refineries and dig more oil wells to meet the demand. Price isn't set based on demand only either, opec basically fixes the price. The oil price will go up for a few months and then stabilize. I mean have you seen oil prices this year? They surpassed prices in apocalyptic movies and people are still using fossil fuels just the same. Maybe tariffs on oil can help hike foreign oil prices but what politician want to be responsible for increasing gas prices? The result will be when you eventually have widespread EVs and charging stations short of supermajority popular support, you will need to beg saudi, russia,etc... to reduce supply or increase prices so people actually switch to EVs.
I for example, almost never drive but at the same time I will not get EVs unless you force me because they are unreliable for long distance, I hate queues (for charging stations) and living in apartments means either they get dozens of chargers for the complex or I have to drive somewhere and wait like an hour to charge (if I can make it to the station). So you really have to twist my arm to get anything short of a hybrid.
That said, I am only responding to your question I don't support or oppose oil subsidies since I am not informed enough on the subject