Unfortunately there is way too much oil known today. Much of it will have to stay underground, if the worst climate catastrophes are to be averted.
Anyway, the main concern for climate change is coal. Not only it pollutes more, but there is much more of it on the ground than oil and gas. Luckily, solar is successfully competing with it.
Oil in particular seems to be unable to stay at very high prices. Historically we just get a recession and it goes down again.