While it't true that oil is used for plastics and chemicals today, it's absolutely not the case that it will be drilled into the future when demand for gasoline disappears.

Oil is not strictly a necessary precursor to much of the chemical industry, it is rather a convenient raw material because the high value of gasoline makes lights virtually free. In the early days of oil exploration, lights were constantly burned as they had no economic use at the scale they were produced.

If oil exploration would continue just for the sake of chemicals, the cost structure of these precursors would fundamentally change and it would become more attractive to replace them with renewable sources.

It will still be less if demand is lower.

And if plastics become more expensive due to no longer being a "free" byproduct of oil consumption their usage will also go down.

If oil is only being used in a product that won't eventually be burned and produce CO2, then that's a big win.