Prompt caching only works based on the prefix. Let’s call your output Y and the low reasoning prompt A and a medium reasoning prompt B.
Previously you were at A+Y. Switching to medium reasoning makes it B+Y. There’s no prefix which can be cached, so the entire B+Y needs to be reprocessed.
Why can't effort just be at the end of the prompt? My understanding is that that is how modes and other "dynamic updates" like the time are handled.