A minimum solution could be an instruction fetch buffer that memorizes the last N instructions (maybe even after decoding?) to alleviate pipeline bubbles and when jumping back.