Split the re-triable action into one function, make a wrapper function that re-tries if needed, and use a third function that makes the decision to re-try and how long to back off.

Then you can test the decision function trivially, the re-try function by mocking the action and decision, and the action function itself without back off interfering.

That’s what you suggested, just saying that I did that in a Python API client with the backoff library and the result is pretty neat.