To avoid replays, though, this should be reset once a code is accepted: only subsequent codes should be accepted afterwards.
So when code N is the "correct" code, any of N-1, N, or N+1 will be accepted. But if N is entered, only N+1 and onwards should be accepted for later attempts.
If I am very quick and try to also log in to her bank with "Birthday Bear" and 123 456 that shouldn't work even though that's still the "right" code for the next few seconds. The bank login should tell me that I need to wait and enter the next code. I can't see Sarah's next code because she's done and put her phone away.
RFC 6238 says of this: "The verifier MUST NOT accept the second attempt of the OTP after the successful validation has been issued for the first OTP, which ensures one-time only use of an OTP."
So, to fix your issue, just use the current code even if it seems expired.
As a side note, it's important to enforce that each code can only be used once. Authy (the server-side service) does that, but many "roll your own" TOTP solutions don't verify this.
Codes are typically valid for 3-5min [...] just use the current code even if it seems expired
I’ve never had an Android device that keeps its clock in sync, for example, so they had to address this.