What do you do while it compiles?

20-40 seconds? That's not enough time for a full context switch.

Take an eye break (look at something more than 20 feet away). Stand and stretch your back. Do a few desk-friendly stretches or exercises (curls, squats). Go to the kitchen for a beverage refill.

Somewhere around the 5 min mark for a wait it becomes time to check email, etc.

It's not enough for a full context switch but if you start opening your email, you might stay there longer than the 20-40 seconds and become a full context switch.

Eye break is a great idea.

Exactly

I like to use Focus (https://heyfocus.com/) on the Mac to prevent myself from actually doing these context switches.

Wow, old days, working with clipper, it took between 15 and 40 minutes. We went to the bar and hoped the compiler didnt break the moment we entered the bar.