Yeah, this is a regression since macOS Tahoe.
Amazing that it still exists after several patch releases, is audio working not a basic test case for Apple?
I’ve found it to be worst when using Xcode / simulator and having headphones on for music.
sudo killall coreaudiod seems to fix it for a while.