Setting a lower resolution format via "ytdl-format=..." in the mpv.conf file may resolve that. Personally I also use "pause=yes" and find that the few extra seconds it has to cache also helps.

As far as I know, it worked out of the box for me after installing both mpv and yt-dlp on Arch linux.