If the whole subtitle file is out of sync (e.g. every line is late 2s), one shortcut is all it takes. In that 1/10 case, you do need to 'fiddle around for ages', but given that all other options (e.g. SubtitleEdit[1]) are way more complicated and time consuming, mpv is all I could suggest. For the problem with audio and image being out-of-sync, I have no idea :)
---