The main One Weird Trick that I remember from messing with ffmpeg is that for in-browser viewing, I had to generate a lot more mp4 reference frames than the default. This was because of deficiency in the HTML5 view extensions in browers a few years ago. I don't know if that is still an issue. The extra frames increased bandwidth consumption, but surprisingly, only slightly rather than big bloat.

No, we use only H264, because nobody has the courage to redo all the work we've done to optimize the encoding with a newer format :)