Basically, if the Result doesn't match up with Expected, the browser is doing it wrong. Ideally browsers should handle YUV colorspaces like so:
H.264 video - look for colorspace tagging in the video by default and use it if available, otherwise fall back to guessing based on resolution. SD video (up to 1024x576) should be converted with Rec.601, HD video (width >1024 or height >576) with Rec.709.
VP8 video - VP8 is defined as Rec.601 only, so always use it.
Theora - Same as VP8.
How browsers actually fare today (tested on Windows 10):
IE11/Edge - Always assumes Rec.601 for H.264 video. Doesn't support VP8/Theora.
Chrome - No colorspace tagging support for H.264. Converts HD video with Rec.709, SD video with Rec.601. 1024x576 is treated as HD already. VP8 is always converted with Rec.601 as it should. Theora gets Rec.601 in SD but incorrectly uses Rec.709 in HD.
Firefox - No colorspace tagging support for H.264. HD uses Rec.709, SD uses Rec.601, 1024x576 treated as HD like in Chrome. VP8 and Theora both always use Rec.601 as they should.
The unfortunate conclusion from this is that color accuracy is pretty much a crapshoot when dealing with HD video on the web. The only way to guarantee accurate results right now would be to convert your video to Rec.601 (if you're mastering HD video chances are you're using Rec.709 by default), serve VP8 video by default and have a Rec.601 H.264 fallback for IE/Edge (I haven't tested how Flash video playback handles this matter so you might also need a Rec.709 H.264 fallback for that).