the in-memory layout of a YUV-encoded frame is different than the one of a RGB one. it needs conversion, which implies an intermediate copy, as well as traversing a buffer that can go up to 1080 rows.
most video players (and Flash on Windows and MacOS), these days, use shaders and multi-texturing to do that conversion on the GPU; it removes a copy, because you just push the YUV frame as it is to the GPU - and your CPU can go back to a low C state.
I think Flash on Linux does this as well, on nVidia: the blog post linked is from 2010. the way Adobe detect capabilities is, in itself, hilariously bad - instead of checking the GL required GL extensions like any normal people, they do a check on the GL vendor string. the Mesa guys pointed that out, but I doubt they ever received a response.