Since Firefox is by far the largest HTML5 browser, encoding your video in H264 alone is shooting yourself in the foot.
Saying so completely disregards the fact that a lot of, maybe even the majority of, the video on the web now is already in h.264 alone, because that's one of the encodings that the Flash plugin supports. And when we add to that the fact that for the foreseeable future there will continue being a non-insignificant portion of people using browsers that do not implement HTML5 at all, meaning video sites which go HTML5 will still have to provide a some form of a fallback - most logically through a Flash player - a fallback that can easily be served to Firefox and Opera users, exactly the opposite of Thom's statement becomes obvious. Not encoding your video in h.264 alone is shooting yourself in the foot - you'd be more than doubling your storage cost and your video transcoding cost and even increasing your bandwidth bill, for no good reason whatsoever.