Good point. Now that I think of it, I've seen at least 4 diff websites that had video annotations and I've never seen even a single video that had annotations that were valuable. They're more annoying than valuable (think ads).
As for flickr... some of them are useful. Facebook's people annotation, for example, is super-useful and it adds a lot of value.