I haven't experienced the buffering issues you describe, but I've definitely seen the resizing issues from leaving fullscreen (screenshot here): http://cl.ly/2y3F0B3V0X09013U0710. I've noticed this doesn't happen after full-screen, however; there's resizing issues anytime you toggle between different video sizes.
I should point out that my connection is pretty fast, around 75 Mbps.
I also checked out the caching issue you pointed out: you're dead-on. There's no caching whatsoever with HTML5. In general, the buffering behavior is kind of odd. Doesn't behave at all like the Flash player.
Weird that you don't have that resizing/offset issue when exiting fullscreen, but rather when toggling between video sizes. Everybody seems to be having different experiences.
I was using Chrome 16 to test the cache. For the HD stuff, I must say: I can't find many HD videos to try it on. HD videos are more likely to be supported by big corporate accounts, and they'll opt for ads in their videos, too, which means that the video won't be HTML5-compatible (that's the real distinction). If the video doesn't have Google Ads, then it can be played in HTML5. Otherwise, it has to be Flash.
Yeah, it took me a while to find some HD videos. I noticed it when stumbling across an advert for Breaking Bad which was uploaded by a user, if I remember correctly.
I haven't tested on all of my computers yet, but I've updated to Chrome 17 on my Mac and the caching issue seems to have gone away. I believe I still have Chrome 16 on my Windows machine, so I'll hopefully be able to test them side-by-side.
I haven't seen the bugs you mentioned, but will look out for them now.
My player is designed for language learning, so the users are constantly seeking: http://bit.ly/z9J8Cc