While I only have a surface-level understanding of video encoding and compression techniques, there are some very good criticisms here. That said, I've been reading through the libvpx source for the past 20 minutes or so. Jason repeatedly slams the quality of the source. The code is much cleaner than he suggests and rather well-commented.
Better than H.264/main or not, I'm excited to see where VP8 goes. With implementations in Chrome, Firefox, Opera, and Flash, it's off to a good start from day one. We'll see where it heads from there.