Is it as clear cut as binaries == license required; source code == no licence required? Is this specific to the codec's patent pool or how patents work in general? What about interpreted languages or bytecode?
Of course, you'd better not plan on using anything you learned from it until you negotiate licenses for your users (or the patent expires).
I think this is all left intentionally vague by patent owners, since they want you to just pay to be on the safe side, there's plenty of straightforward things you technically can't do with patented codecs, but which you'll never get sued for e.g. the person you pay to video your wedding has no licence to sell you the H.264 file the camera produced, and no way of getting one.
So I think it's at least partly based on what they haven't bothered to sue anyone over, though perhaps free speech comes into it in the USA.