Is there some sort of account ID other than an email address? It looks like the email is all that gets sent to the website (although I didn't read deeply enough to be sure). If you change (add new, remove old) your email address at browserid.org then revisit a site you authenticated to before, what happens?