I mean, if you're going to ban Douglas Crockford from your conference it should at least be for his stated prejudice against comments in data-interchange formats. Not vague allegations that may damage his personal and professional reputation, to which he not only has no right of reply, but any response could be damaging by generating more attention. This is the classic trolling strategy, but stepped up a level.
If he actually did something wrong, take him to court and let the facts be decided in law. Otherwise, he's innocent and should be treated as such.
I also wonder, given that the allegations haven't been published, just implied, if he would have a libel case against the conference organisers?