As a scientist I would never make such a statement, moreso for a theory I admittedly don't understand.
Tangentially, it's not Tao's case, but I have seen a lot of bullying in the academia along these lines... "I am not saying it's wrong, just very bizzarre", "I am not saying it's wrong, but my Ph.D. student worked on it for two years and couldn't solve it", "I am not saying it's wrong, actually I don't even understand the details, but please recheck everything..." etc...
In mathematics there’s absolutely no place for judging the bizarreness of statements, either they are right or wrong, and of course such a judgement cannot be made by someone who admittedly doesn’t have the necessary background.
As I already said, often in the Academia saying that a result is bizarre is a not-so-subtle way of implying something is wrong by appealing to authority.