Maybe you should administer the test towards the end of the interview after you've gotten to know the candidate a bit and allowed for time for their nervousness to wear off.
If 80% of candidates are failing, you want to ask it right at the start, to minimize the time you waste interviewing people who aren't qualified.
But that seems to assume that FizzBuzz is the defining test of a candidate. I guess if that works for the interviewer then that's the way to go.
A minimum competency test is reasonable. Better to know such things before the interview stage IMO; its a terrible waste of everyone's time to show up, fail fizzbuzz and leave.