You're reading him too narrowly. He's answering the question of whether the brain can be modeled (math or magic), and arguing against it. Or maybe he's arguing for a middle ground. Hard to say exactly, he's not very clear.

Except the brain can be modeled with arbitrary precision - so he's got an uphill battle in either case.

TLDR: Yes, he literally said "binary computer" but from the context it's clear he means more than that.