Indeed. The whole system relies on trust.
In computer science, it's usual to know who is writing because the specific field is small and you basically know who is working on what anyway. I guess it's even more so in math.
You'd better not piss off anyone in your community.