Even formal mathematical proofs "appeal to authority" when they invoke theorems without re-deriving them every time. It's a shorthand reference to knowledge that time and energy has already been spent discovering. The parallel for this in society, where the present cumulative scope of knowledge is completely impossible for a single person to independently discover on their own, is called expertise. When you are denigrating experts, you are denigrating acquired, shared knowledge.

Pure reason has limits, but one of the neat things about rational argumentation is that it can be applied to itself to discover what alot of its own limits are.

To be fair, a good argument only needs to appeal to two sides of the rhetorical triangle. Logos and Ethos are two perfectly viable platforms to make a statement on.