"Here is why the consensus is _______" is not argument by authority, correct.

"The ________ theory/hypothesis is fringe, the consensus is ________" is argument by authority.

See the difference? Phrasing matters in this case. All too often it is phrased the second way. I don't expect someone to literally enumerate all the evidence for or against something, but there is a middle ground between that and just saying "well consensus is ______".