You’re right, but also, there’s more to it.
There is a difference between what people say and what people mean and context is important. When I’ve heard people say “trust the science” it is always directed to people who have demonstrated that they do not understand the scientific method and the meaning of the word “healthy skepticism”. In that case what you are doing is an appeal to authority, which is completely unscientific, and necessary for building consensus.
So when we hear “trust the science”, it’s really a political statement more than anything, and it intends to replace a person in a robe, or a person in a uniform with a person in a lab coat.
Certainly it would be better to have an informed and well educated citizenry. But for people who live in the real world and do not want to have their society steered by people who have incompatible views with a modern society, saying “trust the science” is a shorthand for “we don’t think you have what it takes to contribute to a conversation in a meaningful way.”