This is true of all subjects, but it’s hard to think of a subject where the gap is as stark as it is in mathematics — even intelligent outsiders have no idea what the subject is really about, and, even worse, they think they know (mainly because we all studied something also called ‘mathematics’ as children and assume that research mathematicians probably do something similar).
By the way, I agree with your original comment.