Someone liking or disliking Comic Sans is just a matter of taste. Fonts are not design, they are just fonts. You can make a good design with Comic Sans, and you can make a bad design with the best font in the world.
There's a lot of things like that. A classic example I'm very aware of is something called "juice" in game development, which you would never notice whilst playing a game until you play one without it, at which point its immediately obvious.
I think that might be directly contradicted by your first sentence. People do care. Design matters, and typeface selection is a big part of that.
If someone plastered a inappropriate font all over hackernews you'd notice right away.
Good design isn't always noticeable but bad design is frequently the first thing you'll see when you open that new app and fonts are a part of that.