(Numbers 100% made up and metaphorical, not scientific.)
I'm one of those who cares. I used to flick through Letraset catalogs before I even knew what typography was, because I thought the different effects of the letter shapes were fascinating.
But I'm perfectly fine with the understanding that a lot of people really don't notice the difference between SvelteStylishFontX and Comic Sans.
So... it's hard to get clean data about this, because if you take the population as a whole you may not see much of an effect.
But in web design, your target audience won't usually be the population as a whole. Design effort matters if your audience is more likely to include those who appreciate it.
Finding and testing that audience scientifically is damn near impossible. So if you think it may matter, it's best to add design sugar and then check the analytics to see what happens.
That's going to tell you a lot more about your customers than a theoretical study in a journal.