I'm with you on this. Five-star ratings don't pass my astroturf gut check these days. One-star [1] or two-star or three-star ratings are usually there because there is some glaring flaw that everyone is upset about, and it's usually worth reading the reviews to see if it's a problem I'll also encounter.
Fifty percent of the time, that "flaw" is a bunch of complaints about price or IAP [2]. The other 50% is crashing on certain devices. Vanishingly few people with thoughtful, in-depth grievances about an app will actually take the time to review it. They'll just delete the app and move on. So there is a bit of a selection bias at play in the ratings set.
[1] I do feel there's a big gulf between one star and two stars. At least in my anecdotal experience, a one-star rating often means the app is so buggy as to be borderline unusable, or else fundamentally flawed in some way. A two-star rating is a very different animal; it often means the app itself shows promise, but it is priced poorly, or else it has stability issues on certain OS versions or devices. Three stars is where you start to enter the territory of legitimately mixed opinion about an app's quality in and of itself. (Assuming, of course, that the number of reviews is large enough.)
[2] Incidentally, these complaints are often valid, albeit expressed in very peremptory ways. I'm not dismissing them, especially when they concern pay-to-play IAP mechanisms. But I do take them with a grain of salt.