I don't work at Amazon, but worked at a similar (European, national) retailer:

You really have to walk the line between ratings-per-SKU and aggregate ratings sometimes.

In an ideal world, the reading experience of a Kindle book is the same as the physical book. So someone might argue that ratings should be aggregated across all version of the product. Would you expect different ratings for hardcover/paperback versions of books? Probably not, at least assuming people review the content of the book, not its appearance.

Similarly, do you really want people to leave ratings on a black shirt in size L, and that rating not apply to all sizes and colors?

I assume someone at Amazon thinks about this, and the defaults (fashion => merge, books => merge, electronics => donotmerge) etc. do OK until complaints come in. There's no profit in not being heavy-handed.