Yes, I agree there a few things that could be done to improve, but they all basically involve giving semi-subjective 'weights' as to the reliability of individual reviewers.
E.g. more likely to be genuine if purchased, if not prepublication (except some people really do receive and review books in advance), if has many reviews, if reviews follow common statistical patterns both per-author and per-book, and so on.
The trouble with all of this is just that it's really, really hard to get right. There's a tremendous amount of 'tuning' involved.
It's probably not possible, but it really would be great if someone could come up with some general elegant theory to solve particularly the 'does this reviewer seem statistically trustworthy', in a way that effectively identifies brigading and harrassment, while still allowing for genuine 'oddballs' whose reviews and ratings go against the crowd.