Taste is important but very subjective. I don't buy a water filter unless it shows up as certified by NSF[1].
A few years ago I almost fell for a "water purification system" on Amazon, with stellar reviews that was supposed to filter out everything. With a baby on the way I figured it was worth the price tag. Good thing I did my research. The water filtration system was not certified or verified by anybody.
edit:
Same thing applies to humidifiers, air cleaners, air conditioners, etc. Don't buy unless it has been certified[2].