It's not impossible to write a neutral or negative review for an item and still include an affiliate link -- the only slant I could see happening is trying to include something positive, but I would argue that we should be doing that anyways when we are criticizing something. If you are an honest reviewer, you may be concerned about people not clicking affiliate links for badly-rated items. The solution is to try to find something similar that you can honestly give a positive review for, then link to that positive review from the negative review. Your readers will appreciate you for your honesty and will be more likely to trust your positive reviews if you follow a policy like this.
The problem becomes a bit harder as a person reading someone's reviews. You can usually identify affiliate links pretty easily just by scanning the URL. If the author is using an affiliate link, it is up to the reader to determine how much faith they have in that author. If they do not believe in the author's integrity, then why are they reading that author's reviews in the first place? After all, affiliate links aren't the only way to skew a dishonest author's integrity (think: free goodies, free trips to "review" expensive items, etc.).