The problem is that people disagree to the extent Apple should be compensated (or their ideas protected). More technology-minded people can see the roots of Apple's devices, and so think it's inevitable and all Apple did was "popularise" it, while others think that Apple's innovations were/are a stroke of genius, and without them we'd be floundering in a world of Windows CE and Symbian for the next decade.
The trouble is, no one's wrong. It's just a difference of opinion. And that's the patent system in a nutshell. People spend so much time arguing Android v. Apple that it's easy to lose sight of the bigger picture, which is that we're really arguing "how far can a company have a monopoly on an idea? and even if they did invent something, is it fair that they can impede the progress of an entire industry to protect that invention?" Because I can assure you, even if Apple was responsible for every chip and every bit in the iPhone, from concept to execution, people would still want Android, and people would still resent Apple for wanting a monopoly.