Completely non-novel patents being awarded by the thousands is the real problem. If the "trolls" were really just enforcing interesting and valid patents on behalf of individual inventors, that's one thing. But they're instead finding the most broad patents they can and using them to extort money from companies who are successful, sometimes regardless of applicability of the patent to the companies in question.
The fact that software can be patented at all is questionable in my opinion -- anything sufficiently complicated to be worthy of a software patent would be copyrightable. Anything less complicated should be held to an extremely high standard of novelty before being allowed a patent -- and even then I would put the time limit on such a patent at 5 years or less.
Just having an "idea" that no one has happened to already patent yet and sitting on until someone else thinks of it and implements it doesn't "promote the Progress of Science and useful Arts", which is the underlying mission of all IP protections in the US at least.
The patent database is full of so much noise that no one is mining it for ideas and creating software based on it at this point; instead everyone is coming up with similar ideas, and pretty much BY DEFINITION those patents shouldn't be valid.