Since QE increases the value of the stock market at the expense of a currency's value, it is actually eating away at the wealth and savings of individuals and redistributing it to stockholders. These stockholders tend to already be quite wealthy themselves.
Herein lies the problem. Very wealthy individuals tend to spend a much smaller fraction of their accumulated wealth than members of the middle class. Take that as you may, but at some point these massive stockpiles of capital grow so large that they can't possibly be entirely spent. As the stockpile languishes, it becomes wasted capital: money that has essentially fallen out of circulation within the economy.
Another way to think of it is that one person can only do so many things at once, which means there is an upper limit to how much capital a single person can put to good use. When you go from many people with moderate amounts of wealth, to a few people with lots of wealth, you severely limit the amount of creative work that capital can be used for.
...or at least that's my armchair understanding of things.