They pay taxes over there alright (e.g. VAT), which is fine, nobody complains. The issue is they don't pay corporate tax on it in the US, despite the fact Apple is US based and does virtually all of its work in the US, and then commissions factories it doesn't own in China to produce its parts. The end result is that it pays much fewer taxes than citizens do, or companies that don't have a dedicated fiscal team (mostly SMEs which comprise the bulk of American firms, e.g. 90% of firms have less than 20 employees, and 99.9% have less than 500).

Now it's one thing to say it's justified because Apple would have to go out of business and lay off thousands of of their 100k employees. But we can all take a look at that financial situation and make our own moral judgement on whether they should have effective tax rates much lower than a regular business.