It appears that this is the crux of the disagreement. Does the loophole that Apple used constitute "giving one company a tax break"? There seem to be reasonable arguments on both sides.
Apple's structures are a correct reading of international tax rules. The rules should be changed; Apple should not be punished for following the law.