Clause 6 of the GPLv2 says that you "cannot impose any further restrictions" on recipients of the software. One possible interpretation of this clause is that the App Store terms of service impose further restrictions on recipients.
Thus authors of GPL software might have a basis to object to other people distributing their software through the App Store, even if full source is provided. ...at least, this was the rationale given by the FSF when they objected to GNU Go in the App Store: