Actually, I think they're being cautious. You may be fine for now, using AGPL code, but if it's modified at some point in the future, then the viral issues kick in. That's probably what the lawyers are worried about. You can't assume that everything will stay the same.

Quite a few people here agree with you.