I think I see where you're coming from, at some point it becomes a question of semantics.
> the Affero GPL only differs in that I would have to give you source code (edit: previously typed "software") if I supply a binary for you to use over a network
That doesn't sound right. You don't have to distribute a binary for the Affero GPL to require you to release your source. From the FSF:
> The GNU Affero General Public License is a modified version of the ordinary GNU GPL version 3. It has one added requirement: if you run a modified program on a server and let other users communicate with it there, your server must also allow them to download the source code corresponding to the modified version running there.