AGPL style licenses impose restrictions on usage, thus violating freedom 0 in the Free Software’s definition or Open Source’s rule 6.
AGPL should have never been allowed, precisely because it opens the door for commercial entities to eat their cake and have it too, being the kind of license that can be effectively used to disallow commercial products built on top.
In my opinion your own modifications that are never distributed (by the copyright definition) represents mere usage. And we’re software developers after all, personally I patch most of the libraries I end up using and some patches I may contribute back, but I’m definitely not required by any license.
“Open Source” and “Free Software” have great sex appeal, I get it, but if you can’t deal with competitors benefiting from your work, which is the whole point of such an endeavor, then don’t do FOSS.
Go proprietary and be honest about it.