nope. AGPL has no clause to say you have to contribute back. you only have to PROVIDE source to your users. thats all it does.
contributing back would mean whatever change you put in your fork must reflect in upstream which is not something all upstream users would accept always but its not a requirement.