As an open source maintainer, you're perfectly entitled to just walk away. Hit that archive button on github so nothing new can come from the repo, and enjoy your life. Nobody could fault on you that. People would prefer you work out a clean transition to a new maintainer, but you have no obligation to do so.
Deleting the repos is the equivalent of setting the house on fire on the way out (especially when they're under their own org). Unlike in reality, it turns out it's totally permitted, but people will still view it as a dick move.