Are there improvements that could be done to allow these bots to perform with less rights? That would be something maybe github could tackle but it's not the worst thing about this problem.
This isn't a new problem, how do you prevent a rougue admin from kicking all other admins and taking over. The simplest and a pretty effective solution is to have another privilege level: Founder. Of which there can only be one, and admins can do everything, except strip the founder of their rights. (And/or transfer the repo, if the founder can't easily undo that.)
E.g. removing other admins, or other permissions-related actions like the re-orging in question
If your problem case is "one rogue admin," having multiple admins and requiring consensus seems an easy fix.
Even an admin shouldn't be able to avoid other admins getting notified and seeing an audit log of what they've done.