Have you tried knockd https://linux.die.net/man/1/knockd ? You send a special sequence of "knocks" to the server (packets to different ports) and it executes a command such as allowing your IP for a time period. No JWTs.
And yes, ssh pubkey + fwnop + WireGuard + fail2ban is ridiculous overkill, but hey, it’s my homelab server. That’s how I learn this stuff.