Part of the problem is that it's just so opaque. You send the token and the server replies "nope", with no further explanation.
I spent literal days trying to figure out why Client Credentials-flow didn't work until I found an answer buried on their help forum saying oh yeah, client credentials flow isn't supported yet. It is now for IMAP/POP, but still not for SMTP IIRC (though OAuth is not requiredfor SMTP yet).
Next it was figuring out the right scopes, which at the time was not very well documented. Again not very helpful error messages by the servers.
Google's mail servers any better?