Undocumented: Sendgrid now requires using a static IP if your app runs on Heroku
It would be 80% fails (406 status code in our logs) for an hour or two, then OK for an hour or two, then 100% fail again.
Sendgrid status page said no issues.
Sendgrid support said they could not see the rejects but they'd look into it.
We simply switched our traffic to our "other" ESP (Socketlabs, excellent IMO with far superior support) until we had time to dig into what was going on in Sendgridland..
On a hunch we tested what happened if our Heroku app used a static IP to send API requests to Sendgrid... and it worked 100% of the time. If we turn off the static IP it goes right back to high failures. Went back and forth a couple of times today with the same result.
IMO Sendgrid is pointlessly filtering IPs for apps that A) are using a secure APIKEY to submit API requests, and B) running on a cloud platform where they have no inherent control over shared IPs.
But, since Sendgrid appears to filter IPs, and doesn't seem to document the need for static IPs, and does NOT surface 406 errors in any Sendgrid dashboard where you OR their chat support can see them, I hope this info will someone else from chasing down the error. (And perhaps considering a ESP with much better support.)