E.g. here in Germany the law explicitly adds the possibility to issue them to people travelling "in the special interest of Germany", which a former PM traveling to an important event he was invited to because of his status (which the National Prayer Breakfast seems to be) would fit. (There even are exceptions where no representative role at all is required, although I doubt they are used much)
Either way, it sounds like situations like these used to be handled a little more smoothly/amicably. If two nations are friendly to each other, and one fucks up, what's the use in antagonism?
[1] https://www.theguardian.com/us-news/2017/jan/29/state-depart...
Do retired small city mayors travel in first class on diplomatic passports and expect to be free from customs and immigration rules that apply to everyone else? They do not.
He isn't a fairytale prince. When he loses his post, he can travel on legitimate documents like the rest of us.