You can't stop being "the guy who decided not to be the King of England"