It's very possible he had other things already lined up that weren't easily changeable.
That alone would piss off any congressman/senator. Pretty sure anyone telling US Congress/Senate that "I can't show up because I have something more important lined up" is making a big mistake.
This is the lawmakers whose ego is rarely matched...