I do see a bit of a difference between a summit organized by an elected official and bureaucrats in unelected positions exerting personal influence. At some level when we elect someone we elect them based upon how they will exercise their discretionary authority. The role of the president isn't simply to carry out process. There is discretion and how they execute the duties of their office and holding a summit as in your example is one of those things. The recourse here is that is an elected official and if the people are unhappy with how they're executing that discretionary duties they simply elect another person.
When you're talking about a bureaucratic lifer at some agency who is supposed to implement regulations and regulatory oversight that Congress has enacted that's a completely different thing. What recourse do the people have if this non-elected bureaucrat is not acting in an impartial way when they execute their duties?