His problem is the clear lack of preparation in several interviews. He gets thrown off and loses track just when the interview gets interesting. The momentum dies because he has no good followup questions and no idea where to go form there.
Instead of playing ignorant for the sake of the conversation like good host does, it seems that he tries genuinely to learn stuff during the interview. His backup strategy is to direct interview into the same set of overly generic questions where the quest has nothing interesting to say. It's kind of irritating.
If you compare Lex Friedman to Sean Carrol (they have had same quests in their program) its' clear that Carrol is better prepared and better host.