Trump extremely clearly got in as a nepo kid, so I'd discard him in this argument.
Personally, I find PT's argument sincere that good colleges do measure something important -- even if you disagree with his takes and arguments. I agree with you that his point lost cohesiveness when Bari asked him. He prevaricated -- to a random point on Democrats -- when asked because he didn't want to spell out "Yes, actually, prestige does matter a lot." Something that most, including the hosts of this great blog, agree on tacitly.