I'm sure it depends on your field: in mine (academic science, which is as plagued by name-dropping and elitism as any) the thing that matters is the last thing you did.
Of course the whole thing is a trajectory. If your goal is to climb the ivory tower (it probably shouldn't be but anyway...) you need to be lucky or rich at some point. But if it doesn't happen at at undergraduate admissions you just have to do actual work, and then get lucky at some later point. So yes, it's screwed up. But that's not my question.
What I want to know: how often are people getting hired based on their undergraduate institution beyond their first job out of collage. Is this a thing? In what field?