It has to do with late game mechanics, which is dominated by team fights. Thus if one team has only 4 effective players, that team will lose, even if this team did everything else perfectly and the opposing team stumbled around early game.

LoL is heavily team focused w.r.t. strategy.