The problem with having no throttling is that the system will just keep on running happily, until you get to the point where resources become more limited. You will not get any early feedback that your system is constantly underprovisioned. Try doing this on a multi-tenant cluster, where new pods spawned by other teams/people come and go constantly. You won't be able to get any reliable performance characteristics in environments like that.

For such clusters, it's necessary to set up stuff like the LimitRanger (https://kubernetes.io/docs/concepts/policy/limit-range/) to put a hard constant bound between requests and limits.