The size of the team is irrelevant. It's like asking if a load balancer is useful for a two person team. What's the system the two person team is in charge of, and for how many users? The two people might be in charge of the backend for a sprawling complex system with millions of users, or they might also be the only users of some small local physical bulletin board made out of cork.

At the end of the day, AWS is still physically made up of physical computers that still have to obey the laws of physics. If you max out the bandwidth of the connection going in or out of, say, an RDS system (which is available via NDA if you are spending enough money with them), suddenly the physical is very important.