In addition, when you are dealing with many thousand application servers and many hundred database connections, the database will not end up happy having to maintain > 10k connections.
So you will end up needing to add a proxy or connect every time. As you indicated, since MySQL connections are so cheap (ie. you can do > 50k per second), it is faster and easier to create a new connection each time.