They started with a simple implementation where one node handles one request at a time... end to end. They used ECS for easy-button scaling. That's a perfectly reasonable approach for starting out.

I would have probably pushed for a shift in orchestration to kubernetes along with some tweaking as an initial uplift. Others would re-write the whole thing in another language. They chose to add a bit of complexity for multiple requests per node support. They all have their pluses and minuses, but in the end it doesn't mean the initial approach was bad, or their refactor wasn't pragmatic or practical.

Dramatic rewrites to a codebase lead to instability and in practice fail as much as succeed.

Your comments in this thread have been breaking the site guidelines, and getting worse as they go along. Would you please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here? Note the bit about curious conversation.