Hacker News Reader

Comment by tracker1 | Hacker News Reader

For banking... accurate, simple, safe, reliable are more important than performance/throughput. IMHO optimizing the above and for developer efficiency should be the first priority and for scale or max throughput later.

The simplest solution is to scale to one worker per node initially if you're doing anything compute intensive... once you've done that, and/or you need better performance for any number of reasons including cost, then you can do more. Now, I'm not sure I would have gotten to 4k nodes before I started to re-evaluate parallelism or better scaling options, but the initial implementation is absolutely fine.

View on HN

> For banking... accurate, simple, safe, reliable are more important than performance/throughput. IMHO optimizing the above and for developer efficiency should be the first priority and for scale or max throughput later.

I get it, but come on - this was not a "performance optimisation" issue, but one of bad architecture; an architecture that certainly doesn't inspire confidence in the priorities you mention: accuracy, simplicity, safety.

Having a worker that does one request, processes that one request and returns a result isn't accurate, simple or safe? Scaling that simplistic interface in a 1:1 manner across many systems via ECS is pretty straight forward.

Now, in addition to probably optimizing what they've done, converting to, for example another container system, like K8s where they can scale vertically a bit better may have been another approach.

The biggest issue that I see is gRPC doesn't work that great with Node. You can't use it with cluster which means you have to self-manage threads/processes and it adds complexity there. Yeah, there's definitely issues that come into scaling in terms of performance optimization.... but where they started from isn't unreasonable imho.

Where I work compliance is job #1.

That doesn't prevent us from thinking about performance. GTFO with this nonsense.

Your comments in this thread have been breaking the site guidelines, and getting worse as they go along. Would you please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here? Note the bit about curious conversation.

They started with a simple implementation where one node handles one request at a time... end to end. They used ECS for easy-button scaling. That's a perfectly reasonable approach for starting out.

I would have probably pushed for a shift in orchestration to kubernetes along with some tweaking as an initial uplift. Others would re-write the whole thing in another language. They chose to add a bit of complexity for multiple requests per node support. They all have their pluses and minuses, but in the end it doesn't mean the initial approach was bad, or their refactor wasn't pragmatic or practical.

Dramatic rewrites to a codebase lead to instability and in practice fail as much as succeed.

> Dramatic rewrites to a codebase

Otherwise known as re-architecting?

Reply on news.ycombinator.com