Hacker News Reader

Comment by tgv | Hacker News Reader

So this is a processor where you would have 97% of the threads doing some I/O like task? But that can't be disk I/O, so that would leave networking?

View on HN

DRAM is the new I/O. So yes, this is designed to handle 97% of the threads doing constant bad-locality DRAM accesses.

And with this, DRAM access becomes the new asynchronous IO.

Neat insight

The protocols for HPC are so amorphous that they bubbled up into the lowest common denominator, completely software defined async global workspace

I think generally the threads are spending a lot of time waiting on memory. It can take >100 cycles to get something from ram so you could have all your threads try to read a pointer and still have computation to spare until the first read comes back from memory.

It could be that eg 97% of your threads are looking things up in big hashtables (eg computing a big join for a database query) or binary-searching big arrays, rather than ‘some I/O task’

This processor is out of my league, but do you have any idea how a program would use that optimally? How do you code for that?

Let's say that I can get something from RAM in 100 cycles. But if I have 60 threads all trying to do something with RAM, I can't do 60 RAM accesses in that 100 cycles, can I? Somebody's going to have to wait, aren't they?

this would work really well with rambus style async memory if it every got out from under the giant pile of patents

the 'plus' side here is that that condition gets handled gracefully, but yes, certainly you can end up in a situation where memory transactions per second is the bottleneck.

its likely more advtangeous to have a lot of memory controllers and ddr interfaces here than a lot of banks on the same bus. but that's a real cost and pin issue.

the mta 'solved' this by fully dissociating the memory from the cpu with a fabric

maybe you could do the same with cxl today

I’m not exactly sure what you mean. RAM allows multiple reads to be in flight at once but I guess won’t be clocked as fast as the cpu. So you’ll have to do some computation in some threads instead of reads. Peak performance will have a mix of some threads waiting on ram and others doing actual work.

> But that can't be disk I/O, so that would leave networking?

Networking is a huge part of cloud applications, and network connections take orders of magnitude longer to go through than disk access.

There are components of any cloud architecture which are dedicated exclusively to handling networking. Reverse proxies, ingress controllers, API gateways, message broker handlers, etc etc etc. Even function-as-a-service tasks heavily favour listening and reacting to network calls.

I dare say that pure horsepower servers are no longer driving demand for servers. The ability to shove as many processes and threads on a single CPU is by far the thing that cloud providers and on prem companies seek.

Reply on news.ycombinator.com