Hacker News Reader

Oh, and not to tar with toooo broad a brush: One guy contacted me the other day wanting a terabyte of memory. He'd actually done all his homework and had a weirdly interesting use case.

I'm actually pretty excited to explore getting his problem solved; it's likely that a terabyte is about the right amount to get him over the finish line, and he's converged on this algorithm precisely because it has fairly horrifying memory needs, but trades off to give a nice big-O runtime.

I've heard that some financial institutions use a YARC supercomputer with 256TB of RAM to do daily "full market watch" of client activity to determine insider trading for compliance purposes.

I can't even imagine the power draw of such a system or the heat generation. Fucking galaxy brain engineering. At that point you need like custom motherboards and special buses, right?

Finance guys tend to be able to afford to have the right people to be sure of that on hand; they also are specifically in the business of cost-benefit analysis.

I have no insight or information this; but sometimes you can throw money at a problem to make it go away, and that's literally the best deal open to you. If custom fab solves a problem more cheaply than the 25 senior software engineers it would otherwise take, then custom fab it is.

That’s about a 2,000 node Cray, something like 14 cabinets which isn’t that big of an XC configuration.

> I can't even imagine the power draw of such a system or the heat generation.

I’m unfamiliar with the machine specified in your example, but I’ve seen in person HPC clusters with more RAM than that, so I’ll use that to give you some napkin math calculations.

Dell R730 with 24x64gb ECC DIMMs = 1536gb RAM. To get to 256TB, you’d need about 170 of them (aka about 5 racks). Power draw varies depending on a number of factors, but let’s be very conservative and take a worse case and say 750W per 1U server. That’s 127,500W, which is just over 600A at 208V, so say a long block of houses with their A/Cs cranking in the summer. Now as to the heat generation, each R730 puts out up to 2850 BTU/hr, so for all 170 of them, that’s just shy of 500,000 BTU/hr. Continuing the comparison to homes, that’s 5-6 home furnaces worth of heat generation per hour.

I don’t have any photos handy of the 64gb DIMMs referenced above, but here’s a photo of 130 16gb ECC DIMMs (aka just over 2TB):

https://www.dropbox.com/s/ww1fwdn7o9se14z/2020-06-15%2020.39...

If instead of the above based on a real world HPC machine, you based it on current gen Dell R750s which can hold 32 256gb LRDIMMs totaling 8TB per machine, you could get to by with just over a single rack of machines for 256TB, albeit for a much higher price (256gb LRDIMMs cost as much each as roughly the price of a used R730 complete w/ 1.5TB of RAM). The cluster of R750s with that much RAM configured as indicated would be well into the 8 figure range.

If you're allowed to call a multi-node setup a "supercomputer", then how about 3 40 machine racks of Xeons where each one is dual-socketed, has 12 DIMMs per socket, and is using 128 GB DIMMs?

3 racks * 40 boxes * 2 sockets * 12 DIMMs per socket * 128 GB = 368640 GB or 360 TB raw. Remove some of that for overhead (kernel, OS gunk, network buffers, whatever code you'd have to run to make it into a cluster), and there's your 256 TB with a bunch left over.

Stuff like this is why I sometimes say that a lot of these startups could be done with "three Xeon racks" if they actually cared about doing it right.

[Note: assumes 2nd generation Xeons (Skylakes), with 24 DIMMs. Ice Lake = more/bigger memory, fewer actual machines.]

I share the same feeling. I am a PhD student with working knowledge of application development. My skills have been quite useful to the rest of the lab as they transitioned to using HPC systems.

> * "I need a terabyte of RAM"

A lot of our work involve running algorithms such as PCA and ICA on thousands of biomedical imaging data. It is totally normal for us to allocation 1TB RAM. We have 3 nodes with 1.5TB RAM in our cluster. Aapproximation algorithms do exist and are used for resource optimization though.

To be clear, sometimes a terabyte of RAM is the right solution. But it's pretty common to find someone needing that simply needs a better algorithm. [or, recently, a mathematical identity that meant the task could be done streamed from disk]

Tbh if you need a terrabyte of RAM to simply run your experiment then you aren’t doing it right. We haven’t reached a level where an algorithm would require at least a terrabyte of RAM just to get baseline results.

Terabyte of RAM is my favorite because I choose to believe he had a perfect data model and was literally telling you exactly what he needed :D

In my recent work doing financial modeling (options, cryptos, other nonsense), I pretty much just try to cache absolutely everything with some arbitrary 1 week TTL and then continue to cache new data every day. Cache hits cause a 10% bump in the current TTL time delta. Kinda heuristic but whatever, works in most cases.

Any other funny stories? I always love these engineering horror stories. Have you looked at Julia at all? Would be interesting to hear if there's been an uptick in usage from the research community from a veteran research code surgeon!

The mathematician that designed the model I referred to at the end ended up passing it on to me. I've now built something of an empire of people and projects using it, "just" by making better tools to manage its inputs, outputs, visualisation and automation.

I was stupidly excited to recently do more work on that same model - I took one of its internal datastructures and turned it from a simple queue into a priority queue using a couple different queueing metrics. In both cases, I had run cases out to a scale where I was able to actually show going from O(1) to O(log(n)) performance - but got beautiful wins because the fastest work is work you never have to do.

Not often I can empirically show traditional CS101 stuff in a real-world chart.

Your work sounds awesome.

(And you probably mean O(n) and not O(1), otherwise O(log(n)) isn't really an improvement).

No, that's the point; the previous algorithm was O(1) [a simple queue] but that led to doing a huge amount of extra work. By using a smarter queue that "cost" an extra O(log(n)), we could skip work that didn't need doing in the first place.

That sounds like a pretty satisfying job. How'd you find it?

This place has been an astonishingly good fit for me, yeah. I was laid off from my last job, and a friend passed along my resume at an opportune moment. That was twelve years ago, and I'm still here.

My group has a perpetually open posting and are always looking for people: https://rand.wd5.myworkdayjobs.com/en-US/External_Career_Sit...

[If the link doesn't work, go to rand.org, search jobs, then search for "research software engineer"]

EDIT: That link is to a specific posting. Our perpetual one is going back up any day now, once we get through the machinations.

What kind of profilers are you using? If you can list a mapping between language and profiler name, that would be sweet

The stuff I've used in recent months:

  - For Java, personally I like the netbeans profiler
  - Octave's built in profiler gets the job done: https://octave.org/doc/v6.2.0/Profiling.html
  - I use JetBrain's tooling a lot.
     - I like the profiler in Rider for VB.NET and C#
     - The one in PyCharm is ... fine. Python profiling is ugly, PyCharm does a clean job handling it.
  - For R, Rprof is ... fine: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rprof.html
  - STATA's built in profiler does as good a job as possible given it's a weird language

For various other things, where a profiler doesn't really exist per se:

  - SAS can be configured to output runtime after each proc. There's no cost, you should do it
  - SAS proc SQL has an undocumented _method flag: http://support.sas.com/techsup/technote/ts553.html
  - Past "EXPLAIN QUERY PLAN", one of the tools I build ends up doing a simple visualisation of SQLite's explain query plan output for *every* query it runs. There's no way to *not* get that estimate

Those are the tools I can think of that I've used in the last few months.

I appreciate you taking the time to write this out. I'm curious about the lack of any mention of C/C++ tooling - I thought there are some mathematicians who write stuff in C/C++ and leverage libs like GNU SL (Scientific Library) etc. I've written a few custom pricers and needed to use those libs and they were pretty efficient, and didn't need to resort to anything crazy to run them in production.

Curious if you ever find stuff written in those languages, as opposed to the gamut you just mention, which are classical "research tools" (language/IDE combinations that let you not need to think about the language specifically.

I don't cross paths with a lot of C or C++ where profiling is a concern. I use gprof for that, but I haven't needed it in a few years. None of the folks I work with reach for C or C++ as a weapon in their quiver.

I do cross paths with a fair bit of FORTRAN, but in my experience, generally if someone's writing FORTRAN, they already know the deal with performance.

Fun fact: When I interviewed for my job [12 years ago], I interviewed with a guy using SQLite and FLTK in a C++ model. I'd been using SQLite and FLTK for some recent personal projects; in the interview, I solved a bunch of performance problems he'd been having for a couple years. I interviewed on Monday, and started working there two days later.