In Graphics: Supercomputing Superpowers | Hacker News Reader

In Graphics: Supercomputing Superpowers

news.bbc.co.uk

I just watched this the other day - http://www.youtube.com/watch?v=J9kobkqAicU Cray-1 Supercomputer 30th Anniversary. I would give up my career and turn it around if given a chance to work in HPC.

What part of HPC would you like to work in? I have some contact with various HPC facilities but have never envied the people who work there -- they're just sysadmins who happen to work in larger, noisier server rooms than most.

Well, I'm just one of those sysadmins (but the clusters on which I work are much more smaller than the 'monsters' in the top500). I don't know if someone could envy me but there are surely other things apart the noise of HVAC systems (by the way we are using remote console when possible :-)). How about working with a lot of different computer nodes combining them with a fast network (Infiniband) and exporting the storage space through the cluster with a parallel file system? How about optimizing every aspect of the software installation (OS, compilers, scientific libraries) to give your users the best resources to run their simulations (biophysicist, electrical engineers, etc.)? To not to mention follow the aspects regarding the structure of the data center: cooling systems, UPS, raised floors, etc.? It doesn't sound too bad to me :-)

Thanks, but I doubt I will ever be so lucky, especially considering even if I did have an opportunity I'd probably lack in experience needed for such a job. Anyways, while fantasizing, my dream job would be to work in r&d on building those HPC machines (software and system wise, algos especially, not EE), or writing/optimizing code that runs on them, especially vector processing units. My area of interest is graphics research.

These kind of articles are why I really love the BBC.

While the supercomputers themselves are interesting, I really like graphic. I think it displays the information in a more clear and compelling way compared to something like a stacked column chart, like http://www.highcharts.com/demo/?example=column-stacked&t..., which would just be too busy with a lot of information.

So where does Google fit in on that scale?

Google has a lot of computers, but they are not tightly coupled enough to be considered super computers.

I'm curious about the applications of these systems. Why is it that finance would be the third largest (known) group for supercomputer use? What kind of number crunching would they be doing?

A lot of it is statistical arbitrage.

That is: you look at the historical prices of all commodities over time and try to figure out which ones tend to vary together (eg copper-mining companies go up when copper prices go up... but you're looking for less obvious examples than that). Then you look at whether current prices diverge from these trends at all, and if they do you buy/sell accordingly. At least, that's the handwavey version I know -- I'm sure whatever they're doing at Rennaisance and DE Shaw is something I don't even know about.

Not a lot of it is statistical arbitrage - yes this is what Rennaisance and D.E. Shaw do, but this is not what banks (sans perhaps Goldman Sachs) devote much of their compute power to. It's just too hard to make money doing this compared to more traditional activities like market making.

As stated, risk management is a big application: VaR and market stress scenarios are computationally intensive, particularly for portfolios with path-dependent derivatives. Pricing is the other big application: it is similarly computationally intensive to value derivatives against the market-implied term structure of volatility.

http://en.wikipedia.org/wiki/De_novo_protein_structure_predi...

How does that relate to finance? I know that protein folding involves massive computation, as it was what our university cluster was being used for 40% of the time.

Sorry, for some reason I only read the first and last sentences of your post. Maybe they are used for economic forecasting by governments?

I don't know why the grandparent mentioned it, but I do know that D.E. Shaw works on both.

There's DE Shaw, the Hedge Fund and DE Shaw Research. Different focus for each entity.

I can think only about trade forecasting, and natural resources management (like estimating oil exhaustion).

Possibly used for risk management, generating lots of different scenarios and calculating their P&L impact. This would include VaR calculation.

Some derivative pricing models require a lot of cycles.

Consolidation of multi-dimensional hierarchical data can be surprisingly compute intensive. Combine that with VaR calculations etc. then it doesn't surprise me that serious kit is required.

Reply on news.ycombinator.com