US Airforce builds supercomputer out of 1,760 PlayStation 3 | Hacker News Reader

US Airforce builds supercomputer out of 1,760 PlayStation 3

techeye.net

19 pointsbjonathan26 comments

I thought the reason it made economics sense for the Air Force to use this gaming console (as opposed to any other computer) was that Sony sells PS3s at a loss in order to spur sales of PS3 games. But Sony can't be subsidizing this by more than $100/unit, which would only save the Air Force of order $1 million. How is it saving them more than 10 times that? Are PS3s really sold at such scales that they are 10 times cheaper than a non-gaming option for equivalent computing?

Yes, and the place they mention is 34th slot in Top500 ~ 190TFlops

So, suppose a bespoke 190TFlop beast would normally cost $20M - at a bargain $0.10 for one flops!(http://en.wikipedia.org/wiki/FLOPS -Costs of computing)

Looks like with their setup they are getting $0.01 each flops.

So you're saying that the fact that the PS3 is sold at a loss has very little to do with the cheapness of this supercomputer? I still have trouble believing economically that Sony gets it's processors ten times cheaper than normal. Why couldn't the Air Force just buy the processors from whoever makes them for Sony? Yes, Sony probably has a discount from the processor manufacturer for buying in bulk, but a factor of 10 doesn't seem possible.

Sony makes the processors :)

http://en.wikipedia.org/wiki/Cell_(microprocessor)

I'm quite sure that IBM actually makes the processors, at the plant in East Fishkill, NY. In the history section of that Wikipedia Article, IBM manufacturing the chips is referenced several times, but never Sony.

Yeah, but they codeveloped them with Sony, so Sony presumably gets a really low price.

This doesn't answer my question. Even if Sony did make the chips all by themselves (which seems dubious from comments below), then they own some sort of magic capital or intellectual property which lets them produce such processors way cheaper than every other company on the planet. What is this magic thing?

TFlops are a very bad metric for computers. A rocket car isn't better than a Ferrari. A monster truck isn't better than a 4x4. Computers and cars have very complex specs.

It can be true if you take it as a given that they want to use the Cell processor. Cell blade servers cost about $20,000. There are no options for getting the Cell between a PS3 and a Cell blade.

But, those tend to have two Cell processors, they're a more recent version, and it's an actual high performance computer, with lots of memory and fast interconnects. So they are in no way "equivalent."

And the Cells in the blade probably have all 8 SPUs enabled.

This doesn't answer my question. There are alternative processors out there. Unless the manufacturer of the Cell processors has some sort of magic which lets them make them 10 times cheaper than all other companies, there should be plenty of competitors which would be happy to supply a medium sized order.

1,760 PS3s 50,000 times faster then average consumer laptop. Even adding PS3 GPU power this statement looks like exaggeration.

The 50x statement looks out of context. Probably the required algorithms benefit from a huge data bus, the special fast XDR RAM, or having processing cores very close.

Might not be that much of an exaggeration, though. The AF probably uses it to simulate things like airframes and nuclear explosions; those are embarrassingly parallel, so you'll get full use of all the cores. At 8 cores per box [1], that gets you up to 14k times faster, if the cores are the same speed as that laptop.

[1] The Cell has one PowerPC core and 8 SPUs; but the PS3 is shipped with one SPU disabled, so they can get the yield up.

And another SPE is reserved for the GameOS, so code running under Linux on a PS3 can only access 6 SPEs.

I'd love to hear their opinion on removal of the "Other OS" option. I also know several other institutions that have built their own supercomputers out of PlayStations and aren't that happy about that move.

Why not? If the supercomputer was pre-existing, the Other OS option won't disappear unless they do a firmware update. The only reason I know of to do an update is to play the latest games, so unless they're having sessions of GTA in between protein folding, I doubt it's an issue.

Replacing broken PS3 hardware is an issue. As in, they can't do it.

Could they downgrade the firmware?

PS3 firmwares can now be downgraded. Somebody discovered a service dongle key, which allows for downgrading FWs (but not the latest). That would make it slightly easier to find replacements.

Also somebody is working on some patches which allow linux to run on top of the same gameos hypervisor that the commercial games do. This allows access to an extra spu and slightly more ram too. It does require an exploit to allow the execution of unsigned code. Here's the git repo: http://git.marcansoft.com/?p=asbestos.git

Hector Martin has been playing with kinect for the past month so that explains the lack of progress.

I don't think this is a big issue if you're buying in bulk. With that many PS3's, you should have a good estimate of how many are going to fail in the lifetime of the supercomputer. I doubt the Air Force didn't buy spares already, since they don't want to rely on Sony building new PS3s.

That was a very dick move from Sony. They promoted it like crazy and got a lot of us investing time on this architecture just to back-stab us in the end.

On the other hand, a large institution can invest in a PS3 development kit and develop their crunching software to run within Sony's OS.

Anyway, Cell is dead and both Intel and AMD have similar architectures coming out next year for x86. I wouldn't bet on PS3/Cell right now.

This made a lot of sense when the PS3 first came out, but since then we've gotten low-power architectures and extremely powerful and cheap GPU's.

http://www.nvidia.com/object/io_72770.html

Those, for example, are dirt cheap and have similar benefits to the Cell. And you don't have to work around a platform that was designed for playing games.

What sort of applications do you run on, or what do you do with a computer 50k times faster than the average pc?

I believe most supercomputers simulate things, e.g. climate science, protein folding, aerodynamics, QCD, etc. There is pretty much a ceiling to processor speeds, so the special thing about supercomputers is that they put a bunch of processors in parallel. The problem of simulating large physical systems happens to be particularly amenable to parallel computing.

Nuke simulations.

Reply on news.ycombinator.com