Met Office to build £97m supercomputer | Hacker News Reader

Met Office to build £97m supercomputer

bbc.co.uk

49 pointshereprompt30 comments

Even the Met Office's press release just says it weights as much as 11 double decker buses rather than any actual technical details. I presume they didnt buy it by weight, why dumb everything down.

Ah Cray has some actual details[1].

[1] http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...

100 tons, according to the radio this morning.

The double decker bus is a standard unit of measurement for height and weight, for large areas it's Wales.

Medium areas are well-covered by football pitches.

Olympic sized swimming pools give a little more granularity.

To be fair, that press release does give

- quantitative information about the new system's power relative to its predecessor ("13 times more powerful"; I assume they mean something like "13x faster on typical workloads")

- quantitative information about its speed in absolute terms ("more than 16,000 trillion calculations per second")

- quantitative information about what it will let them do that they couldn't before ("forecast updates every hour", "very high resolution (300m) models", and few particular projects they intend to use it for)

- a little more tech-spec information, albeit stupidly presented ("120,000 times more memory than a top-end smartphone", which I presume means either 120TB or 240TB)

- how much they're spending

- how long they expect it to take to get it set up and working

and they do give the weight in tonnes as well as in double-decker buses. I don't really see that there's much of a problem here.

To compare to the top supercomputers, Cray Titan cost around $160m at the end of 2013 and has 17 petaflops. Met Office also use Cray, so it might be that they ordered Titan with a little less flops, a lot less memory.

>The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three.

Maybe I'm missing something but should a 13x speedup allow it to run in 1/13th the time? Or maybe the "13x" is only for part of the system and they have a bottleneck somewhere else.

See: http://en.wikipedia.org/wiki/Amdahl's_law

Yes but they said 13x speedup not 13x more processors/cores. Probably just a non-technical writeup tbh.

Maybe there's not much point in running it more than once an hour. For example, maybe the forecast is only reported once an hour.

Edit: A quote later in the article says "It will allow us to add more precision, more detail, more accuracy". So it's probably increasing the resolution even though the part you quoted sounds like it's not.

Is it worth it to buy supercomputers these days or just rent cloud time? I guess it you need it all the time, this may be the way to go, but couldn't you get a bulk rate from Heitzner or OVH?

Last time I did the math owning your own hardware was still significantly cheaper especially if you're an institution that already has some of the infrastructure in place (and covered by overhead costs on grants).

And this is weather simulation so stuff like network topology matters and they will be running their models as often as they can.

The cloud wins for more elastic work loads and people without IT departments and server rooms.

If cloud operators provided Fiber Channel networking, they would probably consume most of the flex capacity of those that need supercomputers. One example I know of (for weather simulations no less) has >100 tflops capacity, but their utilization is about 30% because they have little ability to level load for their customers.

> The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three.

Let's say an order of magnitude speed improvement. :wow:

I wonder if it will be able to model the local microclimate effect of powering and cooling 140 tonnes of compute?

I misread and was really wondering what the Met Police would do with a GBP 97M supercomputer.

I totally thought this was Met Life Insurance, which would have been mildly plausible. ;)

I would love to see the comparison between running this and just buying cloud compute time.

Given its a constant operation which probably requires high speed interconnects. However it would be interesting to see how close in cost cloud is to a bare metal build.

here's a hint - it's nowhere near close in cost to a bare metal build, at least not in my experience.

We run a small clustered database across 8 nodes. Total cost for the hardware was ~ $3000 per node. Cost for a comparable node in EC2 per month? $1500. Even with co-lo costs for power and bandwidth, and buying 10gige switches it's far cheaper for us to host it in house than it is to use EC2.

How about the cost of bricks and mortar? The cost of an administrator? The cost of spares and disaster recovery? Sure the purchase cost looks good, but what about the total cost of ownership?

It's still tiny compared to AWS, which is really expensive.

That doesn't mean it's not useful, but it's still tens of times more expensive than using your own hardware.

Often total cost of ownership is about the same as the equipment cost. So they start to look comparable actually.

That's not what I've found. Even just renting dedicated physical machines has been significantly cheaper overall.

Former engineer on large research grants (in the us) here. At least in the US a lot of those recurring costs are covered by the standard overhead your institution tacks onto all grants for administrative and facilities costs so you buy a bunch of machines with extended warranties (which should be longer then the terms of most grants) and that covers almost all of your costs. Once they go out of warranty you either use them until they die, scrounge for funds to keep them going or hopefully get a new grant.

Also, things are changing, but it is/was hard to ear mark funds for the cloud in a grant so spending the money up front was often what needed to happen.

Cloud HPC Solutions Architect here. In this use case, cloud computing wouldn't really suffice. For a company who needs 24x7 access to compute power of that scale, they really will benefit from buying a standalone system like this.

While some cloud providers now have very high-speed, low-latency interconnects, they only scale so wide in a virtual environment. This is due to limitations in the physical datacenter space and the way node provisioning works on the back-end. If you truly need extremely wide, low-latency, high bandwidth interconnect for a large period of time, on-prem is still (probably) your best option.

What's a good source for learning how weather models work? I'm interested in understanding how the accuracy changes with the number of calculations available.

Literal cloud computing...

Glad they now have a faster way to make inaccurate predictions :)

Computing power actually matters for the accuracy of weather prediction. Having more computing power would allow them to refine the spatial and temporal resolution of their weather forecast model.

http://en.wikipedia.org/wiki/Numerical_weather_prediction#Co...

Reply on news.ycombinator.com