Hacker News Reader

Comment by NathanRice | Hacker News Reader

As someone tangentially involved in the administration apparatus for large, well utilized clusters, I have to dispel some myths related to "Cloud" services and Amazon EC2 for computation. First, if your school has institutional cluster services, these are FAR cheaper than cloud options. Secondly, the type of systems that are good for scientific computation are not the type of boxes typically deployed in the cloud; try to find a 48 terabyte memory node in the cloud.

OSG/Teragrid are much better options than the cloud, though you do have to jump through some hoops to access them. Believe me when I say it is worth it though.

View on HN

Definitely agree. To give a ballpark figure, Compute Canada, the governing HPC body in Canada provided a cost estimate to researchers who applied and received dedicated computer allocation. Although, all this computer time is actually "free" to the scientist:

"The average RAC award in 2012 was 500 core years and 50 TB years. From Penguin Computing's website [list pricing from Feb 10th, 2012] the cost to purchase a similar amount of computational time and ability would be $450,000. ($600 / server month where each server provides 8 cores) To purchase 50 TB from SDSC [again, list pricing from Feb 10th, 2012] it would cost $30,000. ($600 / TB Year) Making the total value of the average RAC award nearly $500,000. The cost to Compute Canada to provide this level of award is $178,000 and that includes all aspects of user support, training and other aspects not covered by commercial clouds." (https://computecanada.ca/index.php/en/component/content/arti...)

As a side-note, my groups award was above the average, The email to our research group stated: "Providing these resources to your research group costs Compute Canada roughly $4,669,960." I don't even want to calculate how much that would cost on EC2!

500 cores is approximately 32 cc2.8xlarge instances (16 cores/instance). If you are running a highly utilized cluster, i.e. you are going for a one year heavy utilization reserved instance, the compute part is going to cost you ~$262,000. If you went with a 3-yr RI, it would be $183,000/yr.

Or to put it differently, ~$0.04/core-hr prior to any volume discounts.

Absolutely. Also, while the article may be right that a lot of research-group clusters being run out of a closet are only getting 10% utilization, this is not true for shared institutional clusters. Those systems are often hitting utilization of 70, 80, or 90%, and are a lot more cost-effective than a cloud provider.

This is why a lot of institutions request that money spent on new compute hardware be contributed to the shared system (in exchange for a large allocation), rather than building tons of tiny clusters.

Reply on news.ycombinator.com