That is just the interconnect. Grid jobs are usually very compute heavy: "[...] includes the majority of the 100,000 processing cores in the CE." That is the expensive part. We bought p2.16xlarge equivalent nodes for just about 1/20 of the "3 Yr All Upfront Reserved" cost of EC2.

Depending on the utilization (i.e. somewhat low utilization) you could save some money in the cloud if you compare it to the usual 5 years depreciation cycle.

But for scientific computing you usually still use your computing nodes longer than the 5 years of depreciation/support contract (often until they are close to fail).

Another reason why cloud is not very attractive is with how budgeting rules work in those old institutions. You propose a cluster and get the money, all calculated from the start. On-demand cloud pricing will be very hard to get approval for.

> [...] other costs like networking and storage [...] I am currently using the equivalent 20k dollar in yearly EBS costs on our scientific computing cluster. You can buy a lot of storage for that. And the internal IT department that exists anyway will manage it.

Cloud can be a very hard sell to scientist.

Edit (a point I forgot): Cloud only makes sense if you can use spot market capacity extensively, then it can become very competitive with local installations (again depending on required utilization).