Hacker News Reader

Comment by fsckin | Hacker News Reader

I dug around a bit and found a research paper[0] about TerraServer. It's got all sorts of neat details not in this link.

TerraServer hardware configuration parameters:

Max hits per day 40 million/day

Max SQL queries per day 37 million/day

Max image downloads / day 35 million/day

Bandwidth to Internet 200 Mbps = 2 Terabytes/day

Concurrent web connections = 6,000 connections

Web front ends: Six 4-way 200 Mhz Compaq Proliant 5500, .5GB ram

Database back-end 1 8-way 440Mhz Compaq AlphaServer 8400 10GB ram, 3.2 TB raid5

324 9GB Ultra SCSI disks

324 x 9GB disks for 2.9TB? You could store 1.2PB with the same number of 4TB disks. Doesn't seem like Moore's Law is holding up for storage, nor for internet speeds.

[0] http://www.math.uaa.alaska.edu/~afkjm/cs401/project_fall_04/...

View on HN

324 disks takes up most of a row. When we had to do service maintenance in Canyon Park (fun with NT 3.51 and 4.0, before the days of Terminal Server) I'd walk over to TerraServer and admire it's enormity.

>324 disks takes up most of a row.

Not any more. Using high density disk enclosures, you can fit that many disks in a third of a rack, and if each disk held 4.5TB you'd have almost 1.5PB of storage!

What an astonishing change in only 15 years!

Just under 20 years ago when I got started in this racket, 30G was consider a huge database, cutting edge stuff, requiring experts and special techniques. Nowadays a serious DBA doesn't blink at 30T.

I wonder if 324 on Lustre would have more throughput than a 1 4TB disk?

Definitely... and _way_ more IOPS.

They would be leaps and bounds faster. Especially comparing random IOPS.

Let's just assume they were using a Seagate Cheetah 9LP. That's a 9GB, 10k RPM, 1MB cache UW SCSI disk that was widely available in 1998, for a whopping... $1200 each. $390,000 worth of disks... wow. That could buy 1300 * 4TB SAS drives today with a total raw storage of 5.2PB!

324 * 9LP's would have around 5.5GB/sec of total read/write throughput. That easily beats a single 4TB disk with around 200MB/sec read/write performance and terrible IOPS in comparison to even a single 9LP.

SSDs compare much more favorably here. 10 x 1TB SSDs for ~$6500 could easily reach 5.5GB/sec throughput and triple the capacity of the 324 x 9GB drives.

You're comparing off-the-shelf disks to enterprise-class storage.

"Enterprise" 4TB disks don't have orders of magnitude more IOPs than consumer versions.

But they are more expensive, so for comparison to be right one needs to take that under consideration.

You can't just compare storage on the capacity, you also have to account for speed. Even these days, 149GB 15k RPM SAS drives are popular for non-SSD usage. Long way from 4TB, but they outperform them any day when it comes to semi-random usage.

I wonder how much of that usage came from people doing the MSDN tutorials. Those were the golden days of MSDN - I remember learning about factoradics and permutations from my 10 CD MSDN library. Microsoft was doing great stuff back then.

I am thinking to do a back-of-the-envelope orders-of-the-magnitude analysis for a setup similar to this for an Amazon AWS EC2/S3/RDS/foo/bar/baz based system.

Moore's law is about semiconductor transistor count -- why would it have anything to do with magnetic storage?

Reply on news.ycombinator.com