Is that really true? RAM is dirt cheap compared to everything else that goes into a server.
And then later under "Lessons Learned" we have: "Put it all on one box if you can. If you don’t need a distributed system then don’t build one. They are complicated and expensive to build."
We had a quartet of legacy-ish databases with sharded customer data (had some interconnection, but we sharded in the application-layer, so the database didn't have to move any data internally). It was a bitch to maintain, but it performed OK, by throwing fairly beefy machines at it.
We began a quest to replace the database with MongoDB/HDFS/Riak/CouchBase/Cassandra/whatever. We wrote a script that converted our data to JSON for easier ingestion and began hammering away. Until we discovered that by converting it to JSON (and simplifying some parts of the data-model), our dataset had actually become small enough to fit on a single server in memory (still ~100GB).
So we ditched the cluster-thingie and moved everything to CouchDB. Pricy hardware, but the saved man-hours recouped that within a month or two.
(And in half a year's time, the database will have outgrown whatever hardware we can reasonably throw at it, so then we'll have to start over for real...)
You need a rdbms. Field names won't be repeated for each query and you will push later the time to 'start over'.
I expect that slimming down the data model was the main factor, or you simply didn't want to use SQL at all (or your data wasn't really structured enough).
Getting to 1TB would certainly require 32GB sticks and would be the point where multiple machines gets cheaper.
If you're renting servers or virtual servers (VPS/cloud), then the physical cost of RAM is irrelevant. Virtually all hosting companies use configured RAM as a proxy for their costs (the more RAM you need, the more likely you are to be using more CPU which means more electric, more disk IO, more bandwidth, and generally putting more wear-and-tear on the hardware which will need replacing sooner).
An 8GB stick of ECC RAM might cost you less than $99 to purchase outright, but Softlayer (the #1 most used bare metal server company by YC startups) will charge $88/month for it ($11/GB/mo to add 8GB to one of their base configurations).
I never understood this. I started and sold a hosting company in the early 2000's. I've run managed hosting divisions for Fortune 500 companies. Hosting is STUPID profitable (>30% margins). Why doesn't Y Combinator grab 10-20 racks in a cheap colo center, buy $100K-$200K worth of equipment, and provision that to YC startups similar to Rackspace's OnMetal offering?
I do DevOps now, but colo is still not as hard as everyone makes it out to be.
It's an interesting question whether there's a market for an investment firm that does do this. One of the things that appealed to me about Google under Eric Schmidt was that it basically functioned as a venture capital firm that had shared technology infrastructure, server maintenance, legal, PR, etc. but let you choose what to work on and come up with your own product ideas, with the one caveat that Google would own any business you built. It'd be very interesting to see a VC firm that did the same - cover all the functional roles so that entrepreneurs could focus on pure product development. I hear a16z is sorta shooting for that, but they cover more of the human-capital stuff like recruitment, executive search, etc. rather than the technical stuff.
An Amazon m3.large with 7.5GB of RAM and a 32GB SSD costs $64 per month if I reserve a year in advance. (And why wouldn't I, given that the proposed alternative is to buy hardware?)
If I wave the magic wand and save 30% on the cost of each m3.large, I save $19 per month per instance.
If my company runs the equivalent of twenty m3.large instances per programmer, 24 hours a day, for a whole year, my magic-wand-powered savings will add up to six additional programmer workdays - about $4600 per programmer. If I start out on January 1 with a year of runway, I'll go broke on January 9 of next year instead of January 1.
So, even if I were to believe in the existence of magic wands, they wouldn't make much of a difference.
I am aware of many companies where advertising, conferences (throwing their own), laptops/mice/etc, rent, general office overhead, credit card processing, and about five other things I'm forgetting are still higher than infrastructure/hardware, in addition to direct costs of employment. (Obviously, less true if you run Dropbox, but COGS is looooooooooow in most SaaS companies for a reason.)
Let's say a 1GB module costs $25 and a 4GB one is $40. (Just picking some random Corsair ones of compatible specs and disregarding things like using matched pairs of lesser capacity instead of single modules. For simplicity.)
Two of those 1GB modules would already cost more than one 4GB one. And that's to say nothing of the rest of the hardware, which you can entirely avoid if you have a single system instead of four.
The only place it makes sense is VMs through a cloud provider, where memory is often one of the big pricing metrics. It's possible there could be savings through some of those providers.