The simple joys of scaling up
motherduck.com
motherduck.com
This feels like the kicker:
> In the cloud, you don’t need to pay extra for a “big iron” machine because you’re already running on one. You just need a bigger slice. Cloud vendors don’t charge proportionally more for a larger slice, so your cost per unit of compute doesn’t change if you’re working on a tiny instance or a giant one.
It's obvious once you think about it: you aren't choosing between a bunch of small machines and one big machine, you may very well be choosing between a bunch of small slices of a big machine and one big slice of a big machine. The only difference would be in how your software sees it: as a complex distributed system, or as a single system (that can eg. share memory with itself instead of serializing and deserializing data over network sockets)
Really, though, you're generating enormous overhead while turning on and off small slices of a 128-core monster with a terabyte of RAM.
over my career i’ve seen countless cases where the complexity of a distributed system introduces as many (or more) reliability concerns as it solves. when everything works well, it can be beautiful. but when things go wrong, they can quickly become a cascading mess of hard-to-debug issues.
> DuckDB Labs from its inception has received many investment offers from venture capitalists, but eventually decided not to take venture capital, and rather opted for a business model where it works with a few important partners. “We look forward to the continued growth and adoption that partnering with MotherDuck will bring. Thanks to the MotherDuck team, we are able to focus our efforts on independent innovation of the DuckDB core platform and growth of the open-source community.” says Mühleisen, who currently combines a tenured research position at CWI with the role of CEO at DuckDB Labs.
https://www.cwi.nl/en/news/cwi-spin-off-duckdb-labs-partners...
https://duckdblabs.com/news/2022/11/15/motherduck-partnershi...
https://www.techtarget.com/searchdatamanagement/news/2525272...
For ~$100/mo, you can get:
AMD Ryzen 9 7950
16-cores
128GB RAM
4TB NVMe
For ~$800/mo, you can get:
AMD EPYC 7763
64-cores
256GB RAM (up to 1TB)
4TB NVME (up to 12TB)
Be interesting to see if wafer scale becomes more common. I would think wafer scale CPU core farms would help cloud providers deliver greater one-big-server goodness.
850,000 cores
40GB of ram distributed among cores
220Pb/s of memory bandwidth
Result: 7,500 trillion FP16 operations per second
Tech dive: https://www.techinsights.com/blog/cerebras-dives-wse-archite...
It's cache or scratchpad memory. Keep in mind this is "on chip", most systems, including this one, have additional memory off chip in the form of DRAM sticks.
edit: For comparison, the NVIDIA A100 has up to 164kB of shared memory per SM while the wafer beast has about 47kB per core if you divide it out.
A m5.large (2 vCPU, 8GB RAM) is $0.096/hr. m5.24xlarge (96 vCPU, 384GB RAM) is $4.608/hr.
Exactly 1:48 scale up, in capacity and cost.
The largest AWS instance is x2iedn.32xlarge (128 vCPU, 4096GB RAM) for $26.676/hr. Compared to m5.large, a 64x increase in compute and 512x increase in memory for 277x the cost.
Long story short.....you can scale up linearly for a long time in the cloud.
That kind of kills this whole argument except in certain pretty unusual cases.
Easiest to just create a new instance scaled up vertically and swap it over.
Those two items can be substantial (5x headcount), or they can be merely "standard" (2x headcount).
They're never zero extra headcount.
Whatever isn't the "unusual cases" you mentioned.
> We scale up and down thousands of times a day.
Sounds unnecessary for most cases.
Only if you're paying on-demand prices. If you're paying spot prices, the spot market is much less linearly correlated.
But one other thing I think they fail to cover is this: most people don't need to build distributed systems anywhere near as soon as they think.
Scale-up makes the most sense for startups and small projects that haven't gained traction or scale yet. It's perfectly possible to architect your applications ready for the day when you _do_ need to scale out, but without spending time and money on the complexities of it until you actually _need_ to.
I am building a new kind of data management system. It can manage unstructured data like a file system as an object store. It can build very large relational tables and query them quickly. Its flexible schema allows the management of semi-structured data like NoSql systems. (https://didgets.com/)
It has distributed features built into the architecture, but I have only implemented the single node so far. On a $2000 desktop computer it can easily manage 200 million files and tables with 100M+ rows or thousands of columns.
The issue I see now is that there is no good way to know what files will match well when reading from remote (decoupled) storage.
While it does support hive partitioning (thank god), and S3 list calls, if you are looking at doing inserts frequently you need some way to merge these parquet files.
The MergeTree engine is my favorite thing about ClickHouse, and why it's still my go-to. I think if there was a serverless way to merge parquet (which was the aim of IceDB) that would make DuckDB massively more powerful as a primary OLAP db.
Not sure on the economy of scale though - if you need redundancy on a scale up architecture you essentially need to have parity compute sitting idle (likely in another location/region/data centre).
If you have an HA requirement your infra spend has just doubled, and the ROI on your DR tends to nothing (unless you can do something clever with on demand DR)? You’re also replicating everything your write to DR (so network and storage there).
Also more practically, you’re just less flexible. Waiting 30 minutes for a machine to checksum 1TB of memory before loading the OS is painful in an outage scenario (every reboot hurts, replenishing cache hurts, losing a massive instance hurts). Do like the sentiment around simplicity though!
Doesn't matter when it's still a lot cheaper than some the big clouds.
You can also HA to cloud with a hybrid approach so on-demand infra is only used to fill in when your main compute goes out.
You're quite correct, but it's not such a simple trade-off.
1. What's a "disaster"? A) "Datacenter burns down" is very different to B) "Instance got overloaded with unexpected spike, need more capacity". For A), your distributed system will still have outages in a particular area, so you'll spin up new instances in the nearest live DC, at a time when everyone else is trying to do the same! And B) is not a disaster.
2. Disaster recovery is, by definition alone, an exceptional event. If you're reaching for DR protocols more than once a year, there's something seriously wrong. Being able to scale out means that you only need a few minutes before you're up to capacity, but shaving off (say) 30m in the event of a genuine black swan event depends on the exact business. I've seen Fintechs, FAANGs, Fortune 100 companies, banks, etc have hours of downtime with no apparent negative effects. It's a black swan event that's not an extinction level event, nor even close.
A bigger problem for the business is that if a black swan event that results in:
> Waiting 30 minutes for a machine to checksum 1TB of memory before loading the OS is painful in an outage scenario
turns into an ELE, then it's the business that has problems, not the systems. A business that is so fragile to downtime is not going to be in business much longer anyway.
In my toy barebones SQL database, I store rows alternatedly on different replicas based on a consistent hash. I also have a "create join" statement, this keeps join keys colocated.
Then when there is a join query issued, I can always join because the join keys are available and the join query can be executed on each replica and returned to the client to be aggregated.
I want building distributed high throughput systems to be easier and less error prone. I wonder if a mixture of scale up and scale out could be useful architecture.
You want minimum network round trips or crossovers between threads (synchronization cost) as you can get.
That's definitely on the cheap side as far as cloud goes.
It's gotten to the point where I could personally never justify going for the larger platforms outside of an enterprise setting - even if I'll run my own business, it'll probably be on one of the latter platforms, because they still have most of the features that I actually need, while being more affordable.
You might be missing the point.