My own company struggles with that - throwing more storage on the live db is easy, so we've kept doing that for years, but pushing around multi-terabyte backups is getting cumbersome and we're going to have to slim down the data in prod even at the cost of engineer effort.
20TB (~18GB usable) drive is $370 + whatever power it will use
6 of them in RAID6 will give you 72TB backup storage, add extra copy and say extra drive as spare and that's 13*370 = $4810 USD for ~5 years + whatever power it will use, ~50-55W if idling most of the time and just writing backup once a day
I think people got completely disconnected from how it actually costs to run something because of vastly higher resell value of storage and bandwidth in the cloud.
> My own company struggles with that - throwing more storage on the live db is easy, so we've kept doing that for years, but pushing around multi-terabyte backups is getting cumbersome and we're going to have to slim down the data in prod even at the cost of engineer effort.
...but that's definitely a bigger part of that issue, cloud or not. Backup is slow, restore is slow, anything that involves any of the two is also slow
Anyway Segate Exos SAS 20TB drive is $409, you can re-do math for "true enterprise" grade hardware. "Enterprise" drive from SAN vendor will be a normal hard drive with some bits in firmware changed so SAN can reject non-SAN-vendor rebrands. They die just the same, we had stacks of them to prove it.
Hell, some of our Intel enterprise NVMes died suspiciously quickly compared to other but that might be a fluke...
Engineering hours: there's a good chance you pay for that once, and that solves the problem.
10 SSDs: will require rack space, electricity, PCIe slots, timely replacement, management software... most of these expenses will be recurring expenses. If done once, sometimes the existing infrastructure can amortize these expenses (i.e. you might have already had empty space in a rack, you might have already had spare PCIe slots, etc.), but amortization will only work in small numbers.
Another aspect of this trade-off: systems inevitably lose performance per unit of equipment as they grow due to management expenses and increased latency. So, if you keep solving problems by growing the system, overall, the system will become more and more "sluggish" until it becomes unserviceable.
On the other hand, solutions which minimize the number of system resources necessary to accomplish a task increase overall performance per unit. In other words, create a higher-quality system, which is an asset in its own right.
> 10 SSDs: will require rack space, electricity, PCIe slots, timely replacement, management software... most of these expenses will be recurring expenses. If done once, sometimes the existing infrastructure can amortize these expenses (i.e. you might have already had empty space in a rack, you might have already had spare PCIe slots, etc.), but amortization will only work in small numbers.
10NVMe SSDs fit into 1U.
The cloud always comes out terrible on hardware. The cloud is not selling hardware, it's terrible to buy any hardware in cloud, it sells spike capacity, and it sells software running on it.
There are many services in cloud that would be a lot of work to create (even if you need just small part of functionality) on-site but if you don't use it and just need to throw raw hardware at the problem consistently, a rack with hardware will be far cheaper solution even after incurring the engineering cost to deploy and automate it.
We have 7 racks and time to manage hardware is some minuscule fraction, and time to manage automation on it is still small part of our time, and we're just ops team of 3.
$1000 doesn't buy you 800GB more of the RAM that your server likely takes, which is likely to run $3-6/GB ($3 for uncertified, $6 for motherboard manufacturer certified RAM), plus $0.50-$1/GB for the DIMM slots over and above your baseline RAM config.
It can still sometimes be smart to "throw hardware at it", but I don't think you can go from 128 GB of RAM today to 1 TB of RAM tomorrow for $1K in most cases.
sounds incredibly cheap
In my proximity I don't know anyone who is doing any sort of Engineering work, be it dev/sysadmin/ML/datascience/whatever. I'm so astonished by the numbers of 250$/hour of fulltime job (consulting likely a different story).
I'm very much curious, what kind of Engineering role that could be, in which industry/country/company, if you can name a few and/or share some other info.
I know good SREs (fancy sysadmin plus some devops, but with more class) on the scale from around $120k/yr up to about $300k/yr. This is in the USA. Everyone I know started in SF but since the COVID diaspora, they are all around the US now.
$250/hr estimate -> $125/hr salary -> roughly $250k/yr
My estimate might have been a big high, but really what I'm trying to drive home is that people are almost always more expensive than hardware.
SAN Flash controllers are hideously expensive because they need to push tens/hundreds worth of SSDs to tens of servers.
If you can solve redundancy other way (say via pair of replicated databases), a pair of servers with local storage will be far more $ effective
I know the corporate idea of "everything on SAN, nothing outside of SAN" very well, it's just plainly solving political problems, not techncial ones.
One major benefit that SAN-backed database storage provides us is, when combined with VMs, enables us to spin up another database instance against already existing data in the SAN (i.e. a staging DB that looks at prod data).
Why $1000 for four hours?
Does it cost your company $150,000 per month per engineer?