Comment by walrus01 | Hacker News Reader

Comment by walrus01 | Hacker News Reader

I work in telecom/datacenter infrastructure and this is fanciful. The whole way they take the wattage load of one machine and then hand wave away all of the rest of the costs of either building and running a datacenter, or paying ongoing monthly colocation costs... Is just scary. I truly don't mean to offend anyone but this looks like a bunch of enthusiastic dilettantes.

Generators?

UPS?

Cooling costs?

Square footage costs for the real estate itself?

Security and staffing?

At the scale they intend to accomplish they will need at minimum several hundred kilowatts of datacenter space. Even assuming somewhere with a very low kWh cost of electricity, that much space for bare metal things isn't cheap. Go price a lot of square footage and 300kW of equipment load in Quincy, WA or anywhere else comparable, the monthly recurring dollar figure will be quite high.

And all of that is before you even start to look into network costs to build a serious IP network and interconnect with transits and peers.

They're not talking about a datacenter. Datacenters need to be reliable. Sia storage pools don't, because security and reliability is achieved at the global network level, not at the level of individual systems or storage pools. 95% reliability means you can be down for two whole weeks out of every year and still be well within acceptable uptime requirements.

Generators? Who needs those? Just wait for the power to come back on. UPS? Why bother? Square footage? Stick some wooden shelves in the cheapest building possible. Cooling? Locate in a cold climate and buy some window fans.

This isn't anything like the sort of infrastructure you're used to dealing with. Think Bitcoin mining farm, not Backblaze datacenter. Any corners that can be cut will be.

Who are the customers for such unreliable systems?

Sia is very reliable from the customer's perspective. Its only individual systems and storage pools that have lax uptime requirements. Thanks to some clever network-level redundancy mechanisms (10-of-30 redundancy), 95% uptime at the storage pool level translates to 99.9999% uptime from the customer's perspective. See the "Uptime Math" section in the OP for details.

It is not the answer to the question above. What is the type of customer uses this company?

Users of Sia are the customers. So your question reduces to: who uses Sia? It used to mostly be interesting for customers interested in reliable large-file data backup on the cheap, but now it has expanded to customers looking to do content distribution on the web, etc.

When you says Sia is reliable, do you mean it could be reliable if hypothetical X, Y, and Z things happen?

Because according to the homepage (sia.tech) there are only 895 hosts storing a total of 206TB right now, which is a very, very small amount. Backblaze, a relatively small player as compared to the big cloud providers has 1.1 million TB of raw capacity as of last year (redundancy reduces the available capacity, but still) [0].

[0] https://www.backblaze.com/blog/hard-drive-stats-for-2019/

By reliable in this context I mean robust against hardware failure. (Including the failure of entire storage sites.) The OP explains the math and associated assumptions for how they derived that "99.9999%" figure, and acknowledges that since the calculated chance of data loss due to hardware failure is so infinitesimally small, other failure modes outside of what they modeled are likely to dominate.

As for the relatively small number of hosts at present, 895 is more than enough for 10-of-30 redundancy to work "as advertised". You really only need 30 hosts technically. The bigger issue I think is the relative immaturity of the software. Sia is still pretty new compared to most other data storage systems; and although I've never heard of any software bugs in Sia resulting in data loss, that doesn't mean such a bug will never be discovered. Be cautious, keep backups, and never rely on any single storage medium to store your data.

> Think Bitcoin mining farm, not Backblaze datacenter. Any corners that can be cut will be.

And yet Sia is about half the cost of Backblaze (i.e. not much savings).

Hard to imagine situations where this is a good trade-off.

Its super interesting to dive into the world of cryptocurrency mining, where some DCs are getting PUE of 1.1 or better with a buildout that's basically amounts to shelves and box fans.

No generators, just eat the downtime. No batteries. No 24/7 staff. No racks, just shelves (folded sheet metal is cheap). Security varies from farm to farm.

These servers don't need to run cool, as long as you are in a climate that doesn't get over 100 degrees you can get away with fans and no AC.

If all you intend to do is generate hashes and run for the exits when your mining equipment is no longer economical, due to difficulty increase, sure.

Reliable data storage for paying customers is a very different thing.

Per-site reliability (beyond a stated/assumed 95%, which is right around the "I put a computer on my desk" level) isn't a design goal, though. You can argue with their math or their assumptions, but you can't say they're wrong for not designing in the reliability features you list above.

Things on my desk are meeting three nines or better of uptime at present, 95% would be 18 days hard down per year.

I think it very much depends on the desk. I mean, power alone at my home just barely reaches three nines. Internet connectivity glitches out occasionally. I'll fat finger configurations and reinstall stuff for some more hours a year. It all adds up.

You're right that you can take the same hardware, add a $70 UPS and some thought and care, and do much, much better. But my point was only that "95%" is a trivially achievable goal even in the most naive setups, which makes me treat their analysis with a little more credence than most folks here.

Which should give you even more confidence in Sia's reliability, no?

Crypto currency servers didn’t need to last though. Their primary components were being replaced regularly with the latest generations.

Here they’re depreciating components over 7 years, so will those components last 7 years under those conditions?

I think it's interesting to dive into the economics of 95% uptime. Maybe you don't need full datacenter cooling; you can just locate in a cool climate and have a fan in the window blowing cold air in. If there's a blizzard, you lose your drives because snow blows in and melts on your drives. If the chance of the blizzard * price of drives is less than needing 750W of cooling, then you win. Yeah, sometimes everything shorts out.

Power is similar. Maybe you just use solar and turn the drives off when it's cloudy. With enough distribution throughout the world, it will probably be sunny somewhere.

I haven't done the math and I'm not saying it will work out favorably. I also don't have a use case for 95% availability. (That's two weeks a year where your data is gone!) But it's something that someone with the right needs could consider, and maybe come out ahead of someone shooting for 5 nines and drives that aren't covered in snow.

OVH, the massive hosting company, is located in a former aluminum smelter next to a hydroelectric dam in Quebec. They keep costs as low as absolutely possible, but their operating costs as a whole are still considerably higher than something designed with one nine of uptime.

I get that you're mostly joking by saying "yeah sometimes everything shorts out", but we have electrical codes in the US/Canada for a reason.

In most places you don't need to build to code, as long as you take other steps to mitigate risk.

Saying "this entire building is designed to catch fire, and if it does, it won't do harm to neighbouring buildings, people, or the environment" is probably a good start.

If you're in the developed world, that isn't going to fly. "other buildings won't burn down" isn't going to get you a pass on these things.

Insurance?

> I also don't have a use case for 95% availability. (That's two weeks a year where your data is gone!)

Gone from only one of the places it's stored. Your data is still available even if just 10 out of 30 servers are online.

They're not talking about a real datacenter; they're talking about a deathtrap crypto mine with hard disks instead of GPUs/ASICs. Can't you run a million hard disks off a consumer cable modem? ;-)

At the (slow) rate Sia is growing, I don't think there will ever be enough demand to justify this design anyway.

How much is a terabyte of dropbox storage?

0. You can only get 2 TB+

Dropbox is $10/TB/mo, which is the "old" industry standard (and still what most big guys charge). Backblaze is $5, but they charge egress fees, so usage matters. Dropbox doesn't charge for egress, but there are limits on "Public" folder bandwidth (and you can't pay for more if you max out). I don't think there are limits on private content, but I bet there's a soft throttle or some limit that will get you contacted.

There aren't a lot of options for <$5/TB and they all charge egress. I've tried Backblaze, pCloud, and DO Spaces. My specific use case is storing full HD movies for streaming via Plex, which requires a fairly reliable 6Gbit/second, and most of them can't keep up. DO Spaces did the best of all the ones I tried, so I'm using that for now, but it's just skating the edge of usability.

Surely you mean Mbit, unless you’re streaming uncompressed video for some reason! :)

I'd love to see someone try running a TV network off of DO spaces :)

Indeed. HD streaming seems to need about 5.4Mbps reliably. Spaces is working well enough, but sometimes I have to pause for ten minutes to buffer enough for the rest of the film. It usually works, but with variable bitrate it must spike at times.

Although it's certainly a typo, ~6 Mbits a second isn't enough for more than poor quality 1080p video, roughly equivalent to streaming from Netflix or Hulu. If you were streaming a backup of a UHD Bluray, for example, the bitrate could be over 100 Mbps.

Did you try GSuite Business account with 5 accounts? Google offers unlimited cloud storage if you get more than 5 accounts (5*12 = 60 USD/month and no egress)

It's 'unlimited' until you use it too much, and then all your google accounts will get perma-banned...

I've read this as well, so I didn't try. I also don't want to spend that much. I have 120 movies in Spaces and it only costs me about $8/mo. My egress is under the free tier so far, and overages amount to about four cents per movie stream.

Any evidence of that? Isn't it likely that Google has plenty of commercial customers using this service with large amounts of data?

I should add that places like Backblaze and pCloud and DO Spaces and others charge per GB stored, so if you're using 1.3TB it's prorated accordingly.

For $6 per month (paid yearly) you get 1tb of OneDrive + office (desktop applications). My math maybe off but if you buy office every 3 years for $200 then you're already even.... And 1TB of storage is just free really.

You could also say, MS Office is virtually free for the 1 TB these days.

It sounds like they're intentionally forgoing backup power and UPSs, as well as 24/7 staffing. However, you're right that this is probably pretty optimistic.

If you’re paying more than $300k/mo for 300kw of power in Quincy the DC sales guy probably bought your purchasing people a boat.

I don’t really agree. Retail Datacenter pricing all in should be well under $500/kW; half of what they suggested.

Reply on news.ycombinator.com