Hacker News Reader

Comment by brownbat | Hacker News Reader

I remember a post here a while back from a guy running Bitcoin mining in his dorm room. One day he realized he could offer his spare cycles to grad students with high computing workloads and undercut cloud computing prices while increasing what he made far over just mining.

It's very weird that we haven't fully arbitraged $/instruction to a single (low) price yet (or storage/hosting, whichever).

If only there were an Uber for unused cycles or storage... let everyone turn their unused capacity into mini AWSs with a common interface and safety and reliability guarantees.

Maybe there are too many barriers, like security.

View on HN

Then he got tracked down by the dorm and expelled, because him or his customers were running bitcoin miners. Turns out the dorm has to cover the electricity and that's not a supported use case for them or the dorm insurance.

> If only there were an Uber for unused cycles or storage... let everyone turn their unused capacity into mini AWSs with a common interface and safety and reliability guarantees

This is exactly what Sia.tech is building for storage. A competitive marketplace for storage providers where anyone can set up a host and start selling their spare storage space. It's not completely finished yet, but it's pretty close.

A bunch of companies are already building products on top of it (including me)

how much replication do you need to make this kind of thing reliable? since anyone can pull storage at any time. does that then make transactions super slow? any numbers you can put on this would be helpful. also, whats your product? (just curious)

Sia uses reed-solomon encoding to spread files over 30 hosts with a redundancy of 3x. So Any 20 hosts can fail at the same time and the files would still be available. This does mean that you need to upload all your data 3 times. Sia also needs to monitor your uploaded files all the time to make sure the hosts behave. This means hosts periodically run checksums on the stored data to prove to the client that the data is still there.

The team is currently working hard on performance upgrades. On regular consumer hardware you can currently upload / download data at a rate of about 500 Mbps. The next release is expected to improve this significantly.

Here's an introduction article which explains how Sia works with a bit more depth: https://support.sia.tech/article/dk91b0eibc-welcome-to-sia

My own product is a file sharing website: https://pixeldrain.com. It uses Sia to store large files because it's cheaper than conventional cloud storage. I plan to make it possible to download directly from Sia hosts as well so I can save on bandwidth costs too.

awesome :) i was able to upload a big file and immediately share it and view it in browser! very nice! what pdf viewer component did you use? i also like the pastebin functionality.

im sure you hear this a lot but... has anyone done a heads up comparison with filecoin?

Thanks, glad you like it. I use Mozilla's pdf.js. It's super simple to implement, just load it in an iframe with the path to the PDF file in an URL parameter. Et voilà, a PDF viewer.

Comparing with Filecoin is hard because there's not much information available about it. The rollout keeps getting delayed too. I know that the founder of Sia has criticized Filecoin's whitepaper a few times because it contains unsolved problems which could cause significant issues during the rollout of the network. Sia took a more conservative approach and worked out all the math before the development of the network started in 2015. Now, 5 years later, Sia has solved all the fundamental issues with the protocols and such and are working on upgrading the performance and building apps on top of Sia's core protocols. In terms of development Sia is about 3 years ahead of Filecoin.

awesome. thanks for your generous replies!!

will spread the word about Sia and Pixeldrain when the topic comes up :)

This is really interesting!

How does Sia prevent hosts from precomputing the checksums to fake they are behaving but erasing the data itself? Does it checksum over random ranges of data?

Which source does it use for entropy so that the network remains distributed but nodes can't predict the ranges? Does it use the last block nonce?

Which checksum algorithm does it use? Is care taken as to not be vulnerable to prepend or append attacks from hosts who intend to host data partially whilst pretending they are hosting full data?

I'm not entirely sure on the specifics of storage proofs, but as far as I know it's something along these lines:

When uploading data the renter (that's what we call the node which pays for storage) computes a merkle tree of the data which the host should be storing. When a contract is nearing its end the host will enter a proof window of 144 blocks (1 full day) in which it will need to prove that it is storing the renter's data. The proof is probably based on the block hash of the block where the window started. The host stores the proof in the blockchain and the renter will be able to see the transaction. If the proof matches the merkle tree (which the renter has stored) the contract will end and the host will receive the payment and their collateral back. If the proof is invalid or was not submitted at all the renter can cancel the contract which destroys the funds in it. The host won't get paid and loses its collateral, but the renter also won't get their money back (to discourage the renter from playing foul)

There is some more info on this on the wiki: https://siawiki.tech/about/trustlessness and the website: https://sia.tech/technology. And here is some incomplete technical documentation: https://gitlab.com/NebulousLabs/Sia/-/blob/master/doc/Resour...

If you want to go more in-depth you can go on our Discord where lots of developers hang out, eager to help others to get started with the network :) https://discordapp.com/invite/sia

EDIT: The whitepaper is of course the best source of knowledge. It's quite old at this point but the core principles still apply https://sia.tech/sia.pdf

Sia founder here. The hashing algorithm we use is blake2b. Definitely secure.

We do probabilistic proofs, so we have the host provide us a small random sampling of actual data (so the host can't rely on precomputing), plus a proof that this actual data is what the contract says the host should be storing.

See chapter 5: https://sia.tech/sia.pdf

Remember that the "cost per instruction" ought to be very different depending on a lot of other factors, including but not limited to:

- how much data do you need to move to actually perform the computation

- whether the computation performance is expected to be reliable or if best-effort is acceptable

- whether there are confidentiality and accuracy requirements on the input and output of the computation.

Most software engineering teams are not working at a granular enough level to properly describe which parts of their computations and data are expected to be reliable or not (in availability, confidentiality and integrity). However this can impact the cost per instruction of a computation by multiple orders of magnitude.

> If only there were an Uber for unused cycles or storage... let everyone turn their unused capacity into mini AWSs with a common interface and safety and reliability guarantees.

Funny, this was our idea with first startup 18 years ago, federation of unused storage and redistribution. There were no takers as no one wanted to contribute their unused storage but wanted others. It was much easier to centrally locate our own storage and allocate to users who required it.

Reply on news.ycombinator.com