HNHacker News
TopNewBestAskShowJobs

super3

34 karma · joined June 11, 2014

[ my public key: https://keybase.io/super3; my proof: https://keybase.io/super3/sigs/zEOl7fqV4ajXDXMzCMtmMOOu9zM9V-tYk7S1B-5IuIw ]
submissionscomments
super3··on Ask HN: How would you store 10PB of data for your startup today?
If you put the data on Storj DCS, it would run about $40k/month for list pricing with global availability and encryption. I'm sure you could get a deal if you asked though. It has S3 compatibility, so would be plug and play with whatever you have now. Egress out of AWS would be free.

Way cheaper than AWS, and a lot less headache than trying to run it all yourself.

super3··on GitHub is degraded/down
There is discount pricing over 100 TB. Its a tech demo so Tadigrade.io sponsors it. Pretty powerful demo on easily storing large amounts of data especially when Github goes down.
super3··on GitHub is degraded/down
It is a tech demo of https://tardigrade.io, because easily storing all of Github certainly gets peoples attention.

Its only expensive because cloud providers overcharge. The listed prices on Tardigrade start at half of Amazon S3, and there are significant discounts if you are storing more than 100 TB.

super3··on GitHub is degraded/down
Have been increasing sync speed over time. It about double where it was a few weeks ago. It definitely does take some time to upload so much data.
super3··on GitHub is degraded/down
Should only be 2-3 PB. Storing it all on https://tardigrade.io
super3··on GitHub is degraded/down
Created https://gitbackup.org for this very reason.
super3··on Copyright Office Ruling Imposes Sweeping Right to Repair Reforms
This is a pretty big win for everyone.
super3··on MongoDB switches up its open source license
The total cloud market is around $180B. 90% are on running Linux, and 2/3 are running on OSS. If it is not in the billions its not enough.
super3··on MongoDB switches up its open source license
What really should happen is that the large cloud companies (really just Google, Amazon, and Azure) should be providing a portion of the revenue generated to the open source projects. The open source companies would make more features and drive more usage. Everybody wins. It makes no sense that the large cloud providers make billions off OSS, and don't give something more sustainable back. Classic tragedy of the commons.

Disclaimer: My company Storj is building a distributed Amazon S3 competitor and we are actually partnered with MongoDB. We share revenue with MongoDB for any customers they bring us.

super3··on Filecoin – A Cryptocurrency Operated File Storage Network
Filecoin whitepaper came out a few months before the Storj one. Filecoin however is not functional, while Storj has been functional for about 8 months.
super3··on Filecoin – A Cryptocurrency Operated File Storage Network
CEO of Storj here. You can sign up to use Storj and get the tools working to upload/download your files in 60 seconds. Last time I tried to use S3 it was 12 steps, and a phone call. Its faster, more secure, 50% cheaper than traditional cloud storage.

Basics are data stored for each. Storj: ~1400 TB, Sia: ~100 TB, Filecoin: Not Functional, Maidsafe: Not Functional.

super3··on Blockchain data storage startup Storj recieves $100k grant funding
While we are excited we got the grant I kinda agree. Our stuff like the whitepaper (https://storj.io/storj.pdf) or KFS (http://blog.storj.io/post/150825969108/introducing-kfs-a-loc...), allowing you to scale LevelDB past 100GB, are way cooler and more relevant to Hacker News.
super3··on Blockchain data storage startup Storj recieves $100k grant funding
27-30sec show the upload process and security, 45-51 sec down the download process and speed. The middle of the video describes the encryption process.

I disagree that its devoid of how does it work, but sure I could see someone wanted more info in the video. What do you suggest needs to be included?

super3··on Blockchain data storage startup Storj recieves $100k grant funding
How it works would be at the bottom of the page labeled whitepaper (https://storj.io/storj.pdf), or you could have clicked on the link labeled Github (https://github.com/Storj/) to look at the code.

Storj is a direct competitor to Amazon S3. We don't have to educate the public, just the developers that need to store data somewhere.

super3··on Blockchain data storage startup Storj recieves $100k grant funding
Potential for more more follow on funding. Certainly doesn't hurt.
super3··on Blockchain data storage startup Storj recieves $100k grant funding
Don't they only store 50 TB of data to Storj's 2,000 TB?
super3··on How Storj increases object storage security exponentially
Thanks for joining in. Have you tried storing data or renting any of your space?
super3··on Storj Crowdsale
Check out some of the numbers before your decide anything: http://blog.storj.io/post/88303701698/storj-vs-dropbox-cost
super3··on Storj Crowdsale
Yeah I came across that a few weeks ago. Glad to know you are making progress. Can you contact me via hello@storj.io, and lets figure out some ways we can work directly together. Storj doesn't believe in competition, we instead want to find ways to collaborate with those in the decentralized data space.
super3··on Storj Crowdsale
So Metadisk is out, and DriveMiner is on its way. As far as technical contributions, going to have to wait until I finish up the revised whitepaper.

We were focused on writing the code first, and now we are in the process of writing down what we learned. Don't take me at my word, but do give me a few weeks to prepare a more formal paper now that most of our core code has been implemented or speced.

As far as the $6M we don't intend to raise that much. What is left over will be distributed through the early mining, and future crowdsales for other platforms build on top of Storj(decentralized Youtube, Imgur, etc.) In that way we can give the network a kickstart with the actual resources that it needs to survive.

super3··on Storj Crowdsale
This exactly. We also wanted to separate the payment protocol from the data protocol.

If sidechains or treechains become viable we plan to merge it back in to Bitcoin proper.

super3··on Storj Crowdsale
You would have to take a look through our code on githhub. http://github.com/storj

That paper was written before decentralized storage was really just a new idea. I've since implemented a baseline core algorithm, and we have one application running.

I'm in the process of rewriting the whitepaper. Hopefully will put out a draft in two weeks(I'll be at conferences all this week). From there I want to actually collect network data(after all I'm trained as a researcher and what is good paper without any data?) before putting out a solid draft to be peer reviewed.

super3··on Storj – Blockchain-based decentralized storage network
Whitepaper briefly goes over this. The user or application software get to decide their replication number N. The network is self healing, as in if a portion of a file misses a heartbeat(check in) the network will recover it from the other peices.

To affect a file large amount of nodes would have to go down simultaneously. Again the user has control over this N = 1 - Data failure N = 3 - Good for just average cases. Probably still Sybil attacks possible if the have 20-30% of the network. N = 100 - Are you expecting tactical nuclear strikes or acts of God?

Cool thing is if you increase redundancy to a large number Storj starts to act like a CDN or torrent network. But thats outside the scope of your question.

super3··on Storj – Blockchain-based decentralized storage network
Data is not stored on the blockchain in our system. Simply "file pointers".
super3··on Storj – Blockchain-based decentralized storage network
I'd take a look at some of my writings for meta transactions and templates here: https://github.com/Storj/Metadisk/wiki

Bitcoin only supports up to 80 byte of data per transaction, our model uses 1 kb or more, which makes unusable for our current platform. We are not creating another cryptocurrency just for the sake of doing so. There are many technical reasons why we can't do this yet.

So we already have a long term plan to implement sidechains or treechains. Currently this is only concept/vaporware, so we are not waiting around until it happens.

super3··on Storj – Blockchain-based decentralized storage network
Already anticipated that. We can just run the Storj client on your traditional cloud VPS. If there is market need those can be booted up in a couple of seconds by our team or anyone who wants to set that up.

Because VPS providers profit margins are so high vs a typical home user, they will be eventually replaced, but it allows Storj to scale pretty quickly using the traditional cloud as network nodes.

super3··on Storj – Blockchain-based decentralized storage network
Note the whitepaper is outdated. We are in the process of rewriting it. Its a better description of Metadisk, which is one of our front end web apps.

The metadata for the file is stored in a blockchain. The actually data may live on a centralized or decentralized source.