Polycloud: S3-compatible object storage
crowdstorage.com
crowdstorage.com
Our website is right now being redesigned and your comments here are helpful to help us know where we need to improve!
For all we know the storage is coming compromised hosts that don't even know they're participating.
I answered the parent.
https://crowdstorage.com/faq/#where-is-my-data-stored
Where is my data stored?
Your data is intelligently encrypted on your My Cloud Home and sliced into tiny pieces that only you can put back together. The encrypted data pieces are scattered across our community of users, using small portions of our member’s unused hard drive space. Spreading your data out in this way makes it more safe and secure than it would be if it was all stored in one location.
Storage comes from services that we offer to end consumers. In exchange for our services they agree to let us use the unused storage on device, i.e. (https://crowdstorage.com/products/device-backup/). We plan on expanding to many more devices.
Storage also comes from being able to transparently leverage more traditional storage that we purchase from various providers.
We also purchase storage in datacenters.
What region in particular interests you?
How do you handle the problem of low interest? Do you preseed a region with storage servers so you have enough shards to store all the data until you have enough in-market storage endpoints?
I agree that the egress pricing seems a little disingenuous but model doesn't typically egress much more than 100%. Will take a closer look, when there's time.
We can seamlessly leverage more traditional storage as well as distributed endpoints. So in a way yes we can 'seed' it. We will likely wait until we have enough storage in a region before launching there.
> the egress pricing seems a little disingenuous
I have made small edits to the website to make it clearer it is 100% of data stored is free egress. This is definitely something we will focus on in the redesign.
> Pricing for Polycloud is only $0.004 per GB per month. There are no other fees as long as you do not egress more than 100% of your data during the month. If you do egress more than 100% of the amount of data you have stored then you will only be charged $0.01 per GB for any overages.
Looks like a hidden fee to me. Note that even the Pricing page [2] does not mention it, and the calculator does not allow to enter egress volume for calculation.
[1] https://crowdstorage.com/faq/#what-is-the-pricing-for-polycl... [2] https://crowdstorage.com/products/polycloud/pricing/
I'm not disagreeing with the point that that pricing is clear or not "hidden" but more that there's a use case here that can make sense.
Storing a 250GB backup = $1.
Sending to 2 machines 500GB - 250GB = $2.5.
Sending to 10 machines 2500GB - 250GB = $22.5.
How is storing this data so much cheaper than sending it? Especially given that it's stored redundantly?
I can't wrap my head around the fact that electrical signals are priced higher than equivalent HDDs.
Example, HGST 4TB MegaScale -- $60, or $13.46 per TB. Storing 4TB at 0.004 = $16. Transferring it 10 times would be $400.
Total $416, or $356 more than an actual physial hard disk.
Imagine getting a 4TB HDD, transferring the data to 10 computers and then throwing it away.
Being able to pay for excess egress is better at least than risking getting your contract killed by Wasabi if your egress exceeds your traffic.
They aren't doing 20x replication. They do 20+20? erasure coding, so the overhead is 2x or less.
> Being able to pay for excess egress is better at least than risking getting your contract killed by Wasabi if your egress exceeds your traffic.
You can upload dumb storage as needed in Wasabi. I assume you can do the same here.
With "sending that backup to 10 machines" I meant "egress 10x of 250GB" = 2500GB. With a pricing of $0.01 per GB after egress tops 100% of stored data, or 250GB, that's 0.01 * 2250 = $22.5. I'm not talking about replication overhead on their side.
Storing a 250GB backup and sending to 10 machines: $1 for storage, $22.5 for egress.
Storing a 250 GB backup, 2250GB of /dev/zero, and sending your backup to 10 machines: $10 for storage, $0 for egress.
It seems incentives also depend on the nitty gritty details around how you are billed which are not defined very clearly. Granularity and timing, whether you pay for part months of storage etc.
However all games aside it does seem like if your own egress were free (or much cheaper than 0.01) it would be most efficient to send your data directly to where it needed to go, yourself.
Then you'd only be paying $1 for the backup.
That said, it almost breaks even after the first month and pays off for itself a week into the second month. To eliminate this incentive, I'd suggest setting the cost of egress at (or below) the cost of storage.
It also would make more sense to me as a prospective customer that I'm not paying crazy overage fees, and I'm paying as if I reuploaded the same data and downloaded it an extra time.
I think this is standard with cloud storage. Check out this comparison [1], showing 2-6x higher costs for downloads versus monthly storage in B2, AWS, GCP, and Azure.
[1]: https://www.backblaze.com/b2/cloud-storage-pricing.html
Pricing page [1]
> We have no hidden fees and the monthly calculator to prove it.
> Egress $0.00
> No charge for egress, ops, or retrieval.
> With Polycloud from CrowdStorage, you only pay for the storage you need. And we never penalize you with fees for accessing your data.
> No egress charges
> We keep it simple—access your data when you want it, without being nickel-and-dimed with hidden fees.
> Hot storage for cold storage prices.
> Don’t overpay to get the speed you need. Polycloud delivers quick access you data whenever you need it.
> Egress $0.00
Signup page [2]
> Everything you need to store your data for only $4 per TB/mo and no hidden fees.
Terms of service [3]
> 6.1 Device Backup. In the event that you subscribe to a paid version of the Device Backup Services, we will put you on a recurring payment plan that charges you for the fees set forth at https://app.crowdstorage.com/pricing in advance for each billing cycle. We will charge the payment method you specify at the time of purchase and, if you do not cancel the Services prior to the end of the current billing cycle in accordance with Section 7, you will automatically be charged the then-current fee for the Services at the start of the following billing cycle.
> 6.2 Polycloud. With the Polycloud Services, you pay only for what you use. There are no set-up fees or commitments to begin using the Polycloud Services. At the end of each month, you will be charged for that month’s usage of the Polycloud Services as further set forth at https://polycloud.crowdstorage.com/pricing.
[1] Pricing: https://archive.is/K3Dyf [2] Signup: https://archive.is/c3fAf [3] Tos: https://archive.is/xBKJL
We are currently redesigning our website and this will definitely be something we focus on making more upfront and clearer.
How is it different than any other market? To bitcoin nodes? To Torrents, Tor network? Wikipedia depends on donations of random nerds.
Like this is such a self-depreciating perspective, that nothing can be done or relied upon unless its done by a megacorp.
To just fill you in, Coffee growers grow coffee because the cost of growing coffee is less than the local wholesale price (most of the time.) It is possible to subsist on the profits of growing coffee(depending on where you are).
The price per TB being charged to the consumer is $48 a year. which is significantly less than the initial setup cost to become a data host. (pi + sd card + hdd) That's before we get to the opex of paying for an ISP, (and any bandwidth overages) plus power and general maintenance.
Thats assuming that the company is selling at the price it pays the "hosts". I'm assuming they have some ambition for profitability.
which leads me back to the original statement: you're reliant on a bunch of randomers Who didn't really think about the economics subsidising this company.
The money is made on egress, that's the case with Storj and most others. For domestic setup, the data connection is a fixed cost you would have anyway.
Stop assuming everyone is an idiot, people do the math for profitability of these platforms, and switch between them.
> coffee farming is less profitable than data hosting.
yes, the money is in roasting. you're missing the essential point, there is a reason why coffee growers don't roast their own raw material, because they cannot get access to capital to get the equipment, let along access to the customers.
but coffee isn't the point.
Unless there is a sustained incentive to store the data, there will not be anyone willing to host it. the amount they charge is not enough to make it sustainable for people to invest in equipment to host the data, so you'll be reliant on best effort, or worse still, short term incentives that are unsustainable, leading to wide spread evaporation of storage.
You pay for gaurenteed storage, which is why its expensive, its a slice of the capex for the equipment, opex for the running and maintenance, plus profit. At this current price, it doesn't cover any of them.
If anyone on this site has built a business that relies on wikipedia having accurate information at all times I'd call them crazy too.
In the same way I wouldn't call this "enterprise" as they have plastered all over the site. Using spare capacity on a bunch of random usb drives that users happen to have online gives me no guarantee of uptime. With a 20+20 they're betting that 21 users won't experience an outage at the same time, and that if a large portion experiences outages that they can rebuild faster than users fail.
Without knowing anything about where the users are coming from, or what kind of contract they've agreed to, you're just giving a company your data that has told you: it'll be secure, TRUST ME!
There are thousands (millions?) of coffee farmers, so they have no problem.
Unless it’s copied to basically every computer I don’t see how eventually someone would have a corrupted file just because they are unable to piece it back together with available computers.
Also there are like 32 shards for each data, so 32 conpuetrs don't go offline in one day.
If the devices storing data go offline we constantly monitor and refresh the pieces to maintain integrity.
> The encrypted data pieces are scattered across our community of users, using small portions of our member’s unused hard drive space
So it's not really comparable to S3 or B2. It's more like storj or sia.
I can imagine doing it without controlling the storage layer, network topology and specially network latency is only harder.
Could you please elaborate a bit more on how do you achieve it?
There are other object storage systems that have strong consistency guarantees that came out after S3.
It greatly simplifies things that an object written to S3 it is immutable.
On a high level, all writes to your storage use some UUID. All reads use a consistent metadata storage (pick a modern database). After your write is complete and you are sure it is persisted, do the metadata update and return success. Everyone gets a consistent view of the operation.
- First, like you indicated is: you can take less margin (and believe me, there's still considerable margin).
- You can play with erasure coding policy. (n+10)/n makes that you can tolerate 10 failures. a higher n makes your storage overhead less. (which means more margin)
- you can use even fancier storage schemes (fe online codes will make your storage overhead something like 3%)
- you can use cheaper hardware.
- ...It could have be laid out more clearly, though.
A couple of questions I have right off the bat are:
How do we know how secure this solution is?
Even if it were incredibly well secured - what are the laws around this setup?
If someone stores illegal materials on this system who is responsible? (The person who stored it? The company? The unsuspecting host?)
What happens if the hosts lose interest or it fails commercially? Does the data get lost without warning when hosts start uninstalling the software?
The problems with these systems is almost never the technology - it’s finding a way to negotiate the millions of different implications storing information on other people’s behalf brings.
We are working on getting as much content up on the website as we can and telling more about us!
Have you looked into the legal aspects of your setup as regards liabilities? It probably doesn’t matter as much from the customer’s perspective as it’s their data so they should know what it contains, but would be worth knowing where this leaves you as a company and/or the network of nodes if some other customer pushed data onto the system that was objectionable.
The roach motel model would be considered making lemonaide by marketing types - not sure as a customer I'd readily agree :/
Anyway my advice is giving people something to de risk the cost of migration. IDK maybe an abstraction layer that automatically keeps a backup copy inside s3 glacier . Or something else ...
It scales up and handles PB of data as well.
So it'll double in price after that. And that's on top of having to round up to the next terabyte or two. But if you want more bandwidth then it looks like it's worth considering.
Side note: It's interesting that above 2TB the deal gets steadily worse.
I think maybe the deal looks like it gets worse because the RAM provided increases significantly.
Unfortunately, no. Users have to regularly back up the data themselves. " That sounds a little contradicting
[1] (Switch to 6*12 tb configuration) https://us.ovhcloud.com/bare-metal/advance/adv-stor-2/
Storage is a foundational service in the cloud. There are huge advantages to having that storage sit adjacent to everything else. Such storage as a service somewhere else doesn’t make a lot of sense in many/most use cases. Now you’re having to pay to move data across comparatively slow networks links to where it’s actually needed. It’s a catchy headline but when putting this to the test in real world scenarios those numbers don’t pan out.
This is the "data back up" use case, where you store heaps of data, and hope to never need to access it. For this use case the conditions seem excellent.
Why do you have that impression?
S3 and similar are used a lot for backups, especially Glacier.
Glacier can get pricey when you store/retrieve your data because of ops, retrieval fees, and egress (if going to the internet). We feel like immediate availability is a compelling advantage of our product.
I don't think this means Polycloud is GDRP compliant. It's my understanding the Privacy Shield Framework was struck down by the EU courts. Might be a bit careful here if you are a EU business.
You say you would use "Standard Data Protection Clauses" if the data goes outside of the US ... but I don't want my data in the US in the first place, so is this really a US only service? How would those clauses be implemented? Does every node have a contractual agreement with you? How would you know if someone took their computer with them on holiday to another country?
Also, how is the data encrypted? "State of the art encryption" is just marketing fluff :)
We plan on launching regions outside of the U.S. that only store data outside of the U.S. soon.
> Also, how is the data encrypted? "State of the art encryption" is just marketing fluff :)
AES256 and/or xsalsa20poly1305.
You can also use SSE-C to provide our servers with a key to encrypt the data (which our servers then promptly forget). https://docs.aws.amazon.com/AmazonS3/latest/userguide/Server...
How do you handle the data leaving the country? With someone just taking their machine on holiday etc
Which community is providing 300k devices? Is Polycloud building on top of IPFS or Sia?
There is a 'Contact Us' button at the bottom of the page you can always reach out.
Hetzner storagebboxes/shares are great value - but no s3 api.
$0.005/GB/Month for storage
$0.01/GB for egress
The only "hidden" cost I'd say they have is they have a limit on some API calls. For example their b2_download_file_by_name is limited to 2500 calls per day and then $0.004 per 10k calls after that.
I like the idea of this but do not yet see the competitive advantage.
Also your FAQ mentions your community of users. Is there a forum/discord where the community can discuss/learn?
AWS S3 Glacier is the same $0.004/GB/month and requires the same minimum of 90 days, but retrieval takes "from 1 minute to 12 hours". They also have large retrieval costs (on top of the regular egress to the internet) and API call charges.
AWS S3 Deep glacier is only $0.00099/GB/month, but requires it to be stored 180 days and also have operation/retrieval fees. "For long-term data archiving that is accessed once or twice in a year and can be restored within 12 hours"
https://aws.amazon.com/s3/pricing/
We feel like the immediate availability of our offering is compelling and a competitive advantage over glacier.
> Is there a forum/discord where the community can discuss/learn?
Not yet, we will have one created soon.
Select Bulk $0.001 per GB (Storage)
Select Bulk $0.0025 per GB (Transfer)