Hacker News Reader

Comment by redm | Hacker News Reader

I launched MediaFire as an unlimited storage provider in 2006.

While it is true that data accumulates as time goes on, it's not as big of an issue as described. Older data tends to be smaller, for example, photos were lower resolution. That is compounded by the fact that storage is getting cheaper. For example, the price for a 500GB drive in 2006 is the same as a 14TB drive price today. If your getting the same subscription dollars, it suddenly becomes cheaper, every year, to store legacy data.

In my findings, the reason unlimited doesn't work is a small percentage of users (very small, like ~0.1%) who push it to the extreme. In other words, if your exhaust 50% of your resources to service 0.1% of your users, at a fixed subscription cost, the model doesn't work.

View on HN

I've long suspected that r/DataHoarder is why we can't have nice things, i.e. cheap online data storage that is slightly subsidized by the average user who only stores a little. Thanks for confirming from the point of view of a major storage provider.

When stock brokers were charging commissions this was their model too.

They would take the user profile of most of the users that gravitated to their specific sales pitch, and make a commission structure that worked for them and let that subsidize the outliers that traded different products like high volumes of sub-penny stocks. (There are some fixed costs to stock and options trading, which are now subsidized by selling data instead of charging commissions)

I follow r/Datahoarder for best practices and methods for getting my own stuff (like HumbleBundle or purchased eBook libraries) to my own local storage but, some of them really abuse online storage providers. I cringe sometimes reading all the posts.

Why on the world someone try to mirror nearly everything to somewhere like Scrat from the Ice Age?

It’s mainly because they can, but I also bet it’s that feeling that you’re “cheating” the system (like why people cheat in online video games)

https://en.wikipedia.org/wiki/Compulsive_hoarding

Some people even hunt for "Unlimited for life" early adopter plans and grab them, then abuse them.

One of the backup services imposed a 12TB limit. Evernote reverted their unlimited data storage pledge, Google retired unlimited data in GSuite. I cannot say all of them due to hoarders but, Evernote CEO openly said that "People tried to zip and upload their backups to our servers. We're not that big".

>Google retired unlimited data in GSuite

They actually did not. The $12/user/month plan with an unenforced data limit was rebranded as the $20/month Enterprise Standard plan with unlimited storage explicitly allowed. Existing users are currently grandfathered in at the $12 price but may be forced to "upgrade" at some point in the future.

"Customers that have 5 or more End Users will receive a total amount of Google Drive storage equal to 5TB times the number of End Users, with more storage available at Google's discretion upon reasonable request to Google." is not very comforting language.

I think you're looking at the Business plan. Enterprise is separate. You can't get an Enterprise plan without contacting Sales unless you're grandfathered in.

https://workspace.google.com/terms/user_features.html

I'm looking at Enterprise.

> You can't get an Enterprise plan without contacting Sales unless you're grandfathered in.

Having to contact sales and prove myself is not exactly the same as being retired, but it's pretty similar in a lot of ways.

Oh I see. So it's actually the same as the old plan (unenforced storage limit), but without the (also unenforced) minimum of 5 paying users.

I also just read that you can in fact get an Enterprise plan without contacting Sales by buying a Business plan and then upgrading through the admin console.

So maybe less retired, more downgraded.

I understand and while we all understand that it isn't technically "unlimited", they still are just using the service they pay for in the way it was advertised to them. If it isn't unlimited and you don't want to deal with that 0.1% of users, do not advertise it that way.

Again, I think it's ridiculous to use online storage services in that way but still, is it really fair to blame the consumer here?

I use some of the techniques as offsite "well, maybe it's with that cloud storage?" disaster recovery backup options.

I have used Backblaze, and was storing 16TB there, and that was $50/month.

I did a simple envelope calculation, decided I could either hand-roll a offsite system with 2x consumer harddrives, or look for other options.

I have 28TB backed up with another provider now.

Why haven't I tried either AWS or Azure cold-tier storage?

I have maybe 2 hours a month where I can spend looking into solutions to my offsite backup situation, and I am somewhat concerned that I could mess up the AWS solution and cost myself thousands of dollars. This coupled with a lack of time, significant competing concerns (keeping a job during a global pandemic job market; work-load; personal life), and "eh, I'll get to it"-laziness, means I have to put this on the back burner.

If you handed me a simple solution using rclone, zfs send, btrfs-send, or a pre-canned "this is what it would cost" Bacula approach, I would move to it.

> r/DataHoarder

To be honest, 99% of the posts there are about building your own NAS.

And I don't fully agree with the "we can't have nice things" narrative in the Google Photos case. How bad was Google hurt by the actions of these few "power users"? Before that, how much did they profit by driving out competition with their "unlimited" plan? (Which, by the way, was neatly integrated and advertised into their own mobile OS...)

The uber strategy (speaking of ponzi scheme...).

99% of the users of unlimited plans are only storing a smalll amount of data

I don't really get the gnashing of teeth about Google withdrawing the "Unlimited". Google never promised anyone "Unlimited" anything, the details of the contract always said "we reserve the right to modify this in the future".

People who aren't data hoarders, who contracted with Google in the spirit of the contract ("more storage than you will need"), are still getting what they signed up for. People who contracted with Google not in the spirit of the contract ("Haha, unlimited, huh?") are also getting exactly what they signed up for - the letter, rather than the spirit of the contract. Google is keeping the spirit of the contract with people who took them up on the spirit of the contract, and Google is keeping the letter of the contract with people who took them up on the letter of the contract. I cannot see any fault in this.

I don’t think it’s them, I think it’s the people at r/plex that are using google drive as their data store and people bragging about having 100s of tbs of content.

I doubt that is so much of an issue as most of the content will be deduplicated across users.

Standard practice is to encrypt media stored on cloud storage to avoid DMCA fuckery.

That tendency to become cheaper over time is known as Kryder’s law and it’s held up well with some notable jolts like the 2014 floods in Thailand:

https://blog.dshr.org/2020/11/storage-media-update.html

I think you’re right on with the extremes - even outside of paid services I’ve seen that generally hold true where a small percentage of your users are responsible for most of your storage growth. Setting a generous base tier avoids a lot of micromanagement for people who are never going to be major consumers.

> In my findings, the reason unlimited doesn't work is a small percentage of users (very small, like ~0.1%) who push it to the extreme. In other words, if your exhaust 50% of your resources to service 0.1% of your users, at a fixed subscription cost, the model doesn't work.

How do you handle such users at MediaFire?

It appears MediaFire is no longer unlimited

There's nothing you can do other than move away from unlimited storage. The model, as the OP states, works well initially, however over time, "power users" emerge to want to make use of unlimited. The only choice is to remove unlimited storage as a fixed subscription fee. You may have metered billing (unlimited storage but you are charged per GB) or to do away with unlimited storage all together.

Keep in mind, that change is great for the business and 99.9% of the users. The business saves money on resources, and those resources are re-invested into the other 99.9% of users.

Edit: grammatical error

Marketing wise, it's better for the business to market with "unlimited" than creating a limit, even if it won't affect 90% of the users. For users, it creates a different feeling of freedom, even if you never use that freedom.

If you still want to play that game then restrict the service in other ways. You could for example throttle user upload to 10mbit/s if they have uploaded more than 500GB per month. That way no single user can practically upload more than 3.5TB per month. The 2 petabyte abuse scenario would have been avoided while still being able to brag about some users storing 50TB of data on your platform.

A side note on this:

I always wondered when there'll be a lawsuit to this effect (the thing that originally prompted this thought was one of the carrier networks throttling download speed after some amount of data).

In the extreme case, suppose you halve the upload speed after each chunk of size x. Then the amount of time to upload n*x amount of data will be 1+2+4+8+...+2^(n-1) units of time, which obviously diverges. As such, it's not unlimited storage even in theory. I know this is the point of such approaches, but I'm curious why there aren't more lawsuits to this effect.

Similarly when you get e.g. 10GB of data on cellular networks and are charged for each subsequent block, there's really no way to use your 10GB. You either have to stop just before you hit the limit, leaving some data on the table or you have to go over and pay for another block. Ideally the networks should use up all the data and then ask you if you want to purchase another block. Otherwise there's no way you can actually use exactly what you ostensibly pay for.

I think you missed that the speed limit would affect uploads after transferring 500G _per month_, not total? So in a year you would be able to store 6 terabytes. This would probably work out fine, because e.g. typical use of cloud storage is probably backups, and the first uploads are the largest.

I think some networks have operated in the way you want: you get full speed to some specified limit, and after that you are throttled (significantly). That works out fine to use the complete limit. On the other hand if you do want to use 15G, it's easier to do, when the block gets charged automatically.

I think it would be very hard to have a lawsuit about this if the contract includes these details.

You're still calling it unlimited while placing a limit. You've been clever about how you impose the limit, but it's still there.

Unlimited storage means just that. They never said infinite inbound data transfer speeds. "If you can get it to us, we'll store it" is totally reasonable.

>"If you can get it to us, we'll store it" is totally reasonable."

No it isn't. On the one hand you say we'll store it if you can get it to us, then on the other you build an obstruction to data getting to you. Data propagation is part of storage. I have to get bits off my processor, to your physical media to be stored, and you just built a barrier to keep that from happening. You don't get to use "unlimited" for marketing attractiveness, then bait-and-switch.

So maybe to you it sounds reasonable, but to anyone that values being up front about the service they are offering, that's nothing more than selling a product you can't deliver, i.e., deception and false advertising.

I have unlimited data transfer on my home internet connection. Is my ISP pulling a "bait and switch" because my internet bandwidth isn't also unlimited? Ignoring the whole "physically impossible" thing, of course.

The entire universe is a bait and switch using that sort of logic.

If the provider deliberately reduced speeds to ensure that you would never be able to get past a certain amount, then it's a cap. A naturally occurring limit wouldn't bother me, but that's not what we're discussing here.

Yes. Your ISP is. Because as one gentleman found out when doing terabytes worth of off hours remote backups, your ISP is more than happy to come back and bite you if you actually put those contractual terms to the test.

The fact is, every last ISP has been caught extracting higher fees from users for purportedly unlimited plans, to which the user was given the understanding they werre paying to not be throttled then hiding behind weasel worded contractual clauses to avoid culpability so as to not be held to task by having to improve their network infrastructure to handle actual traffic needs.

I'm more than versed in networking. I may not work backbone services, but I'm aware of the foundations of it, and I've gone out of my way to get familiar with the business environments and models around it.

Yes it's all statistical multiplexing. Yes. QoS is a purportedly necessary scourge to everyone in the modern day. Why? Because we allow ISP's and connectivity providers to sit back and provide lackluster links, overpromise what's being delivered, and wiggle off the hook.

My assertion is you shouldn't be able to market unlimited. All it does is create a misleading and unmeasurable relationship where no one can be held to account, and the status of the "relationship" between the customer and provider is opaque to the customer.

Yeah, I know, it just reminded me of this general thought I've had for a while (that being said, it's still limited once you throttle it so the same argument would apply.)

It’s still a finite time to upload for any given data size.

It’s not like uploads have infinite bandwidth in the absence of these limits.

There are networks that let you do that. GiffGaff in the UK is one. They're even fairly generous in giving you a couple of extra GB for free if you only go over occasionally.

Of course it's better marketing wise to market your products/offerings/capabilities as better than they actually are. Even if 90% of users don't notice the difference, it really shouldn't be surprising that some small fraction of users will notice and will be unhappy.

Actually, that's a pretty interesting solution, I wonder if that would work. Unlimited storage, but you can only upload 100GB per month. Or 1TB per year, so you can upload loads of data at once.

The ISPs have tried some 'fake unlimited' plans over the years, which got a lot of pushback. eg, unlimited data but you're capped at 2G speeds after 10GB. It's a model that totally sucks for normal users who happen to hit the limit, but could work better in a storage context: full speed instant access to a 100TB archive blob isn't quite as much a day-to-day personal essential as basic mobile data. And if you DO need fast access, there should be a path to pay for it.

I don't think it's quite the same - ISPs trade specifically in the amount of data uploaded/downloaded, so when they say you get "unlimited data per month" but in reality that's capped at 100GB, then yeah, that's misleading advertising. This is more akin a storage facility that says "we'll store unlimited number of your items, but you can only give us 1 tonne a month". But you know, if you keep supplying them a tonne a month for 10 years, then yeah, they will keep your 120 tonnes of stuff.

"How do you handle such users at MediaFire?"

You silently and/or passive aggressively throttle the whales in such a way that they go away.

I learned this a long time ago (1998) while consulting for a very large and successful early web host (Simplenet). They allowed unlimited bandwidth and one of my first questions was "how do you make that work?".

They mercilessly throttled the top 1-2% of users until they just went away. Easy.

That very well illustrated to me what happens when interests are not aligned. The provider is going to actively work against you - as opposed to actively enable you.

> You silently and/or passive aggressively throttle the whales in such a way that they go away.

Carbonite is an online backup provider that got caught doing this by the U.K. government in 2012: https://en.wikipedia.org/wiki/Carbonite,_Inc.#Misleading_adv...

Thanks for the great service MediaFire is. It outlives many file sharing services out there, the links are pretty safe to share, knowing that they will be retained for practically forever.

"In my findings, the reason unlimited doesn't work is a small percentage of users (very small, like ~0.1%) who push it to the extreme. In other words, if your exhaust 50% of your resources to service 0.1% of your users, at a fixed subscription cost, the model doesn't work."

I maintain that there is a deeper problem and that is the concept of aligned interests.

The provider can say all manner of nice things and we can all smile and feel good but if the economic interests don't line up no matter how trivially there is going to be an antagonistic relationship. I don't care how much tailwind we're all getting from diminishing hard drive costs - the relationship remains fundamentally broken.

I want my provider to enthusiastically embrace my own goals - especially in critical areas like data backup.

Mathematically speaking, even though there is a recurring cost, if the cost goes down over time it's entirely possible for the cost to be finite overall. Simple example: If the recurring cost halves every year, and in the first year it's X then the cost to store it forever is 2X.

Just because it has halved every year for a decade doesn’t mean it will continue. Cf Moore’s Law

That’s why he said “mathematically speaking”

Wow MediaFire, I remember years ago when I was 11 I absolutely loved the service. I thought it was so cool you could see how many people downloaded your files haha thanks for building MediaFire

>"I launched MediaFire as an unlimited storage provider in 2006."

Oh hello there. Best wishes and hope life is wonderful for you.

The company that saved me big bucks on Amazon S3 fees that were absolutely insane. Yet the same company that completely ignores requests to fix disappearing files that remain in disappeared state despite re-upload.

Reply on news.ycombinator.com