Microsoft limits institutional cloud storage space for environmental reasons
scholar.social
scholar.social
> Storage of this “dark” data takes up space on servers and results in increased electricity consumption, generating 4% of global greenhouse gas emissions in 2020 alone.
Where "dark" data is data that is unused. However they link to this World Economic Forum Page [2] which says
> In 2020, digitisation was purported to generate 4% of global greenhouse gas emissions.
So the 4% figure is not the storage of only dark data, but the storage of ALL data. The WEF page says that half of all data is unused, but how much of the energy usage is due to the unused data? I would think the bulk of the energy would be used when the data is accessed, and data sitting on a drive not being accessed isn't really using any energy.
This is really misleading.
[1] https://www.microsoft.com/en-us/education/products/microsoft...
[2] https://www.weforum.org/agenda/2022/10/dark-data-is-killing-...
To the extent that the changes become permanent, they probably lead, working from home, to a far greater reduction in useless greenhouse emissions than the data ever could.
It's not only misleading it's just detached from reality.
[0] https://www.ey.com/en_ch/decarbonization/how-digitization-ac...
But giving institutions 100TB of base storage with more for paid licenses isn't exactly stingy.
To me personally the security claims don't make much sense either.
When I face disk space limitations I don't delete my spreadsheet of survey results that's full of private information. I just delete the original video of that public lecture that I've put onto youtube already anyway.
Now, for professors there might be a case for shunting the files they haven't touched in a few years into secure offline storage in case it contains forgotten private information. But it sounds like this applies to student accounts, and there's already a process to eliminate forgotten years-old data: Graduation.
IMHO it's far more likely this is a cost-saving measure.
Storage or ALL processing? SSDs typically need very little power to just store data. They need some power to read and write. But the lion’s share of energy use for “digitization” is compute in various forms.
To be able to store that amount of data on SSDs, you need tons of enterprise mixed load SSDs, and they don't come cheap in any sense or form.
To offset costs, you add tiering with mechanical disks, with varying capacities and readiness states.
Spinning up cold storage takes time and energy, and if you need a three year old CSV file summarizing your results, that's a lot of energy and time. The storage pipeline storing rest of your research might go to "sleep" again until you make sure that you gonna need rest of the dataset and everything to re-run some of the experiments.
The data access pattern maybe hitting some cold paths too often as a result, so Microsoft might be trying to confine research institutions to a warmer tier, and since that tiers are more expensive to operate, they might be slashing storage space. Otherwise the resident data might poison other user's access patterns.
This is not to say Microsoft is doing right, I believe the opposite, but this is why they might be doing it.
> So the 4% figure is not the storage of only dark data, but the storage of ALL data.
Even that sounds like a misread. "Digitisation" can refer to the sum total of moving a product or service or process to be digitised; that would also include the compute cost as well as the storage.
Producing, storing, and shipping masses of paper documents (and related printing equipment) isn't particularly efficient. And video/audio would need storing on physical disks/tapes, and so on.
This isn't dark data. It must be kept, so it is being used.
Lay-offs similarly also reduce the demand on energy since people have less money to buy things that are energy intensive to produce, and have less money to pay their electricity bills and fuel.
Sure, in the long term this move by MSFT may reduce new energy producing capacity coming online, but so will lay-offs, and pay cuts, and recessions, depressions, famine and war.
I don't think it's bad that MSFT is cutting cost in this way, it makes them a more efficient business, but this is being done for cost-cutting, not because MSFT cares deeply about the environment, it just so happens that money is a proxy for energy (i.e. the ability to generate entropy) and most cost-cutting will also reduce energy demand regardless of the nature of that cost-cutting.
Excess employees will be mulched.
Again, cost-cutting is great, especially when it can be done with no impact to consumers and employees, because those are humans I care about.
In this case, it is at the expense of consumers, the people who are affected are getting less for their money. Shrinkflation if you will, but because of the corporate, bureaucratic and dysfunctional mess between the people paying and the people choosing MSFT — and the way MSFT operates, which is by bundling basically everything but the kitchen sink — there really is no practical way for the consumer to shop around with their money for a better cloud storage deal.
MSFT could still be the most inefficient storage, cloud storage space from competitors could be much more energy efficient for the same amount of GB — but the storage will stay with MSFT because they bundle it with everything else.
This has nothing to do with the efficiency of cloud storage or the carbon footprint of cloud storage. It has to do with giving people less value for the money that MSFT gets, and it works out because MSFT has done everything they can to ensure that there is no way this will result in revenue reduction.
From a shareholder perspective, this is great, you should give customers the least amount you can for their money so you can maximize profits, but if you tie this together with the immoral business practices of MSFT it really becomes very detrimental to society, the environment, and consumers.
I think that only approving of environmental things if they cost you is a bit perverse, though. This is lower-hanging fruit. Why not take it?
To MSFT there is no lower hanging fruit to more revenue than reducing what they give a customer while charging the same amount of money, so no I'm not particularly impressed that MSFT is stiffing their customers. I'm also not impressed that MSFT manages to do that while also getting cheers because they figured out the right words to say to get people on their side. To me, that is deeply unethical behaviour.
This is just 2024 again. My theory is here that regardless of what you might otherwise think of ESG, I think one of the key things to bear in mind is it imposes is that customer value is not the top priority.
I understand of course that the longer-term idea is that, say, all businesses get better at waste disposal, and once that happens businesses can prioritise customer value again, but that will take a long time, if ever, to manifest all its goals fully. Additionally, in reality it seems that it encourages businesses to become performative in the way you're highlighting.
If you cheer on all other forms of shrinkflation, then sure, this is something you would also like.
I should be clear that I don't think profit maximization is bad, I'm pro market economies, and this is just a fact of life in a market economy, but it's also nothing new, and it has nothing to do with saving the planet.
The main thing I'm bothered about is that this practice is cheered on by environmentalists when it is done by a company that has done everything possible to disrupt market dynamics. If one laundry detergent manufacturer does shrinkflation, I can fairly easily switch to another, if MSFT does shrinkflation on cloud storage, and they bundled it with everything else, I can't easily switch.
MSFT will also keep doing this, their first aim with this to become irreplaceable, and then they can start jacking up prices, and now they also figured out how to get some praise for saving the planet while they engage in incredibly immoral business practises. This is impressive, but more in that it actually works.
Well, I think I agree. The only reason I brought up ESG is it gives all detergent manufacturers the incentive to provide the same shrinkflation, and gain points for it. Not to say they will; I just thought it was an interesting angle.
I do think that offering infinite space and then limiting it to 20GB isn't quite the same as shrinkflation. It was silly to ever offer unlimited space. But they are definitely tightening the screws, I agree.
A lot of the people who had MS offering instead of google were saying that they are safe with their choice. 1TB is much more reasonable than "unlimited" and Microsoft would keep it. Now here we are.
Happy to be part of CERN so I don't have to worry about my institution storage situation at all. And it is not like one drive or google drive would be suitable for me anyway.
Not sure anyone gave you generous free FTP space back then, and Dropbox wasn't invented yet. But people did absolutely abuse gmail for file storage.
1: https://ourworldindata.org/grapher/historical-cost-of-comput...
Later on I was kind of envious of my friends with an Amiga 500 and its 512 KB, until getting a 386 SX, with 2 MB in 1992.
Yeah, really understand where the professor was coming from.
So, I am still amazed by how people on the two sides spectrum grew up.
The "interactive" compile-debug-fix of his day.
Something that I never experienced myself.
People don't throw away hardware just because the software stopped getting updates. Although not recommended, there's even people still using Windows 7.
I tried to give away my old laptop to a charity. They wouldn't take anything more than four years old, and nothing that wasn't running windows 10. Anyone want a free laptop? I've got one sitting in the closet that I cannot seem to give away.
And AI's are destructive for the environment why? Is it more environmentally friendly to waste a lot more human time with less compute power doing mundane repetitive shit for hours that an AI/LLM can summarize or answer in seconds?
With that logic it's also more environmentally friendly to use an ox to plow the fields instead of a diesel John Deer.
Ironically, on another business unit, everyone should throw away their perfectly working computer at the expense of the environment, and buy as Windows 11 capable computer.
Priorities.
> all institutions’ tenants will receive 100TB of free pooled storage across OneDrive, SharePoint, and Exchange, with an additional 50GB or 100GB of pooled storage per paid user for A3 and A5 subscriptions, respectively (not including Student Use Benefits). With pooled storage, an institutions’ total storage limit is the maximum amount of storage all users combined can use.
> In addition, effective February 1, 2024, users of Office 365 A1 (free subscription) will be limited to a maximum of 100GB of OneDrive storage within the institutional tenant’s 100TB of pooled storage.
McGill seems to offer desktop apps to students, so they have paid licences. [2] This means they could offer 50 GB per student, and I imagine they could allow the student to have more OneDrive files and less Outlook data. Furthermore, if they care about research data, that should be in a common SharePoint space with well-defined access and edit permissions.
[0] https://www.mcgill.ca/polling/channels/news/important-onedri...
[1] https://www.microsoft.com/en-us/education/products/microsoft...
Computers were originally built to serve the needs of academics. Now academics are told to serve the needs of (American) computers.
Something will come of it, particularly because of how much buy-in there is/was from academic institutions who jumped ship from Google Docs+Drive to Microsoft cloud offerings.
It always has to be the end user who sacrifices for the environment.
I like having one connector for everything.
The e-waste related to including chargers in the box is a real thing. That change also allowed them to halve the size of the boxes, effectively doubling the shipping yield. On a billion-device level that's a non-trivial change.
And yes, it's good for their bottom line. And yes, it effectively increased the price by $30. Sucks for me, yes. So what?
Apple(and everyone else, as far as I know) includes the cable though. And it's hard to wear out a power brick that just sits plugged into a socket all day.
>>assuming they're even compatible, since not everything used USB3 a few years ago
Apple chargers were always just AC to USB-A though, so all you need to change is the cable. Which....comes included with the device. It's a bit more of an issue if it's now a USB-C to USB-C cable, but a USB-A to USB-C cable is not exactly expensive.
For me, gallium nitride have changed the charger game for me and I refuse to use stock apple chargers on account of their ungainly largeness.
Charging a macbook with a tiny charger is so satisfying.
Perhaps I was being too flippant in my original comment because I really miss the apple that cared a lot less about profitability and would give you a free new model year macbook when your old one broke (probably a lot worse for the environment too lol)
If only any USB-C cable was compatible with iPhone or Pixel it would be fine, but I have 4 USB-C cables that don't work with them.
That company had tremendous hardware but the serious affliction of being grotesquely obsessed with proprietary everything. I have nightmares trying to sort through my old boxes of cables.
Keep living with attitude like that, eventually you’ll realize.
I'm a big fan of tackling the low hanging fruit of recycling and waste reduction, and not including a charger is definitely up there.
Now the phones only need to be unique to radio requirements.
It is saying like why should IC engine performance always suffer because for fuel efficiency.
Ultimately the supposed and alleged environmental harm *always* in every single case is a result of human consumption so some human has to suffer for these allegedly environmental moves if not the person buying iPhone then the Apple employee or Apple Shareholder or average tax payer.
At this point I’m not even mad with Microsoft
Ideally, you would be able to buy a ready to go NAS and set up your own cloud at home in a few easy steps.
* The average person does not do backups. No, he will not do backups. No! He seriously won't. Cloud storage solves this very real, very basic problem.
* The average person does not understand file systems. No, he will not understand file systems. No! He seriously won't. Cloud storage solves this problem by Abstraction(tm).
* The average person does not use much disk space; the biggest chunk is probably the photos on his phone. Combined with the previous, a 1TB HDD/SSD in a NAS at home is "WTF" and "WTF" for him.
Note, we aren't average people.
It should be technically feasible to make setting up backups to your own NAS at home as easy as backups to iCloud. So if people are backing up to iCloud, I do not see why they would not backup to their own NAS. Same for why they do not need to understand a file system just because they have a NAS.
> Combined with the previous, a 1TB HDD/SSD in a NAS at home is "WTF" and "WTF" for him.
I don’t know what WTF means here. 1TB is cheap these days, so not sure what the issue is being pointed out in your 3rd bullet point.
A proper backup is 3 copies, 2 on different media, 1 offsite.
Absolutely no average person is going to bother with any of that. Three copies? Ridiculous, he will say. 2 different media? I ain't made of money, he will say; also WTF is a "media"? CNN can backup my stuff? 1 offsite? I'm not going to juggle stuff in and out of my bank's safe deposit box, he will say.
Cloud storage solves all of those concerns for less than pennies to a byte.
>Same for why they do not need to understand a file system just because they have a NAS.
Managing a computer of any kind requires understanding them, and that includes understanding file systems. Most people do not understand file systems, please stop telling them to because it's never going to happen.
>I don’t know what WTF means here.
"What the fuck" as in "WTF is '1TB'?", "WTF do I do with 1TB?", and "WTF is a 'NAS'?". Remember, we are talking about average people. Not computer wizards and sysadmins like most of us here.
Why is this relevant to people having the bandwidth to do backups at their own site? And getting cutoff by a big tech company with no recourse is not a “proper” backup either.
> Most people do not understand file systems, please stop telling them to because it's never going to happen.
I never did. I setup a Synology NAS 8 years ago and I have not had to look at it since. Surely, the UI software is even easier to use these days.
> What the fuck" as in "WTF is '1TB'?", "WTF do I do with 1TB?", and "WTF is a 'NAS'?". Remember, we are talking about average people. Not computer wizards and sysadmins like most of us here.
We have different expectations of people. At one point, average people did not have electrical, plumbing, washers, dryers, refrigerators, laptops, smartphones, routers and access points, etc.
I think they would be able to handle one more appliance where plugging it into network, scanning a QR code to setup, and pull out and push in HDD in the event a HDD or NAS break.
By far the biggest reason most people do not do backups is because It Is Fucking Tedious And Expensive. Cloud storage is the answer to that problem.
It's only when we get to big enterprise companies (read: businesses that wouldn't classify as small business) that cloud storage becomes impractical or even legally infeasible. Of course, remember we are talking about the average guy at home.
>I setup a Synology NAS 8 years ago and I have not had to look at it since. Surely, the UI software is even easier to use these days.
I have a Synology NAS too, and I agree it is very easy to use... for special people like us who breathe computers.
>We have different expectations of people. At one point, average people did not have electrical, plumbing, washers, dryers, refrigerators, laptops, smartphones, etc.
I'm not sure if you realize that most people call up a professional technician or engineer for all of that if something comes up. Fridge or washer breaks down? Call. Toilet backs up? Call. Breaker keeps tripping? Call.
Now yes, an average person could hire someone to setup and maintain a NAS for him at home, but that's going to be far more expensive and a hassle than just forking over a Hamilton ($10 bill) or two to Microsoft or Google every month and making it all someone else's problem. It'll be managed better and he won't have strangers coming into his home, too.
Computer literacy is at an all-time low because the overwhelming majority of it has been hidden away as much as possible by large tech giants in favor of their own corporate "clouds".
And no - the "next generation" isn't doing any better on that either because they were raised on these devices and often don't know, or worse, don't care about that sort of thing. It's really bad.
2) Filesystems et cetera, are not rocket science, Windows gives you even a way to automatically create mirrored disks, it's just a matter to read a tutorial
I don't feel I have any obligation towards average people, it's their works and it's their duty to understand basic tech, it's not rocket science. Like all other people try to inform themselves about pipes in a bathroom, kitchens, medicals, etc.
If they don't, if Google or Microsoft change their ToS (That I guess the average person hasn't bothered reading) or if Google has a data loss, then should not complai, if Microsoft changes their policies, they shouldn't complain, being an average person is not a justification to live leeching society, ignorance has never been a justification
You asked "how many cloud services will have to give people unpleasant experiences for those same people to stop going all in on cloud services", and the answer to that is "Ain't nobody got time to 'puter.".
No, I'm not being facetious; I'm serious. The average person wants to use computers to enrich their lives, not manage computers and be sysadmins. Cloud storage is just one natural evolution path of that demand.
Hell, I'm not an average person and I'm also buying into cloud storage now. I'm in my mid 30s now and life isn't stopping; I want to become a greybeard, not a neckbeard.
Also, I'm going to reply to this claim here since it was edited in after I already replied:
>Filesystems et cetera, are not rocket science,
File systems are absolutely rocket science.
It's been well over 30 years since that paradigm was introduced to try and make how computers handle data more relatable to most people and it has not worked.
It is the year 2024 and most people still put everything on their desktop. Kids in particular[1] do not grasp file systems at all.
File systems are an abject failure of human interface design and engineering.
[1]: https://news.slashdot.org/story/21/09/27/2032200/students-do...
Tbh, it's an office-based metaphor created by and for office admins. The general public just doesn't relate. Maybe if they'd called directories "wardrobes" or "drawers"...
There is hundreds of companies offering nextcloud hosting, and yet the presumption is that only the big 3 is viable.
Honestly I don't have any sympathy for university expecting this for free when they could easily pay $5/student/month or some even more discounted price.
So to answer your question, I would guess about 4-6 more years.
[0]Cloud vs on prem, monolith vs micro service, functional vs OOP, to name a few.
[1]I've priced some cloud that cost more in six months than on prem hardware and still gotten push back.
[2]Somewhat morosely, "[Things] advance one funeral at a time"
Reproducibility and vetting research output, as well as other forms of accountability requires research data to be present and available. Hence the requirement to safeguard that data. However, a personal share hosted on institutional cloud storage isn't the best place to store research data for lack of accessibility, discoverability, etc.
McGill does offer research data management services. [1] These services are meant to guide researchers as well as cooperate with them working towards long term archival solutions and policies for safeguarding research data.
The linked post nowhere mentions research data management, or the fact that there's awareness about the existing of services at the institution.
I seem to vaguely recall that some of the glacier data was just put on Blu-ray discs and filed away. Seems like a viable solution and with the cloudesq abstraction it doesn't necessarily matter so long as it's being done properly.
Of course I suspect the environment is being used as convenient excuse to people with environmental policies.
"And this is why we [McGill] are asking you to reduce the data stored in your OneDrive and in your Outlook email."
https://www.mcgill.ca/polling/channels/news/important-onedri...
But, amazingly, Microsoft did argue this nonsense:
1. security risk of storing files
2. environmental impact
3. ability for Microsoft to innovate more by deleting your stuff
https://www.microsoft.com/en-us/education/products/microsoft...
Ministry of Truth...
https://en.wikipedia.org/wiki/Ministries_in_Nineteen_Eighty-...
Edit: our IT admins are really baffled too and will now verify with Microsoft if this is an actual thing or just poor phrasing from McGill.
Update: there'll be a change apparently but not this draconian. Storage will be limited to a minimum of 100GB/user with an additional 100TB "pooled" storage for everyone. While this is a shock to me and my admin, McGill's policy is still exceptionally draconian.
Clearly it's primarily a cost saving measure, and I suspect it will be a significant change for some schools, but I suspect the numbers chosen were based on what most schools are using.
Compare that to datacenter NAS with RAID and with replica NAS redundancy and operating overhead (space, power, people), you're looking at a 9 months to 18 months price tradeoff before factoring in how to maintain a rotating set of users/accounts' storage. Storage is an area it's not difficult to do better than cloud if you are single user just storing one user's content, but it's much harder to do better than cloud if you have user management at scale to add in.
Weird, you should tell Microsoft this, because I don’t know what it was before but the link in that thread says they are:
https://www.microsoft.com/en-us/education/products/microsoft...
Limiting to 100TB pooled and $30/TB/month for extra storage.
One of the reasons explicitly given being “Environmental Impact”
Not sure what 'thread' you mean. If you mean Twitter, you are correct, I did not 'actually read the [Twitter] thread', I went to source at McGill and linked that here, then further linked to Microsoft policy update after using Kagi to find the Microsoft propoganda regurgitated by McGill:
https://kagi.com/search?q=Microsoft%20is%20introducing%20cha....
To be clear, Microsoft is not making McGill limit student storage:
"Every institution’s tenant has a 100TB base storage capacity. Microsoft 365 and Office 365 A3 and A5 paid user licenses add 50GB or 100GB respectively to the pool of storage. In addition, institutions can purchase additional pooled storage in 10TB increments for $300 estimated retail USD monthly to add to the tenant pool. To find your capacity, use this calculation: 100TB + (#A3 paid users x 50GB) + (#A5 paid users x 100 GB) + (additional storage purchased) = capacity/limit."
McGill is making a choice, there is a reasonable price to maintain whatever pool McGill want, and Microsoft's argument for matching Google academic threshold drop was posted 6 months ago, no scramble needed.
> all institutions’ tenants will receive 100TB of free pooled storage across OneDrive, SharePoint, and Exchange, with an additional 50GB or 100GB of pooled storage per paid user for A3 and A5 subscriptions, respectively
This is a bad change, period (plus the justification given is basically stepping on a gaping wound), but there are storage guarantees much greater than what McGill's are doing.
> Sustainability is an important consideration that drove Microsoft to make that decision.
Why is a university IT's announcement written as if by Microsoft's PR office? The style of the announcement is very weird. Has McGill outsourced all IT support services to Microsoft, thus it is written essentially by them?
Newspeak is very specifically about the idea of making some ideas impossible to express. You seem to be calling Microsoft's bullshit pretty well.
(Simlarly, panopticism is about supressing expression of ideas though the threat of surveilance and subsequent repression, and going by how much political discourse has eroded, our version of surveilance capitalism doesn't seem to limit anyone from voicing their ideas, no matter how far it strays from prior social expectations of decency)
Isn't this fraud? The customers (institutions or others) had agreed on a certain amount of storage space for a certain price, but now Microsoft is changing the cards on the table.
The "environmental reasons" smells like bs.
Having to store data for five years seems to me to be a lame excuse.
Then O365 business.
McGill really went all-in on cloud storage without ANY local backups. An institution. Wow.
I bet next you’ll tell me they’re gonna open a consumer Backblaze account.
It is quite frankly insane.
But in general it is a trend that IT services are outsourced and that IT departments, and in general administration, in many universities suffer cuts, which is really bad. Apart from the problems this causes, imo this also pauses a security risk as if people's needs are not well covered by services provided, people will find their own solutions without knowledge or consideration to security.
Any time something is expensive, and you do not want to pay for it, cite the environmental impact in your refusal.
To expand on this: A few TB diskspace are dirt cheap.
a Seafile seems much more appropriate.
The reason why things gets worse is not that the cloud companies are evil monopolies(they are for profit companies who cannot put the common good ahead of their shareholders financial interests) but that people refuse to look at smaller less hyped alternatives to what gets marketed by the biggest companies in the cloud sector.
It's teoretically possible if your vigilant of able to force users to use your published workflow to run windows with purely onprem resources but it's not the path of least resistance.
MS have done a good job of tying everything into a big entangled system where it's hard to disentangle any of the "on prem" products like windows and office offline from the cloud backend used by office365, and a lot of this is introduced gradually with standard(often security critical) updates it gets used without a lot of policy.
You can backup the office365 data elsewhere(were using aws s3 buckets for office365 backups where i work) but that's not cheap nor particularly easy to do and the need to do so is often overlooked when the contract for o365 is signed.
There was a trend of ripping all movies from BlueRays to Plex.
If someone is environment friendly I think we should go back, huh.
Storing BlueRays do not require any energy.
They shall limit data collection "for environmental reasons". /s