Microsoft 365 Outage
status.office365.com
status.office365.com
Screenshot: https://rog.gy/ss/3b14f7a2ac.png
Now, the pirated version never phones home and always works even when MS decides to steal your products back.
Just yesterday I had a similar issue with Resharper, where our local licence server was down for the day. Luckily I was able to enable a 30-day trial. (and I guess Resharper isn't that mandatory to get work done)
Me too. After a disastrous episode with DropBox while travelling in foreign countries nearly a decade ago, I instituted my own File Server that's accessible via the Internet. All of my data is 'safe at home' with daily backups.
I now keep my primary 'work' and 'private' collections (10-30g each) and a large shared set (350g) synced across desktop & laptop, and also 3 off-site archive servers, using Syncthing. This includes a literal satellite connection for one site.
Its less of a hassle to use something like nginx reverse proxy docker container in front of your web services, as it comes preconfigured with some best practices regarding TLS. (TLS1.2+ only by default) https://hub.docker.com/r/jwilder/nginx-proxy
If you use docker, be sure to google around how to manage firewall with containers (must use DOCKER-USER chain instead of INPUT)
Ofcourse, you will get a bit involved in setup step if you choose to tie nginx proxy, letsenccrypt and nextcloud containers. But search engine is your friend.
What was that? They cut your access because you logged from a different IP?
Except that it worked in reverse. My laptop was 'emptier' than the home server, so the home server's files were deleted to match the laptop. Hundreds of megabytes of files were destroyed before I managed to stop it.
On top of that, I only had a 2 gig per month quota for internet access while travelling, and all the DropBox activity soon exhausted that in days.
On later trips my own home server coped very well, with less overall unwanted activity.
Because you're less likely to brick your IDE than Microsoft is?
To be fair, that hasn't happened in a pretty long time for me - but in the earlier days it was commonplace. It used to be bleeding edge features that would do it, other times it was just because they took so long, that interrupting the process of upgrading or installing caused a mess. Or installing the newest version alongside the older version, etc.
Visual Studio is a massive and complex creature (much like other "big IDEs", such as Eclipse) - although it has become much more streamlined in more recent releases - considering how complex it is.
EDIT: To be clear, problems often involved corruption of registry values, library/DLL hell, small database or configuration files and caches, etc. that happened with Visual Studio in the scenarios above.
Sure, sometimes installing new versions alongside old versions could cause issues. But that's a managed process that one team member would do before the whole team migrated and any issues would be worked out during that stage. (one of our old upgrade steps was to install 2015 before 2013 because doing it the other way around would break something in 2013; 2013 still being needed for some projects).
I have never been in the situation where the entire company (or even entire team) was locked out of their IDE.
Also, while this isn't bricking the IDE, you can definitely ruin solutions & project files in a very hard to debug way if you ever need to manually update those files.
Restores operating system image in a few minutes.
Yes, since the inception of these kinds of tools, recovery from these corruptions became easier by restoring from an image. Still, in the early days, you could burn a lot of time doing anything of these things!
Visual Studio major version updates (uninstall old plus install new) used to be a good way to do that and often wreak havoc on the overall stability of your system as well. At least, on a system where you didn't want to strip it to fresh OS install as an intermediate step.
This isn’t even the first time this has happened. I do most of my development on my desktop computer, sometimes I go long stretches without using my laptop. More than once I opened up my laptop on a flight to try and get some coding work done, but surprise, license is expired and I don’t have any connectivity to renew it. Absolutely ridiculous.
I don't think you can target core 3.0+ at all in VS2017, and only 3.1 is supported now. What that means for security updates I don't know.
Title: Can't access Microsoft 365 services
User Impact: Users may be unable to access multiple Microsoft 365 services.
More info: Any Microsoft 365 service that leverages Azure Active Directory (AAD) authentication may be impacted by this issue.
Current status: We've identified and are reverting a recent change to the service which may be causing or contributing to impact.
Scope of impact: Any user may experience access problems for Microsoft 365 services.
Anyways the Azure status page is a bit more grown up:
"Title:" makes it look like a grade school book report.
Title: We're investigating a potential issue affecting Outlook.com
User Impact: Affected users may be unable to access Outlook.com services or features.
Current status: We've identified a recent change that appears to be the source of the issue. We're rolling back the change to mitigate impact.
Next update by: Monday, September 28, 2020, at 11:00 PM UTC
Current status: We've identified that reverting the recent change did not alleviate impact to Microsoft services as expected. We're working to explore additional options for mitigation.
[I actually enjoy these updates but they indeed read more like our internal incidents channel]
The page you've linked is kind of the "our real page is down" page, and after the issue is resolved what we usually see is all that replaced with something like "go check the admin center for outage information".
It just seems that office 365 is part of that older monolithic Microsoft that we all used to hate.
Ironically found on https://www.microsoft.com/en-us/research/publication/distrib...
The cloud is the ultimate rent-seeker's dream. The world's computing power in the hands of a few enormous companies... who then sell it to you in tiny pieces and charge for every little thing you use on their services. And when something goes wrong, you are powerless to do anything about it.
For example, right now I could be using Azure SSO, which would have been free with Office 365 at the level we're bought in at, and then all of my other cloud products would be just as inaccessible as Office 365 is. Even when it is working, I could have an edge case issue and never see a feature added or a bug fixed to help me at all. Or, I could go with a 3rd party authentication company, and get a product that their company lives or dies on. My experience is that, if chosen well, you'll get a much better experience even though you have to do some legwork here and there to make things work together.
You can't even sack anybody for it.
Microsoft can still probably do a much better job compared to a small team.
This eliminates a lot of moving parts and things that can fail. Imagine an Nginx server running on a single machine. There's very little that can go wrong here beyond hardware failure (and certain types of failures can be mitigated with things like RAID), and yet it is probably enough to host most internal websites.
Now compare this to something like Azure App Service which is obviously more complex to be able to support many tenants, load-balance, etc. There is much more that can go wrong with the entire App Service infrastructure (due to its complexity and moving parts) than with a single machine running Nginx, and the complexity will also delay disaster recovery efforts (this outage is now lasting for more than an hour - you can reinstall an entire Linux web server from scratch in half that time).
I worked for a company, where I really did have nearly 100% uptime at my local data center in our office. I wasn't a luddite, and the cloud wasn't something I was afraid of or didn't understand. I just, at the time, had an infrastructure in place before the cloud existed, and it worked well enough at a great price point. We were on the same electrical grid as a hospital, our infrastructure handled our scale, and just really didn't have problems for a long span of time.
Still, someone came in and said.. what, you are still doing stuff on-prem and not the cloud!? How legacy! How dated! And so the push to the cloud came next.
We migrated to the cloud (and some colo facilities for some equipment we chose to keep), and on the 3rd day of being freshly migrated, the cloud provider had a major outage and went down for longer than we'd ever had on prem. The next month, the colo went down and their diesel generators failed, and it was down for an entire afternoon.
Oh sure, there was some SLA money returned in that event from the colo...
I'm just saying, I've been on both sides. Tell them the truth - even the cloud has outages, but it is certainly more "convenient" to have hundreds of engineers work on fixing an outage at global scale, and all you have to do is wait for it to start working again - then it is to have to fix it all yourself.
Yep, that’s the argument I’ve made, both to management and even our non-techie customers. When we show them our new SaaS product, they get concerned about cloud outages. Our response is “if AWS goes down, you have bigger things to worry about since other things will be down too.”
The plus side I’ve mentioned that our company’s software developers are freed from doing network IT maintenance and infrastructure and can continue focusing on our actual core products instead.
It's one thing to experience the sysadmin practise getting taken over by developers. The next thing is usually the latter needing to be "freed" from this burden... Seems like a big fad.
I'm not sure if even that number is low enough.
Was worried there for a second.
OneDrive for Business is almost unusable on Windows 10. The web apps are incredibly slow. Teams is very inconsistent for users between desktop and mobile usage. It's a mess.
I'm planning on switching to Amazon next year. I hope I can find an alternative to the Office situation (maybe offline licenses).
Luckily Microsoft still sells offline licenses for Office 2019. There are several features missing that are exclusively tied to the Office 365 subscription, but overall it’s still the same Office productivity suite you would expect. Here’s to hoping they keep doing that in the future and don’t pull an Adobe…
At line:1 char:16 + $loop = "loop" do {$response = $null; Start-Sleep -Seconds 5 try {$re ...
+ ~~
Unexpected token 'do' in expression or statement.
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : UnexpectedTokenwhile ($true) { $response = $null; try { $response = Invoke-WebRequest "https://login.microsoftonline.com/common/oauth2/" -ErrorAction SilentlyContinue | select status*; if ($response) { "OK $($response.StatusCode) $(Get-Date)" | Write-Host; } } catch { "Down $(Get-Date)" | Write-Host; }; Start-Sleep -Seconds 5; }
Oooh that's panic stations.
Imagine there's some foxhole prayers happening.
Big exhale.
https://www.nbcnewyork.com/news/local/nationwide-reports-of-...
A few times during outages I managed to get hold of someone that worked at Microsoft who stated something along the lines that they only state that a service is degraded or has an outage if enough customers complain - or if the websites they have to sell / demonstrate products to potential customers are unavailable.
The link is javascript:window.external.AddFavorite(location.href,%20document.title);
We (like many other orgs, I expect) have gone all-in on AAD, so this has pretty much taken down everything company wide with the exception of the factories. Good thing the plant floor isn't tied in.
How frequently do you think this sort of thing would have to happen before your org considered a significant change in arch?
Looking at it now, Azure AD is the only service that isn't green.
https://status.azure.com/en-us/status
A few days ago a similar outage occurred with Google Accounts.
Wasn't there also a widely-publicized outage in 2016?
Microsoft Teams also appears to be down so it's not just authentication that's affected as I was already logged in and it still doesn't work, just hangs trying to load forever.
Edit: appears to be resolved at 1:07 am London time.
I thought it was just the network here being overly keen to block requests since I'm relatively new and am still finding out all the quirks
This also explains why clicking "Sign in" would randomly download an HTML page (ie logout.html) instead of redirecting to the proper auth endpoint!
"You know you have a distributed system when the crash of a computer you’ve never heard of stops you from getting any work done."
I think I once did something stupid like try to patch GoOffice into OpenOffice and couldn't get it to build. But the distro packages still worked!
Edit: maybe not that short, in 20, 30 years maybe?
1990s hosting company: Linux Server 99.99% uptime. Windows - no guarantee. 2020 - same