Zoom outage caused by accidental 'shutting down' of the zoom.us domain
status.zoom.us
status.zoom.us
I guess that's what happens where they had to accept substandard domain, because they were unwilling to be creative about their name.
False, the audio equipment manufacturer uses: https://zoomcorp.com/
The https://zoom.com domain shows content from the video chat platform.
https://domainnamewire.com/2019/03/23/did-zoom-pay-2-million...
Anything at all.
The only limit ... is yourself.
> In May 2012, the company changed its name to Zoom, influenced by Thacher Hurd's children's book Zoom City.
It cites https://vator.tv/2020-03-26-when-zoom-was-young-the-early-ye... where Jim Scheinman says:
> “I loved this fun little book as much as my kids, and hoped to use the name someday for the perfect company that embodied the same values of creativity, exploration, happiness, and trust. And the name works perfectly with a product that connects us visually to one another and that always works so fast and seamlessly.“
To save people the agony of visiting Wikipedia for themselves to check, changed from Saasbee. Which, good call.
Fits great with the idea of bringing people together with video.
During the pandemic many people used zoom more than their cell phones.
The speed of starting a call sometimes could take a bit more but once established was higher quality than the alternatives at the time.
A while ago and, out of curiosity, I did a Whois Lookup to see what big tech companies are using as their domain registrar and found that Microsoft, Google, Amazon, Tesla, Netflix and Shopify are all using MarkMonitor. On the other hand Apple uses "Nom-iq Ltd. dba COM LAUDE", Meta (and its children) uses RegistrarSafe and Nvidia uses SafeNames.
Someone had fun with that one.
Imagine being a small startup with a similar problem. Godaddy will not even entertain you.
Your reply doesn't seem sarcastic, so I take it you genuinely r/whoosh'ed (that's a reference to a subreddit about situations where someone is acting clueless).
.us is not the “root DNS” and your misidentification is muddying the waters.
.us is a TLD (Top-Level Domain) and more specifically, a ccTLD (cc = ‘Country Code’).
https://en.wikipedia.org/wiki/.us
And the English Wikipedia says that its registrar is a subsidiary of GoDaddy named “Registry Services, LLC”.
The root DNS servers and registry are not run by GoDaddy or a subsidiary.
https://en.wikipedia.org/wiki/Root_name_server
They are operated by important entities. Not companies that release sexy commercials featuring Danica Patrick. I keep getting confused between GoDaddy and Carl’s, Jr.
Its also very reasonable to use the more well-known name of the parent company to describe sonething done by its subsidary.
The legality of that system seems a little questionable to me, but IANAL.
the whole point of MarkMonitor is more in the trademark realm, rather than a cloud sysop role.
"Mark" is what trademarks are called in the ... trade.
If you register a ".ps" domain, it doesn't matter if you use MarkMonitor or Namecheap, they can't help you when the ongoing genocide results in the removal of Palestine as a country and ".ps" no longer is a valid country code top level domain.
Similarly, if you register a .us domain instead of a ".com", ".net", or ".org", MarkMonitor can't help you when GoDaddy inevitably screws up.
History has borne this out: .com domains are well-managed. ccTLDs like '.io', '.su', and '.fj' have all had significant security or availability issues because they're run by "eh, whoever the hell the country picks" with no standards.
Financially, a proper gTLD also can't raise prices unilaterally and weirdly, while if you pick a ccTLD, the country has free reign to arbitrarily change prices, delete your domain, take over your domain, etc etc.
Do not use a ccTLD.
If you're based in Germany, I don't see a reason why you would want to avoid .de domains.
And conversely, when not based in Germany, you'd need a proxy Administrative Contact anyway. (Registrars can probably provide that for you, but it seems like asking for trouble.)
Under German law, as far as I understand this is true for publications "addressed to a German audience" regardless of your domain's TLD, your server location etc.
That sounds like MarkMonitor is at least partly at fault here.
100% on the GoDaddy staff.
I’m curious about where are you seeing what Mark Monitor requested? It doesn’t appear in the official status update. Is this public information formally posted somewhere we can all see?
Look into what’s happened with pricing on domains like .org and .info. They’re increasingly absurd, with the restrictions on price increases that once were there largely being removed, at the pushing of the sharks that bought the registrar. Why are these prices increasing well above inflation rate, when if anything the costs should go down over time? Why is .info now almost twice as expensive as .com?
That's not completely accurate. Section 2.10c of the base registry agreement says the following in relation to the uniform pricing obligations:
> The foregoing requirements of this Section 2.10(c) shall not apply for (i) purposes of determining Renewal Pricing if the registrar has provided Registry Operator with documentation that demonstrates that the applicable registrant expressly agreed in its registration agreement with registrar to higher Renewal Pricing at the time of the initial registration
Most registrars have blanket statements in their registration agreement that say premium domains may be subject to higher renewal pricing. For registry premium domains, there are no contractual limits on pricing or price discrimination. AFAIK, the registries can price premium domains however they want.
> The foregoing requirements of this Section 2.10(c) shall not apply for (i) purposes of determining Renewal Pricing if the registrar has provided Registry Operator with documentation that demonstrates that the applicable registrant expressly agreed in its registration agreement with registrar to higher Renewal Pricing at the time of the initial registration of the domain name following clear and conspicuous disclosure of such Renewal Pricing to such registrant
Furthermore:
> The parties acknowledge that the purpose of this Section 2.10(c) is to prohibit abusive and/or discriminatory Renewal Pricing practices imposed by Registry Operator without the written consent of the applicable registrant at the time of the initial registration of the domain and this Section 2.10(c) will be interpreted broadly to prohibit such practices
Yes, premium domains can be priced higher, but the Renewal Pricing has to be "clear and conspicuous" to the registrant at the time of initial registration. Are you aware of any litigation related to this?
It sounds like you think I’m being deceptive. Do you know about any registry premium domains where someone has a contractually guaranteed price?
Also, based on my own anecdotal experience, ICANN doesn’t interpret 2.10c broadly and they allow the registries to push the boundaries as much as they want.
Yes, it is.
"Their enforcement team works with platforms to remove infringing content and can even help with legal proceedings if needed. They don't just find problems—they help solve them through their connections with major online platforms and their understanding of takedown procedures."
What you're paying for is MarkMonitor's people having the cell phone number of the guy at the operations end of whatever point in the chain screwed up. At least that was their original pitch. Now, they have a whole range of tracking services which you can get elsewhere.
This is a better situation to be in than some internal tooling that failed to notify someone because it got forgotten after the developer left.
If one dev had written it, how many times would that tool have failed by now? When the original dev left the company a decade ago, the tool has been transferred between teams six times, it failed a migration and the email address it used to send errors to no longer exists so nobody noticed, and it's literally gotten lost in the shuffle?
Found 12 confirmed bugs in that window using only binwalk and osint.
The worst was that I noticed the zoom.us godaddy account password reset email address was the personal gmail account of Eric S Yuan, the CEO.
So, I tried to do a password reset on his gmail account. No 2FA, and only needed to answer two reset questions. Hometown, and phone number. Got those from public data and got my reset link, and thus, the ability to control the zoom.us domain name.
They were unable to find a single English speaking security team member to explain these bugs to, and it took them 3 months to confirm them and pay me $800 in bug bounties, total, for all 12 bugs.
The one bright side is this did convince my employer to drop them.
It doesn't say the password reset link was used to change the password, which would deprive the account owner access and grant unauthorized access which of course would be illegal.
https://www.justice.gov/archives/opa/pr/department-justice-a...
If you do not want your gmail password reset, I recommend hardware 2FA.
You pay Markmonitor a shitload of money to make sure this doesn't happen. They should have dedicated people at GoDaddy and direct communication channels.
This is a significant fuckup on Markmonitor's part, even if GoDaddy did something different than was requested from them.
Source: Have been OH SO EVER PRECISCE AND EXACT in my communication with certain idiots, and they still screw it up. Several instances of "put this here carefully", only to return and find it all the way across the room upside-down and broken, come to mind.
I don't know why you're trying to spin it as Mark Monitor fault.
I'm not saying that this couldn't have happened with a gTLD But why put your brand at the mercy of a government like that?
Edit: .eu might be an even better candidate for this requirement, but you can ask British former domain owners how that worked out
gTLDs just subject you to an additional layer of incompetence, namely from the company running it. The government where they're located can still come knocking. It's also not like e.g. .nl is run by the Dutch government officials, it's a nonprofit started by some people in the 80s iirc
ICANN have a mostly hand-off approach to ccTDLs. The intention is that each country decide on their own regulations and management when it comes to their country code specific domains.
.nl is a very special case, and it is true that the Dutch government was not involved. .nl was the first country code TLD created outside of the US, when the domain system still was part of ARPANET and operated by the United States Department of Defense. .nl was then transferred to a foundation 10 years later, and that's where ownership now resides.
ccTLDs are somewhat of a mess. Many are created in universities, then transferred to a company or foundation. Others were sold to companies from the start. In some cases, government have sold their ccTLD to other countries.
.se for example was created in a Swedish university, and then later the government took possession of it (or the university gave it to them, can't really say). Now there are laws that explicitly defines how it should be used and governed, which then a non-profit foundation manage the implementation.
As you might notice from the dates and names, this was very early in the history of TLDs.
ccTLDs also have to be run by some organization, which is often a private company. Maybe the country's oversight over this organization is better than ICANN's oversight over gTLD operators. Maybe it's not. Historically, the worst technical incidents have occurred at ccTLDs.
I don't know if that's actually the case, I've heard some shady sites are using .su(Soviet Union) to avoid judicial actions.
So then you don't have to produce an offence that takes the TLD down (whichever kind) but one that makes a judge within the country that the TLD operator operates in approve a takedown notice for your domain name or even get the TLD operator to cooperate voluntarily
Ironically that one country happens to be the one that also controls gTLDs like .com, as others have pointed out, so arguably .us is the one ccTLD that isn't any more or less likely to be reliable.
Dodged a bullet there given that .io is at risk of being discontinued altogether. It hasn't been decided yet, but better to not have that dangling over your head.
So ICANN has a non-trivial choice to make. Either they maintain the position that switching costs are bearable and let .io disappear, or they admit that TLD switching is impossible and save .io, which will make it hard to argue the threat of (registrants) TLD switching keeps the industry competitive.
You guys want to kick indigenous people off their land for military bases? Enjoy your new bill for .io domains.
And Mauritius have treated the Chagossians like dirt for decades, with no signs of that changing.
None of this is to deny the Chagossians were extremely ill treated by the British, but the idea that the Mauritanians have any interest in the welfare of the Chagossians is ridiculous.
A lot of Pacific islands territories have complicated histories like this (e.g. Hawaii, New Zealand), but the focus usually ends up on whatever bastards most recently took over from the previous bastards (relative levels of bastardy notwithstanding).
The thing with the island of Diego Garcia is quite strange and I strongly suspect there is corruption involved. The UK wishes to divest itself? Instead of holding an auction where the rest of the planet can bid on purchasing the territory, the UK decided that Mauritius would take it (who doesn't really want it) and to entice them, the UK is going to PAY Mauritius to take the territory and leave the base alone. The amount is £90 million annually, adjusted for inflation for 99 years.
This is a lot of money, why not just NOT turn it over and not have to give away £90 million a year for a century? So, it begs the question.. is someone from the UK side benefiting from this no-bid deal?
Give the island to me, and I won't charge the UK to have the base.
As I understand it there were no Maori before NZ was settled, that culture formed there from the Polynesian"moa-hunters". Some descendants of those settlers became the Maori, but that a different claim.
Furthermore, there are just generally very few records so I think it's very difficult to make definite claims like you or GP do.
https://en.wikipedia.org/wiki/.su
See also '.yu' and friends, which have already been deleted.
I tend to trust my government (Canada) and I appreciate that WHOIS information is hidden by default for .ca domains. I live here and always will so it seems fit to use the national TLD for representing myself and my work.
Literally every single TLD is administered by a government.
.com itself is under jurisdiction of USA and operated by Verisign
Barely. The NTIA gave up all their leverage over .com in 2018. The only thing the US can do at this point is let the cooperative agreement auto-renew to limit price increases.
I wouldn't be surprised if the US withdrew from the agreement altogether at this point. Then .com would fall under the joint control of ICANN and Verisign.
False. I’m not sure what you’re trying to assert, but governments don’t necessarily need to control/admin gTLDs, and as far as ccTLDs go, they’re under jurisdiction of the corresponding nation, usually, but they’re going to be “administered” by a tech company that holds a contract.
Anyway, “.com” does indeed answer to U.S. jurisdiction, despite being technically a gTLD, but registrations are not restricted to US-based entities. The main things that keep “.com” associated with the USA include the history/legacy of this quintessential “original” domain, as well as a general support from major countries that provide a “second-level” commercial domain, such as “.co.uk”.
... which is a problem lately ... and may have been even in the past for some niches ...
I'm not a customer (wouldn't buy my domain overseas) and have no solid opinion on GoDaddy besides that I hate the name. I hear the horror stories also. I'm just wondering if this is a knee-jerk reaction
Network Solutions still exists.
Nobody gives a shit about how many good outcomes between incidents there are. They care about how many good hours happen between incidents, and they care how big the incidents are.
So if you make a tool that your coworkers use 5 times as much as the old process, that tool better make things at least 6x more stable or people will start talking about how the process fails 'all the time'.
"all the time", as near as I've been able to figure out, after people have been yelling at me, my team, or a team I'm privy to, is not "every day". No, all the time just means that it happens every couple of weeks and one time happened twice in one day, twice in consecutive days, or with two customers in rapid succession. Usually the day they're screaming about.
So if you're doing that thing every day all day long, where you used to do it rarely, but you made some progress on making it more frequent, nobody cares that it's every 100th run that fails, when it used to be every 10th. They just see the drama has gotten more frequent (and nowhere near as frequent as their narrative says, but you've already lost that argument)
GoDaddy: I am so sorry about that. I can offer you a one-time coupon for $10 off your next purchase or renewal. Would you like me to apply this to your account?
---
Most companies just hope an apologetic zoom call is enough to retain your business, and most of the time it works. Not enough has been written about the asymmetry of your SLA credits to your revenue impact for a given vendor outage and how that should guide your build vs buy decision framework.
SLA’s are generally more helpful for getting out of long term contracts with unreliable vendors than actually making up for revenue lost during an outage.
If you have 100% SLA credit under 99% availability you can't aford to be less than 99% available and I know that your SLA means something to you, not just an aspirational bullet point.
> This block was the result of a communication error between Zoom’s domain registrar, Markmonitor, and GoDaddy Registry, which resulted in GoDaddy Registry mistakenly shutting down zoom.us domain.
Markmonitor is used by some fairly large corps and web properties. It’ll be interesting to find out exactly what this miscommunication was.
(I'm not affiliated with either, but happen to know the technical details of the outage)
I had Google Domains for years, until they abruptly and bizarrely abandoned it, then I left for Porkbun. Never had a problem with either of them. I get yearly auto-renewal notices. Everything works, and it’s very boring, which is precisely what I want from a registrar.
One of the really nice things about the service is they handle a lot of the general business continuity and security stuff that can really suck with traditional registrars. One of their main lines of work is they’ll work with you to resolve tld-squatting and typo-squatting by working directly with the registrars.
Even before an infinite number of vanity or scammy tlds started showing up it would be pretty difficult to find <your-growing-unicorn-startup>.biz to add to your portfolio of domains since the owner may just have forgotten to update their email in their registrar and were coasting on a 10 year registration. Maybe the squat was intentional and it’s now a 1:1 replica of your homepage with a phishing or other credit card scam going. Stuff like that really sucks to do yourself while handling your other responsibilities. MM was pretty successful at getting in touch with the owners in the first place and having the registrar yank and transfer in the latter case. YMMV of course.
Once a lot of tlds started showing up, and especially the porn related ones, they worked with the new registrars directly (like GoDaddy in the .us case here) in the “sunrise period” to make sure something like google.xxx doesn’t become a front page article about an actual porn site (in case you’re wondering, that one doesn’t go anywhere at all). Your other options are to work directly with each registrar or ICANN.
I found them surprisingly easy to deal with, and happy to have me on record that my toy domain had nothing to do with either their client or any money. I assumed as long as that remained the case I would never hear from them again, and for the decade or so longer I kept the domain, that was exactly how things went.
Oh, well. It's been a long time since I was so naïve as not to do a quick informal trademark/brand search before I register a new domain, so I don't really expect to hear from them again any time soon, either.
What services are they offering that makes them attractive to corporations?
Mark Monitor will apply a lot more filtering to complaints.
Ironically this is allegedly what happened in this case, a complaint about the domain got it taken offline.
It's like, as I'm sure I'm paraphrasing from something I read God alone knows how many years ago, if your publicist lets you walk into a press event with a giant blob of snot hanging out of your nose. There surely is a reason why that error occurred, and it probably is at least a pretty good reason. But no one is very surprised to see the intro invite from your new publicist.
It isn't a relationship you blow up on a whim, but Zoom that can't route call traffic is Zoom that's not generating revenue, and while the reputational impact is negligible if it happens once, it had really better happen only once. Zoom is the incumbent; no one remembers they were revolutionary once, now everyone only notices the parts they don't like. (Being a skilled but politically naïve sysadmin is much the same.)
Basically, this is why Ma Bell - which had about the only stronger possible "uptime" expectation, in that no one uses Zoom for 911 - was so uptight you couldn't even plug in a modem until about five minutes before divestiture, and specified everything down to the number of turns in the splices their technicians made. There was a fad among programmers, when I was a child, to consider such practices stodgy.
https://www.archdaily.com/973183/the-building-that-moved-how...
What MarkMonitor can provide is things like facilitating RegistryLock, which makes it even harder for changes to be made. And account reps that know what's going on. I hate working with account reps, but if they're knowledgable and easy to work with, it's ok.
They do some trademark monitoring (thus the name), if you want to get your own related app taken down from Google Play :p (I'm not bitter, it was amusing). And presence services if you need to hold a domain in a weird location that wants a presence, they can probably arrange it, which is handy at times.
I'd love to know more details on this incident, MarkMonitor had a bulletproof reputation as a registrar that won't fuck up. Godaddy doesn't, but then I didn't realize they had taken over the contract for .us
(As opposed to a DNS server, including root servers - and even then DNS has provisions for downtime, not to mention redundancy in configurations)
If it is, you can buy custom insurance for the event from an insurance company, and pay the same kind of yearly fee.
And remember that with build vs buy, what you build will often be worse than what you buy, because at least what you buy is getting bugs fixed from bug reports across the world from other customers. An internal tool will rarely be as stress-tested and battle-hardened as what you can buy.
ServerHold is used with Registry (GoDaddy in this case) is disabling vs ClientHold is when registrar is pulling the plug (MarkMonitor)
So what would have MarkMonitor said to GoDaddy to cause them to ServerHold a domain?
It turns out that they had typo’d 12 into the request type field instead of 1, and type 12 was “Covid lockdown protocol with security enforcement” leftover from 2020 and latent in their systems.
Depending on MarkMonitor have chosen to integrate with each other to handle the sort of trademark management that is MarkMonitor’s premium offering, either or both parties could have simply been off-by-one or typo’d in a transaction to cause this. It’s absolutely plausible to create a confusing nightmare outcome with a one-byte error. (And we’re having quite incredible cosmic rays today, so I hope they’re using ECC RAM!)
serverHold is generally only set by registry when they have some pending action which almost always legal related.
You can see a list of Status Codes here: https://www.icann.org/resources/pages/epp-status-codes-2014-...
When this outage happened, I assumed that they finally “made the switch” over but something went wrong.
Something I heard is that there was a Twitter account @zoom_us that was also deleted today.
Since when did we accept, as a society, guilty until proven innocent? I recognize GoDaddy is not the government - but this is unacceptable. A human spending 3 seconds looking at the domain would understand it's a false-positive and should not be removed.
At least since the Digital Millennium Copyright Act.
IE, it explains what DNS is, but it doesn't explain why the outage happened. Instead, it merely gives a timeline with a lot of context that's useful for someone who's still learning about what DNS is and how it works.
I’ll never deal with a .us domain again, even if it means missing out on a good text string.
See for example that AWS Route53 uses com, org, net and uk domains for the nameservers.
It’s not DNS
There’s no way it’s DNS
It was DNSMost likely all are running some version of Bind, or something custom.
Now "ZOOM" was supposedly based in Canada and they were supposedly giving bargain-basement fares to Americans as well, from select origins to select destinations. All I needed to do was to get to Lindbergh Field (San Diego International) and ZOOM Airlines would fly me to London Gatwick. And their aircraft had cute friendly livery with big "ZOOOOOOOOM" lettering on the side. And the price was totally cheap.
Well they did their job fine; I landed in Gatwick, took a train to Heathrow, and flew on Iberia into and back out of Barcelona. Unfortunately, before I departed, my father phoned my fiancée to break the news that "ZOOM Airline" was bankrupt, and all their flights were grounded. They had run out of fuel in Scotland, and nobody would top up the tanks. My return ticket from London to San Diego was worthless.
So Dad puts me on a British Airways flight and I got home safe. But from August 2008, or before, I have harbored a visceral animosity towards any foreign actors named "ZOOM".
Rather this does not sound like a communication error unless they are leaving out a lot of critical details and context or the domain management interface has been de-frictioned and dumbed down too much.
To me a communication error implies someone followed erroneous instructions without asking the obvious, " ... but isn't this a big business that is still live and why don't I have a legal order in my hand?" In fairness this did happen recently with he.net because a sub-domain was reported but it was done intentionally even if they failed to do even basic due diligence. After Covid I would expect most people would know zoom.us would be in use by a lot of people whereas only specific groups of people would know what he.net is.
I am curious if the process has changed due to laziness and now registrars can just select any number of domains and click a button to place them on hold without management or executive approval. If so that should be in some audit trail and should require confirmation and approval by a senior leader.
So 51% of the time it’s weird, but not the same weird.
https://www.markmonitor.com/blog/2024-markmonitor-year-in-re...
Their domain expired because at some level people made some pretty boneheaded mistakes.
Whomever their actual registrar actually was (GoDaddy it seems) stopped pointing the zoom.us nameserver record (NS) at AWS Route 53 which Zoom obviously uses.
% dig +short zoom.us NS
ns-387.awsdns-48.com.
ns-1137.awsdns-14.org.
ns-1772.awsdns-29.co.uk.
ns-888.awsdns-47.net..us (and other many TLDs) uses EPP to communicate between registars (MarkMonitor here) and Registry (GoDaddy). It is probably an admin error rather than code[1], some manual approval or other human review workflow for high value domain and someone clicked/filled in the wrong value at GoDaddy or MarkMonitor would be my first guess.
[1] would have been observed and fixed long before today, transfers happen all the time after all
I'm guessing you already know, but for the others: This is precisely what the DNS protocol was created for.
Another added bonus of domains is the potential for subdomains to be used. This could be usful for many purposes: as load balancing/pooling mechanism (fictive example us4.zoom.us) and for compartmentalisation (api.zoom.us).
Why Markmonitor is terrible: https://news.ycombinator.com/item?id=43712299
Why Zoom is terrible: https://news.ycombinator.com/item?id=43712438
If you think it is not enough to call them terrible, reply to these comments as to why not.
https://www.inquisitr.com/markmonitor-sends-false-copyright-...
https://torrentfreak.com/record-labels-defeat-false-dmca-tak...
https://torrentfreak.com/court-dismisses-charters-claims-of-...
https://torrentfreak.com/hbo-wants-google-to-censor-hbo-com-...
https://torrentfreak.com/after-4-years-copyright-holders-sti...
They haven't even always been good to their own clients: https://torrentfreak.com/anti-piracy-lawfirm-defrauded-right...
https://www.theverge.com/2019/7/8/20687014/zoom-security-fla...
https://citizenlab.ca/2020/04/move-fast-roll-your-own-crypto...
The endless stream of news on privacy problems and vulnerabilities that have come to light since then have only made me feel better about that initial opinion.
https://techcrunch.com/2023/08/08/zoom-data-mining-for-ai-te...
https://www.tomsguide.com/news/zoom-security-privacy-woes
https://cybersecuritynews.com/zoom-app-vulnerability/
https://www.theregister.com/2024/02/15/zoom_privilege_escala...
https://gbhackers.com/zoom-security-update-patches-multiple-...
https://thecyberexpress.com/multiple-zoom-vulnerabilities-de...
What a blunder.