Mozilla site down due to "overdue hosting payments" [fixed] | Hacker News Reader

Mozilla site down due to "overdue hosting payments" [fixed]

linuxmom.net

161 pointsmotownphilly102 comments

Please don't read too much into this ;) We moved from self-hosted Discourse to hosted Discourse. The transfer was initiated late from the Mozilla side (my bad) and the automatic system from Discourse kicked in.

You work for Mozilla?

Yes

Please tell guys who is responsible for js engine to fix handling of large cookies that google is using to slow down youtube on firefox. It is impossible to watch youtube logged in because of that. God bless you sir.

I really only use Firefox and I’ve never had an issue with YouTube logged in and I use it a lot.

What's the bugzilla number / link ?

WFM. You might want to either ask help on support.mozilla.org or file a bug on bugzilla.mozilla.org if you have a step to reproduce.

As others said, having more details would be helpful (or a bug report on https://bugzilla.mozilla.org/).

In general, we quickly act on obvious bugs/regressions on major websites. Often, such major issues on big websites are caused by certain add-ons or antivirus software.

and it is now fixed!

Thanks for the explanation!

Is there any chance to move away from the Discourse? It's a bit too slow (on any page opening), but the biggest issue is its hostile habit of catching the browser "find in page" hotkey (replacing the local find with the remote site search).

I hate that feature with an intensity that's hard to describe.

I know you can press it again or something but for the love of $deity don't fuck with the defaults.

That said I'm equally angry my browser so happily allows this.

Sure if it's a proper web application like OnShape then sure, override default key bindings, but ask me first and remember my choice. If I say no then just don't feed those keystrokes to the webpage.

> but ask me first and remember my choice

This. Just this. I'd say websites should be able to "offer" their own search, so when you do find-in-page it shows the default search with a button to "change to custom website search ([ ] remember)" or so.

Hint: on most websites, you can press Ctrl+l to focus the address bar, and then Ctrl+f isn't hijacked by the current page any more.

Still wish pages would stop hijacked hot-keys in the first place, but hope this helps.

While I do understand the frustration, I think the "find in page" hotkey isn't a big enough reason to change the platform. Myself, I am happy with the performances. By the way, what would you replace it with?

With all other recent news from Mozilla (large scale firings, multiple leadership changes, the new ToS and removal of the promise to never sell our data...), I won't read too much into it but simply add it to the list.

https://news.ycombinator.com/item?id=43185909

Given Mozilla's continual frittering away of cash, would it not show some constraint to not pay for "cloud" hosted stuff for things that could easily be hosted by Mozilla and probably for less (with a less absurd choice of software) - it's already pretty much game over anyway, as a long time defender of Mozilla it is impossible these days to argue.

Why is a SaaS locking out a fairly high-profile customer like this, and publicly airing B2B accounts-receivable drama, to the customer's community?

How big can the bill be, for something you could run on a 20 year-old PC?

Given the strange coincidence with the social media outrage over TOU fiasco, I don't know who to be scratching my puzzled head at here.

It's a pretty good way to get the bill paid to be honest

I also run a B2B SaaS and I have found that over time the customers who I think are going to be terrible for paying, typically tend to be very good at paying and the customers who I think are going to be very good for paying (in that they have good standing in the community in things like that) tend to be pretty bad at paying on time (and with NET30 and all)

Ultimately I don't really understand why I shouldn't be paid on time (I've got to live too, and I have suppliers likewise) so I think this is pretty fair game, I sold you a product under the promise that you would pay in 30 days, if you're incapable of doing that (ignoring exceptional reasons) I don't necessarily want to go and lend you extra credit just because you're Mozilla

Having a terrible accounts receivable department isn't a exceptional reason.

Because if you don’t, they might go somewhere else.

Losing a non-paying customer does not sound like a bad thing.

If it's the first time in 10 years and they're in your upper tier, then perhaps it is.

I'm totally with you on wanting to get paid in a timely manner. (I used to do NET 10 for consulting invoices.)

But you'd do a courtesy call before cutting off a B2B customer? Even just to make sure they got the bill, and that a payment they made hadn't mistakenly not been credited. Which would be an opportunity for an immediate funds transfer or CC charge, if necessary to keep the site up. (Also an opportunity for them to say, "Actually, if you could just make it read-only until the news cycle is over..." :)

They will tell you it must have slipped though the cracks, and that they'll get right on it, but no payment will materialize.

That doesn’t mean you get to skip that step though

It's the age of automation. They likely got more than one courtesy automated reminder e-mails. If one doesn't respect automated reminders, one will respect automatic service cut-out.

Exactly. Which is why I said "Which would be an opportunity for an immediate funds transfer or CC charge, if necessary to keep the site up."

I'll work collegially with customers/partners who I think are being honest, or who can be nudged towards that from a somewhat different industry convention. But I wouldn't take "the check's in the mail" from someone who I thought was being dishonest. And an immediate payment would be appropriate for even honest people to do, in light of neglecting to mail a check in time.

We've been overdue on something like this, but a public notice would have left us moving to a competitor.

It's easy for it to happen if a colleague is away for a month, or has left, and the bill isn't being seen by anyone. Do you read (do you even receive?) out-of-office replies to your invoices?

Of course it shouldn't be on an individual's email, but with annual billing it's easy to overlook.

IMO if I lose someone's business because they can't pay the bill... Good? Right?

Is this a case of can’t pay? It sounds more like a error. So you would loose out on future revenue.

You'd lose out on headaches from your customer not paying and having to waste resources following up on it.

If they can pay, then just pay for the service they got.

Seems pretty straightforward.

Yeah, but you also lose all the income from that customer to save a few bucks for sending an extra invoice…? The customers who pay a little late have never been a noticeable cost for us.

The ones that contact support for every little thing or those that have a million strange feature requests on the other hand.

> lose all the income from that customer to save a few bucks for sending an extra invoice…?

Maybe they sent all the extra invoices? It sounds like you are assuming discontinuing the service was the first and only thing they tried.

Once they figure it out, they can come back as a paying customer. :)

It's not the provider's responsibility to help their customer get their sh#t together.

This is a fine attitude to have if your problem is that you have too many customers.

The opposite is not a fine attitude to have especially if you have too few customers. Once they know that they can ignore payments by using that kind of excuse, something very bad will happen to your subscription revenue.

You typically get multiple reminders. If that's "an error", then that's an error that will happen again and again and again.

That's a problematic customer you don't want unless you're otherwise broke or want to keep them for other reasons (e.g. publicity).

Multiple reminders to the same person isn't much use if he's in hospital, and you're ignoring the out-of-office replies.

The time this happened to us, I was pleased the company looked at our website and phoned the receptionist. We paid the bill within an hour, and moved the responsibility to the finance and IT teams.

Maybe there are too many customers who just ignore bills for services they no longer want to use, I don't know, but given these companies usually have a sales team it would seem worth the time to make a call or write a personal email to an existing customer to maintain their custom.

I know of a company that missed its yearly Microsoft Exchange Online payment. After four month Microsoft cut of their email. Which one could argue is also a "public notice" because emails bounced. (That did the trick and the company paid immediately.)

You are telling me:

1. Microsoft should have tried to find a different way to contact and reach out.

2. That you would have moved your eMail/Groupware to someone else because of that.

To each their own.

A sales person from Microsoft emails us every 3-6 months or so trying to upsell us on a more expensive 365 subscription.

I think their time would be well spent contacting a customer who hasn't paid.

Blocking sending email would also be a more reasonable step before blocking receiving email.

And yes, I'd consider moving.

Or if your marginal cost is small (eg, if your primary product is software) so a slow or delinquent payer doesn't actually cost you anything.

At least where I live & work, I have to pay taxes the when I invoice, not when I receive payment, so they cost me 20% of the billed amount, at least temporarily.

> It sounds more like a error

It seems like Mozilla has made quite a lot of errors in the past few years.

Bad e-mail management is not a responsibility on my part, and a disorganized client moving to another company is not really a threat either.

You’re making assumptions with details we don’t have. For all we know Mozilla is eight months behind on payment and this happens regularly.

In your scenario, the hosting company’s reaction seems extreme. In mine, it seems mild. The truth is we just don’t know and should refrain from bold defences until more details are in.

> It's easy for it to happen if a colleague is away for a month, or has left, and the bill isn't being seen by anyone.

Why TF would important things like that go to a single person's individual mailbox?

Take a couple hours to set up a canned ticketing / helpdesk thing or something, and have bills and other vendor alerts go to that email alias.

Because the companies make it easy to sign up with an individual's address, and that's often fine to test out the service.

Some then make it difficult to use a more appropriate address.

It's not ideal, but there's a huge amount of IT that doesn't follow the official policies — like all the teams using AWS rather than going through their official procurement procedure.

Suspending a service is one thing. Advertising that it’s due to non paid bills is inexcusable.

Also consider how many bridges you would burn if it’s in error. Or it’s a disputed payment.

This is probably how people who never pay their debts think.

"It's ok if I am late with the bill, they will feel bad for me and never risk me reputation damage by airing out my debts".

But in the absence of a sense of responsibility or obligation, shame might be the only thing that might work to make them fulfill their obligations.

If the bills are indeed not paid and the reason I don't see why it's "inexcusable". As long as you're not claiming they can't pay them or haven't paid them out of malice without proof you're just being truthful.

Keeping quiet when it's something simple is more weird to me. God forbid someone assumes massive corpo X is going bankrupt because of a web hosting bill..

No giving notice is public service against fraudulent organization. It indicates clearly that only proper way to operate with such organization is payment before service is provided.

Paying your bills should be first priority. No excuse, no crying. If you fail, fully own to it and terminate the chain of command involved.

Some companies love to blame vendors for their own non-payment. The blameshift works when users only have the company's (the actual customer's) word for it.

Having basic transparency like "this service is in read-only due to nonpayment" really helps internal users to realize the company is being a bad customer. Which then pushes the company's reps to actually pay for the service.

So, no it's not inexcusable.

Leaving your vendors hanging is literally taught and practiced among certain circles of entrepreneurs. Though not spelled out in explicit terms, they operate on the basis of "Fuck you, I got mine." They want a perpetual jackpot machine, and will stomp on anyone to get what they want, more money in the pursuit of even more money. They don't operate on the old mantra of "the boss is the last to get paid."

You'll be able to find them a few years down the road complaining that the mean old IRS is being so unfair because of the little bit of tax evasion that's totally not as bad as those real scammers on welfare.

I couldn’t believe this either until I saw it with my own eyes.

The entitlement is strong with some people. It is "a" force. But more tied to mediocrity than midiclorians. . .

I learned that lesson at the first place I worked. It was an accountant's office and the partner in charge of paying invoices would literally go through the outstanding ones and see which he thought he could get away without paying, even if they were overdue.

So even companies you might think will pay promptly, don't always do so.

+1 to this.

My employer can be really bad at paying NET30, especially on fluctuating "pay for what you use" bills due to all the bureaucracy. Since it's not a fixed amount it can't be rubber stamped by anyone... And dear lord help you if you issue an invoice followed by credits separately...

We always pay the bill, but the purchasing employee has to review it (they're the only ones that can evaluate the claimed usage), then the department head has to approve it, then department finance processes it.. the division finance has to then rubber stamp it... Then the controller had to release the funds.

I see notices all the time where we're ~3-5 days late (often times the money has already been sent, but hasn't hit the sellers account yet)

Totally agree but at least I don't know more about how long this has been developing for and what attempts have been made at curing it - if this is indeed the intention... There's a thread here suggesting the idea that it's a stunt to get extort more money from donors. While I wouldn't go quite as far, I think it's undeniable that there's a rift between long time users of Mozilla products and their current strategies (or at least their tactics if they don't have a strategy). I still believe Firefox is the best browser out there, mainly because they've resisted the temptation to take ad blocking APIs away but having to disable all suggested xyz, save to cloud, experiments, user metrics, ... Just takes a while and leaves you with little trust.

Exactly.

I feel a good desire to fork the browser (ff or one of its forks) to be able build it for myself with a set of debloat patches. Just to avoid worrying about tracking and the "sell your data" stuff.

Well you don't know how long it has been unpaid for and I think it's good that it's transparent so that at least there is no confusion about why it's in read-only mode

> How big can the bill be, for something you could run on a 20 year-old PC?

Easily 7 figures, possibly 8 figures. Enterprise sales are a weird realm.

The only thing Discourse said was "This site is in read-only mode because there are overdue hosting payments". It's a system message that's automatically shown when the customer forgets to pay their bills.

The drama comes from Mozilla's nonpayment, not Discourse's policy. Weird to try to pin the blame on Discourse.

It makes sense to correct the title. Currently it says „Mozilla site down…“ but it is not the front page and not „down“, so most people using Mozilla products are not affected.

Better version would be „One of Mozilla sites is in read-only mode…“

Organization with hundreds of millions of dollars of annual revenue and spend can't maintain servers?

I would wonder if this is really something much simpler: an excuse to make things read-only, while implying that people really should give them money if they want things to work.

Really, I can't lose any more respect for Mozilla at this point. It's all gone.

If you have ever done subcontracting work or been a supplier to a major automotive corps, you’ll know the can be extremely late to pay at times. They use their muscle to get a free line of credit from their suppliers by simply demanding to get an extra 30 days on their deadline every month.

Not saying this is Mozilla’s policy, just pointing out it may be an accident or it may be a lack of funds, or something entirely different.

I believe in coincidences. Coincidences happen every day. In a world that operates largely at random, coincidences are to be expected. But I do not trust coincidences.

I’ve done things similar to this as a purchasing manager (in a different industry). Providers sometimes required a payment schedule that was critical to their cashflow (and I did feel very sympathetic), but I had to pressure them into a very unfavorable schedule - otherwise the contract would never be approved by finance. Small companies sometimes had hard time understanding that there were rules I simply couldn’t bend, and it was rather unpleasant at times.

However, that was always sorted out during negotiations.

> supplier to a major automotive corps, you’ll know the can be extremely late to pay at times.

I'll add telcos to this list. Absolutely the worst customers to have. Net90 paid on time was a good thing.

I guess the money stops flowing when you di not need a monopoly fig leave, because the monopoly busters are exited.

I suspect the moral of the story is devops and operations staff need to keep all payments, secret rotations, certificate expirations, etc. on a calendar so no one forgets to pay the bills. I hope the employee who forgot to pay the bill learns from this.

Ops is an interesting domain, nobody knows what they do until they don't do it; so everyone wants the jobs to disappear, and when they do it's fun what happens.

I'm beyond certain that someone will reply "use a cloud provider", which is ironic, as cloud providers just concentrate these kinds of ops people and charge you through the nose (often an order of magnitude more in my experience) than having people responsible.

Unpopular opinion, probably, because people seem to like dehumanising operations issues on this site- sure, things can be automated, but at some point there's got to be some responsibility.

^This is the most sensible understanding of the space, in my opinion

I agree, ops is like a toilet. Nobody cares, until it does not work.

If a technical person is even close to involved with paying for a SaaS Discourse forum hosting something is extremely wrong. This is part of business operations or finance. This is already a "cloud provider". It's the same as paying for Google Workspace or an HR platform.

I would expect more professional operations considering the very high salaries of Mozilla foundation management and I would not blame any particular employee for this problem.

You do know most companies in the world have a department specifically for handling financial matters, right? (Hint: it's not the IT department)

Also, I don't know if you're aware of this, but technology has advanced sufficiently that nobody needs to manually pay a monthly bill anymore. I know it sounds crazy, but there's these things called "bill pay", and "recurring credit card changes", that have existed for 20ish years now. Might want to read up on the latest trends!

More likely no-one "forgot" to pay. Typically administration optimizes payments. Getting a large corp to pay a small contractor or service provider at all let alone on time can prove to be a nightmare.

> This site is in read-only mode because there are overdue hosting payments.

Coinciding with techie social media outrage over the TOU fiasco, and repeatedly mishandled by corporate, so now the read-only mode means the outrage can't spread to Mozilla forums, where it might reach a wider audience?

The title should be:

"Mozilla's Discourse forum is in read-only mode due to overdue hosting payments"

The current title gives the false impression that mozilla.org is down.

discourse.mozilla.org is an alias for mozilla.hosted-by-discourse.com.

Given that the discussion has moved to late payments generally, a question for freelancers and others who are subject to this:

Would you be up for a service which allowed you to automatically share intelligence on which companies are late payers? Benefits would potentially be:

- Find out in advance how late your will be paid

- Early warning of creditors doing aggressive cash management (aka going bust)

The idea would be that it would be funded by selling information about payers (not payees) eg, to insurance companies.

NB the obvious way to implement this is to get access to bank transactions, which I know requires a lot of trust. But maybe with eg the Open Banking API, there's some way to do it such that you can trust that only the right information is shared.

I learned the hard way (major outage) back in the 1990s that no accounts payable or accounts receivable should ever be tied to a single person’s email address. You should use a shared mailbox account like ap@example.com when signing up for all services, or supply that as a separate billing address if the vendor allows. Then you make sure there are always at least three employees monitoring that mailbox, even as people come and go.

Very unprofessional of Discourse to publicly shame a client like this. If they are not being paid and they care so much about the cost they should bring the website down instead of resorting to petty shaming.

I think we should have more public shaming of companies' negligence. If you don't pay your bills you're publicly shamed in your credit score.

Also consider that those types of things happen after several rounds of ignored warnings.

As reported by parts of the tech press this last year, Mozilla is seemingly on its way to certain bankruptcy.

I guess we should expect more incidents like this the time to come.

> Mozilla is seemingly on its way to certain bankruptcy.

Mozilla has hundreds of millions of dollars in cash and over a billion in investments according to their last financial statement -- so, no, they are not.

It doesn’t matter much what they have in cash and non-liquid assets if they don’t have a balanced budget.

90% of Mozilla’s direct income (cash) comes from Google. And it looks like that funding may go away soon [1].

If Firefox had a big marketshare they might be able to capitalise on that, but right now it’s the lowest it’s ever been, and they keep on burning bridges.

[1] https://www.zdnet.com/home-and-office/networking/why-googles...

More of you switch to data-harvesting Chromium derivatives and scams, and we'll have no true alternative browser, the only one that is Manifest V2 compatible.

> the only one that is Manifest V2 compatible

Could you provide a reason why that is a feature?

All the extensions I’ve implemented runs fine as V3 manifests, and they allow much more fine-grained security permissions. That’s something I’m happy with, even as a developer. It limits the security impact of whatever flaws my extension may have.

> How Does MV3 Affect uBlock Origin?

> uBlock Origin relies heavily on the webRequest API to block unwanted content before it loads. Under MV3, the webRequest API is limited, and extensions are encouraged to use the new declarativeNetRequest API instead. This new API allows for predefined rules but lacks some of the dynamic capabilities that uBlock Origin utilizes for advanced content blocking.

https://ublockorigin.com/

Same for NoScript.

Fair enough. But it’s clearly a trade-off.

Some may want extensions with unchecked, unrestricted permissions to be able to do all-the-things (tm).

Me on the other hand (and possibly others), with open-source projects often going haywire and crazy over time, feel more comfortable knowing that extensions cannot do more than I have permitted them.

I guess this is for the market to decide. If V2 provides enough value for enough users, it will stick around.

> If V2 provides enough value for enough users, it will stick around.

Unless these snowflakes enter panic mode because "Mozilla is selling all my files" and stop using Firefox for reasons that cannot be explained by logic, which I see is majority of news unfolding after Mozilla changed their wording.

The comments here are hilarious. So many people getting on their soapbox about someone forgetting to pay a bill. Something that has undoubtedly happened at your organisation, and happens at most, given enough time.

This website fucking sucks.

yep so much manufactured outrage and fake drama on several stories today.

Happens to the best of us

This doesn't feel like an oversight or an accident. It'll be interesting to see how they try to explain this away.

Why? Are you getting bad vibes or something?

It is way more likely to be an oversight.

Look for comments from "sylvestre".

Reply on news.ycombinator.com