Hacker News Reader

Comment by sytse | Hacker News Reader

At GitLab our fallback from Slack is Zoom https://about.gitlab.com/handbook/communication/#emergency-c...

I'm posting this because I found a lot of people don't know that Zoom includes a complete chat client that includes channels.

And #HugOps to the engineers at Slack working on this. I appreciate that they posted a periodic update even when there was no news to report: "There are no changes to report as of yet. We're still all hands on deck and continuing to dig in on our side. We'll continue to share updates every 30 minutes until the incident has been downgraded."

View on HN

I'm quite surprised that you don't use Mattermost as a 'Slack fallback' at GitLab.

Indeed, if you champion FOSS, why would you recommend a proprietary piece of software as fallback?

My point was really that it's GitLab's own product. "GitLab Mattermost" [https://docs.gitlab.com/omnibus/gitlab-mattermost/]

I'm amazed they use Slack at all. Let alone as a fallback.

No I'm pretty sure that's just a sort of 'integration', Mattermost shipped with GitLab?

https://about.gitlab.com/blog/2015/08/18/gitlab-loves-matter...

> Like many companies in the last year we've switched to using Slack to improve internal communication. [...] Since Slack doesn't offer an on-premises version, we searched for other options. We found Mattermost to be the leading open source Slack-alternative and suggested a collaboration to the Mattermost team.

I'm not really sure why it's 'GitLab Mattermost' and not (at your link) 'GitLab Nginx' et al. though.

Ah I see now, having read more of the history. Calling it that seems pretty misleading/odd.

(We use GitLab and Mattermost (integration) where I work. I've been 'remote / WFH' for the past 7 years.)

I agree it can be improved and created https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_request... to do so.

They posted a giant list of the services they use recently.

They use a ton of services.

Likely you don't want your backup to be one of your systems and another part of the company probably uses Zoom already so it is probably easy to fail over to that.

Here is the list of services that we use https://about.gitlab.com/handbook/business-ops/tech-stack/

This includes many proprietary ones, we generally choose the product that will work best for us, considering the benefits of open source, but not excluding proprietary software.

Mattermost is not part of the single application that GitLab is. There is a good integration between with GitLab and our Omnibus installer allows you to easily install it. But it is a separate application from a separate company.

Or use mattermost with Slack as the fallback

Maybe my DevOps folks should not be privy to all internal communications?

That is one reason we did not go with Mattermost.

If you do not trust your own devops, why are you trusting someone elses devops?

It's just due diligence. Think of what you have access to if you have "god mode" on corporate chat: HR, the CFO's DMs, private messages between other coworkers, and so on. Most won't fall for this temptation, but even those with strong anti-spying morals can be weakened by circumstances. Best to remove the temptation by design.

Because someone else's devops can't use it against you institutionally. Nor is going to insist on having an opinion on things that they're unaffected by.

This isn't a slam at devops, it's about the need for institutional information hiding; not everyone needs to know about and weigh in on every decision being made.

We structure our company similarly. With effort DevOps is god on everything except HR, Sales, Finance, Chat, and C-level management which are operated with 3rd party services controlled by the individual departments and "owned/managed" by the C-suite.

Low maturity risk management functions.

Slack won’t protect you from this as it’s possible for admins to export even private DMs.

https://www.nbcnews.com/better/business/slack-updates-privac...

Only Slack Workspace Owners can export, not Slack Admins.

You also have to be on the ‘Plus’ plan, otherwise it is a roach motel.

Devops should have nothing to do with your chat server. It should be your IT department, just as with the email server.

DevOps at a lot of small companies also manage the internal IT stack and sometimes even take on most of the IT duties. Once you get larger you start having "IT" as something separate from DevOps but with the actual infrastructure managed by operations. Once you're really big the teams are truly separate and IT owns their own infra.

I'm guessing because they don't want the support burden on a rarely used but necessary fallback solution vs. something plug and play.

This is the reason these "closed" ecosystem apps like Slack/Zoom are multi billion dollar companies and have massive uptake. Simple and easy to user.

As someone who has to use Zoom Chat to interact with a client on a daily basis, please, do not recommend Zoom Chat to anyone except as an example of how not to do chat software.

Though, I do agree wholeheartedly with your sentiment that the Slack team needs all the positive vibes they can get right now.

As someone who has to use Zoom Chat every day, this a thousand times. (We still run an XMPP server on the side just to avoid the horror that is Zoom chat.)

Yeah, this is baffling. I'd rather run turn of the century ICQ than Zoom chat...

I continue to be impressed by GitLab's operations and documentation! While, yes, others may have similar backup plans, as an outsider, it feels like GitLab's handbook seems cooler even if only for their publishing, and making public of their practices and processes. I'll caveat that I'm not really a fan of zoom/slack/hangouts (I'm an unashamed fanboy of matrix and its numerous clients), but gitlab's approach is still really neat! Kudos to gitlab!

Aren't you worried about so many security vulnerabilities found in Zoom?

Unpopular opinion, but WebEx beats the pants off Zoom. Of course, it's neither free nor open. But it does support strong end to end encryption and authentication and has regulatory compliance to a bunch of things, if that's important to you. I get that there is WebEx hate because "enterprise" etc, but we use it around here and it works quite well.

+1, both are a PITA but Webex at least has a really good web client.

Nice tip! Zoom chat is cool (although chats without gifies are way too productive ).

Oh wow -- after years of using Zoom I definitely did not know about this. Thank you for pointing it out!

Is this accessible from the web? Can't seem to find it with this Chromebook.

At Papa we use Discord as backup.

Why not Mattermost or Flock?

Just a reminder that it's probably not a wise idea for anyone to get further in bed with Zoom than they already are.

https://www.washingtonpost.com/technology/2020/12/18/zoom-he...

"Business takes the easy and ethically questionable route to continue making money" news at 11.

I'm not condoning Zoom's actions but this is hardly a problem unique to Zoom. Few if any businesses will stand up for consumers and citizens unless it's directly aligned with their profit motive. In this case, the business choice is to operate or not in mainland China. If they choose to stand up against the Chinese government they're going to have difficulty continuing to operate in China and risk losing that entire market.

Google played this PR game many years ago in China (rejecting some of the governmental policies) and ultimately caved to Chinese policies to do business there.

Businesses are not the organizations we should look to for empowering people, that's simply not their goal no matter how much their marketing team may want to sell that idea by following trending (popular) social movements that they've already done market studies on to assess potential fallback.

I think it's a pretty bold claim to state that Zoom's actions aren't unique.

What other business in this space has given China unfettered access to US users and data? I'm not aware of it occurring with Webex, Teams or go2meeting. The "one rogue employee" thing falls flat pretty quickly when they're the only ones that had this issue.

This feels like their encryption thing all over again, there's an "oversight" that is equivalent to a backdoor that only gets fixed when they get caught.

I didn't realize they shared any user data outside China (misread the WP portion). It appears they did share 10 users' data which is a bit questionable but I'd hardly call that unfettered access to US data.

The fact is all of the US businesses operating in China give surveillance ability to the Chinese government for the Chinese users and are operating in an ethically questionable space being primarily based outside of China, at least in my opinion.

It's really not too different than the businesses sharing US citizen data to the US government, much of which Snowden and others before him exposed. I suspect there's a lot more surveillance going on everywhere than the general public know about and the businesses best positioned to do the surveillance are probably doing it.

And yet a surprising number of firms with sensitive info continue to use it. Law firms etc

Is this an attempt to refute the claim using Zoom is bad, or an indictment against those still using it?

an indictment that so many people who should know better, still use a tainted and non-benign product.

Elaine Chao’s sister is married to Xi, while Elaine, as transportation secretary under Trump, was busted inviting family with business ties to the CCP to official US government meetings.

The fear on this forum is imagined political thriller more than realistic.

Every technologist is grifting off the military industrial complex.

The lesser of two evils and the product just works. They might have a few governance issues they need to fix. But at the end of the day, they signed a BAA with us and will take the liability and fallout of a breach.

Imagine if they did that for the US government, which is easier to compel since they are in US territory.

One nation is currently operating concentration camps and arrests and seizes the property of prominent citizens who criticize the government. Are you sure that's an equivalence you want to draw.

Like Guantanamo Bay or prosecution of Assange for his journalistic work to expose wrongdoing of government? Or maybe you’re talking about for-profit prison system and mass incarceration practices? But you’re probably talking about China, right?

Once again, that is a false equivalence.

No one imprisoned in Guantanamo Bay is a US Citizen and neither is Assange.

The US prison system is super fucked up but it is not the same as ethnic cleansing.

You are comparing apples to concentration camps.

> No one imprisoned in Guantanamo Bay is a US Citizen and neither is Assange.

I think you should I know I -- and probably others, are reading this as "b-b-but, they're not US Citizens, so they don't deserve [the same] rights"

I hope that's not what you mean, because if it is, that's really fucked up.

That's exactly how I read it. And that's probably the same position of lots of Americans, which in and of itself is quite fucked up.

We have thousands of brown people in camps along the border, in brutal conditions, without access to healthcare(unless you count forced sterilizations as healthcare). Do you consider those to be apples as well?

That forced sterilization claim was entirely debunked and was misleading to start with:

https://www.channel4.com/news/factcheck/factcheck-were-mass-...

https://www.snopes.com/ap/2020/09/18/more-migrant-women-say-...

And 70% of those people in those camps are released within 30 days, often times within one week back to their country of origin (or given asylum).

Why are they in camps along the border? Why are the Uighur? Did the "brown people" break any laws? Did the Uighurs?

Are the "brown people" in camps along the border a single, ethnic minority? Are all "brown people" in the country subject to arrest and under surveillance just for being "brown"?

Well, yeah. People _are_ subject to arrest and surveillance for being brown/black in the US.

Not really more surveillance than anyone else. And the discrimination and mistreatment for people in the US is bad but nothing compared to camps.

No they aren't.

> No one imprisoned in Guantanamo Bay is a US Citizen and neither is Assange.

That's a glib retort.

A takeaway from your position is that it's ok so long as you do it to citizens of other countries.

> it is not the same as ethnic cleansing.

See the above.

That's always been the difference between the US and China and why so many countries have hatred for us and yet little to none for China. They don't fuck with other countries on the level that we do.

yes, I remember when I got my trump kidney from a poor anti-fascist liberal. /s

America is fucked up, that doesn't mean that other countries aren't also fucked up or aren't doing worse things with the data they collect.

Yea, but you live here and so you should think about the implications of this for yourself and your countrymen and not through the lens of international competition. That is a distraction.

Essentially, the China case proved Zoom is willing to cooperate with a nation state. The US is the nation state we live in, Zoom is HQ'd here. Therefore, the risk to us is high.

As an aside, the organ harvesting idea comes from the Fulan Gong, who are similar to Chinese Scientologists. It is not clear to me that their claims are accurate.

Yes. The Chinese state and the US state are both proven to spy on their citizens. For reference, see the heroic Edward Snowden's 2013 leaks.

What an overwrought headline, the employee in question has already been fired.

Sorry, but an executive is not just "an employee" and any alarms are rightfully justified. Took a little bit of cajoling in my company but we've successfully moved to self-hosted tools for the most part (Jitsi and Rocket.chat) with just a couple of projects with outside contractors using Slack.

It's weird that you describe the headline as "overwrought" and call the person an "employee" when the headline is more accurate than you.

This was an executive, not just an employee. That's a huge distinction and I can't help but think you intentionally downgraded his position to cover-up his behavior. "Just an employee" "Not a big deal"

But when you read the allegations, they seem like a very big deal that an executive was spying on users, giving their information to the Chinese government explicitly for oppressive purposes, including folks who are not in China, and went out of his way to personally censor non-Chinese groups meeting to discuss the Massacre-Which-Cannot-Be-Mentioned.

I would say the headline understates the gravity (it's very much a 'by-the-books' headline that you KNOW went through ten levels of Legal), and that your hand waving here feels much more dishonest than the headline.

Regardless of intent, it's undeniable that at some point there were insufficient controls to prevent this executive, or any executive in the future, from gaining this level of surveillance access.

And it's also undeniable that the consequences for Zoom (really, just needing to fire a few people, and not even the people who designed those controls if there were any) are so minimal that they have no incentive to strengthen those controls.

For some organizations (mine included) the benefits of Zoom outweigh the risks of Zoom having proven itself to not have those controls, namely the possibility of both political and corporate espionage. As with all things, YMMV.

Not only that, but this line stuck out to me.

> and other employees have been placed on administrative leave until the investigation is complete.

Zoom at least suspects he did not act alone.

It was an executive purposefully brought in for legal compliance with that country's requirements. That he was fired is a huge signal in how seriously aggressive zoom is about protecting data that they would even be willing to go up against national governments. I feel like the firing is a huge part of the story.

The optics are still very, very bad for Zoom. I have zero trust in them.

There are remarkably few organisations I somewhat trust (even then on a sliding scale) but on that spectrum Zoom sits at the "wouldn't touch them with someone elses bargepole" end.

The company in question is still operating. We don't know if the employee was just a scapegoat.

Please reconsider using or supporting Zoom in any way. https://www.nytimes.com/2020/06/11/technology/zoom-china-tia...

Reply on news.ycombinator.com