Zulip – Threaded real-time chat for distributed teams
zulip.com
zulip.com
Here are some case studies from Zulip themselves about specific organizations, how they moved to Zulip and how they are using it:
- https://zulip.com/case-studies/rust/
- https://zulip.com/case-studies/asciidoctor/
- https://zulip.com/case-studies/lean/
I know a bunch of the Rust people hang around on HN, maybe they can fill out any details or tell more about how it's going so far with using Zulip :)
However, once I did get used to it I think it's been great. Organizing chat into defined topics is super useful for following discussions you're interested in, filtering out those you aren't and for keeping conversations going across time zones.
It's more discoverable than chat but not as rigid as an actual forum. I feel it works well once the learning curve has been surmounted.
We have always had a practice of prioritizing feedback and feature requests from open source communities, because improving the productivity of the global open source community is an important part of our mission as an organization.
One of the largest such investments has been the native public access option (no account required!) that we released earlier this month:
https://blog.zulip.com/2022/05/05/public-access-option/
(I lead the Zulip project)
I'm at a point of choice fatigue with communication platforms where Zulip being siloed is more important than it being open source. I'm phasing out proprietary platforms in a way that will take years and may never complete. I don't want more apps, I don't want another user account, and I certainly don't want a bunch of user accounts on different servers.
Behold, the two five year old tickets requesting that they stop passing notification text to Apple/Google cloud messaging:
https://github.com/zulip/zulip/issues/6954
https://github.com/zulip/zulip-mobile/issues/1190
For those who don't know: if you have an app and want to send a notification to a user, you use Apple or Google push notification services. In both cases, the notification is plaintext but transmitted via encrypted channel to Apple or Google, then transmitted via encrypted channel to the user's phone. But in each case, the push messaging provider gets to see the messaging text.
So companies that give a shit about your privacy send a token instead, that says "hey, wake up, something happened, fetch a notification." Apple likely doesn't do anything with the notification text, but it's a given Google datamines the hell out of it.
They also still don't have any form of e2ee. Here's another five year old ticket requesting end to end encrypted chat, where a bunch of people say "here's a long list of apps that support e2ee" and Zulip devs go "gosh golly how do they handle keys? Math is hard."
I'm going to assume good faith in your comment, but I'm really having a hard time doing it, and I'm one of the people who worked for SilentCircle! Another one of those people currently works for Zulip (hey Austin!).
TL;DR: It would take a ton of effort and would basically require a pivot of the business model. Yet you present it as if it's some afterthought that can be tacked on, completely ignoring the fact that all none of their competitors (Slack, Teams, etc) support E2EE.
And therefore demanding E2EE for groupware might seem almost nonsensical, since wide proliferation of access keys makes the whole secret room subject to increasing likelihood of accidental or malicious compromise - ending up more privacy theatre than actual privacy?
e.g. if there are 1000 different individual keys to access a conversation, how private is that conversation really?
However, this does make sense when you consider your attack model. When thousands of people know a secret, maybe forward secrecy isn't worth the trouble, and you can simply have one master key that people share between them (to prevent the server from reading any of the communications).
There are tons of issues even then, though. For example, you can't send an email notification to someone and show them the content of the message that mentions them (because you don't know it), plus how do you even know they were mentioned? There are probably hundreds of small issues like that, and the fact that the server knows nothing leads you to push most of the functionality to the clients themselves, which leads to more problems.
It is obvious that E2EE would involve a big rework of the codebase, but it is neither impossible nor unnecessary. If there are architectural problems with it, I have to question the original developer intent, as a lot of stuff that E2EE requires to work also improve privacy in general.
I of course have no right to demand work from software developers, regardless of the openness of their code base. It's just, that's really too bad.
There's reasons unrelated to this for why you want to not show the message/sender - namely devices like a desktop you might be screensharing from or if you're concerned about someone seeing it over your shoulder.
I almost leaked a message just taking a screenshot because I didn't notice the notification immediately so I keep it disabled.
The push notifications issue is important to me. I'm the one who opened #6954 five years ago, when we were first implementing mobile push notifications, and I've encouraged contributors to pick it up a number of times over the years. I appreciate your bringing attention to this -- it will likely make it easier for me to get folks excited about working on this mostly invisible improvement. https://news.ycombinator.com/item?id=31573940 elsewhere in today's thread has a bit more technical context for those who are curious.
As a bit of side comment, I think how long ago an issue was opened is not what makes it important. About 250/1650 of our open issues are at least 5 years old, and while some old issues are quite important, many of them are still open precisely because they are not important. So while I agree that #6954 is important, my view is that would still be the case if it had been added to the issue tracker yesterday.
I'll post a separate comment about end-to-end encryption of conversations, since that's a complex topic.
(I lead the Zulip project)
Do you want a Zulip where account recovery is impossible by design? Or where conversation history is unavailable to new users [1]? You might say "well let people opt in if they want e2e" but that's not helpful either. If even 1 user in a group is not opted in then the entire group has the same security level.
[0] https://blog.cryptographyengineering.com/2012/04/05/icloud-w...
[1] This must be the case unless you invent a p2p system for users to bring a new user up to date on the group. And even then people could voluntarily lie about past messages. There are neat ways around all this but the point is that this system is very complicated and doesn't currently exist in practice.
First this one [0], which is the problem and then this one [1] how we solve it.
Matrix is good enough to capture the network effects and provide the consolidation pressure the field badly needs.
One thing that I don't see mentioned that is a huge deal that you don't realize at first. The topic for a message can be changed after the fact and by anyone. So you are having a big discussion and at some point it veers off into some interesting tangent (or pointless flamewar). The original person didn't know it that idea was unique and had interest, but afterwards a big chunk of topic_A was actually about topic_B. So later a helpful person will go back and change the topic of the message that started it and all the replies will also move to that new topic name.
This means that someone in another time zone or who was busy during the original conversation can come back later and see the ideas discussed and focus on just one ones that are interesting without having to read huge discussion threads. And the threads can be on-topic because people can prune it after-the-fact.
Yes, this does mean you need to be a "team" and trust each other to make reasonable edits to the chat history. Just like a wiki, it sounds like a crazy idea, but works great in practice.
The whole topics (sub-channels, basically) idea seems interesting as well, though personally the solution itself seems pretty similar to two other great pieces of software out there:
- Rocket.Chat https://rocket.chat/ (the Jitsi integration was especially nice)
- Mattermost https://mattermost.com/ (really simple to setup)
Both of those also seemed serviceable, decently familiar and usable. I guess even Nextcloud Talk could work if that suits your environment better https://nextcloud.com/talk/ though personally it felt a little more barebones in comparison.Has anyone extensively used any of them and can share what details/differences jumped out at them? Admittedly, i probably missed most of those myself and just saw them as similar, self-hostable chat solutions.
Threading feels bolted on in Mattermost, but is the primary feature of zulip.
Think of Zulip as a realtime version of NNTP, and Mattermost as a more modern IRC with a "reply" feature. The "right" way to use Zulip and Mattermost is quite different, though each one can be easily used as an inferior version of the other.
We of course use our own product every day and when I first joined the company in 2019 I also found that threading felt "bolted on". Primarily because channels rendered everything in the same linear feed regardless if it was a message or comment so there wasn't much point in taking the time to first hit reply.
However, Collapsed Reply Thread [0] has been a game changer for me and not only do I no longer feel that way about threading, my productivity would plunge without it. That feature has been in public beta since last summer [1] and will go into general availability in two weeks to officially become the "right" way to use Mattermost. Not to say that it's the same as Zulip, just sharing an updated perspective.
[0] https://docs.mattermost.com/channels/organize-conversations....
[1] https://mattermost.com/blog/collapsed-reply-threads-beta/
I'd also still like to help you setup tools that your team already use if it's still relevant. Please talk to us in the ~Ask R&D channel on our community instance: https://community.mattermost.com/core/channels/ask-r-and-d
I get the point of having topics (less people involved. Kinda like subreddits)... but in Slack I create channels targeted for very specific topics (like the #annual-summit-tuesday-night-catering one) and they work just fine.
- the "topics" are grouped under "streams" that are defined at the organization level. Slack lets every individual set up star groups for channels, but in practice channels are an unorganized mess for all. And, because they have that container, topics are just zero-friction to define.
- Streams, not topics, have membership and permissions. I like this because it means that streams can be defined as an _audience_ for a number of topics. Whether that topic is relevant to you at the moment is a different question, but I often want to be able to skim all of my (e.g.) `team-galactus` topics quickly before moving on to stuff about `greater-denver` or whatever. I also don't want to have to invite people to (or exhaustively document) all of the various topics that they might have a connection to.
"You can do that by changing your code like this: [code] Let me know when you have the results."
"Okay, but I'm going to be busy for the next few days. Will post here when it's done."
{Three days later}: "Your suggestion worked! Let me know how to proceed from here."
You can either keep Zulip open all the time - which makes me cringe - or you can go without any form of notification. Then a week later you decide to check back and see the message, but they forgot to @ you.
I'm working on a similar threaded messages project called Booklet (https://booklet.community), and I think good email summaries are the key to the product. It's hard to do, but there are some cool data opportunities to improve relevancy.
In particular, a lot of preparatory refactoring towards supporting this has been merged over the last year, and we have a contributor planning to focus on this project over the summer.
https://github.com/zulip/zulip/issues/12309 is probably the right issue to follow for now. My plan for today included writing a much more technical version of that issue with a specific implementation plan, before someone submitted Zulip to Hacker News :).
I like that feature. But it seems a pretty small thing. What are the other things that stand out about it? other than being open source.
However, it is still has a ton of bridges and other integrations, and all the shiny things you'd expect from Slack.
Zulip 5.0: Threaded open-source team chat - https://news.ycombinator.com/item?id=30846659 - March 2022 (86 comments)
Zulip Cloud security vulnerability with reusable invitation links - https://news.ycombinator.com/item?id=30479430 - Feb 2022 (35 comments)
Why Zulip will stand the test of time - https://news.ycombinator.com/item?id=29595926 - Dec 2021 (80 comments)
Zulip 4.0: Threaded open source team chat - https://news.ycombinator.com/item?id=27149123 - May 2021 (170 comments)
Zulip 3.0: Threaded Open Source Team Chat - https://news.ycombinator.com/item?id=23860338 - July 2020 (133 comments)
Zulip 2.0: Open source team chat - https://news.ycombinator.com/item?id=19284321 - March 2019 (96 comments)
Zulip Server 1.9: HipChat import and much more - https://news.ycombinator.com/item?id=18400988 - Nov 2018 (123 comments)
Zulip – Open-source, threading-based Slack alternative - https://news.ycombinator.com/item?id=17622987 - July 2018 (99 comments)
Zulip 1.8: Free software Slack alternative with email-style threading - https://news.ycombinator.com/item?id=16863675 - April 2018 (148 comments)
Zulip Server 1.6 released - https://news.ycombinator.com/item?id=14506426 - June 2017 (14 comments)
Dropbox has open-sourced Zulip - https://news.ycombinator.com/item?id=10279961 - Sept 2015 (313 comments)
Dropbox Acquires Zulip, A Stealthy Workplace Chat Solution Still In Private Beta - https://news.ycombinator.com/item?id=7419408 - March 2014 (14 comments)
I don't think I've heard of Gale, nor can I find it now that you mention the name... Which I guess is consistent with your "fell off the face of the Earth" theory. If you or anyone else can find a link, I'd love to read about it :).
https://web.archive.org/web/20020126184926/http://www.gale.o...
https://web.archive.org/web/20011221165350/http://gale.org/e...
https://web.archive.org/web/20020214082814/http://gale.org/l...
https://web.archive.org/web/20011127185333/http://fugu.gale....
I think it was popular in a Caltech crowd?
I think a zephyrgram was a "puff"?
I think there was a fairly sophisticated tiered location/tag hierarchy that is very very broadly related to topic/stream/reaction emoji.
Some of the contributor have usernames that look very familiar (I think that's the same jtr whose username I've seen elsewhere, for example).
https://zulip.com/help/start-a-call
All of our supported providers support screensharing. Our philosophy on this is that having a nice integration with multiple dedicated video call providers is almost certainly going to be a better user experience than building our own video call stack.
Yor suggestion would only be relevant if a problem needs to solved now or never. Or for a completely useless chitchat channel that people follow only realtime but nobody would ever read and reply to afterwards.
Nothing except social pressure prevents people from misusing topics. It happens to all of us, when a different thread gets spawned during a discussion. Sometimes when someone does not know where a question or bug would belong to. But you can move the messages in question to a suitable topic (existing or new) at any time. That used to be a bit buggy in the past, but has been working smoothly for a couple of releases.
Within the same zulip stream everyone can do it (in theory causing edit wars, but it has never happened in our company). To move it to a different stream, you need to be admin. That's sometimes a bit weird, but our admin is quick to do it when discussions went offtopic.
Edit: I can't answer why it was worse in Slack. Luckily I haven't had to use Slack for 4 years. I only remember that it was much worse. Wasn't it so in Slack the user replying explicitly needed to select a threaded reply (and few did so). And even if they had done so, the one coming to work 8 or 24 hours later needed to look through everything (or nothing) because the (possibly threaded) messages had no topic?
You have to choose server-wide between sending none (very private), sending redacted notifications (somewhat private, somewhat useful) or sending the full message in the notification (they see all your messages, but very convenient). Or I guess you can modify the mobile client and set up your own push notifications.
Our plan is to add end-to-end encryption for mobile push notifications, and we've got a design, but limited progress on implementation. One can follow the issue here: https://github.com/zulip/zulip/issues/6954.
For security model reasons, those push notifications need to be sent to Google/Apple via a server operating by Zulip itself. So your local Zulip server needs to share a secret with the Zulip mobile app on your device, and then it can encrypt the push notification's content so that only the destination mobile app can read it (and the intermediaries cannot).
It's a basic cryptography problem at a high level, but there's subtle details around what metadata can be safely encrypted (the client does need to know which Zulip server's secret key to use!), backwards-compatibility, library choice, etc., that make it not trivial, which is why we haven't managed to implement this yet.
Plus, the server already has a shared secret with the client (presumably), the API key. Backwards compatibility might be an issue, if you aren't storing the client version on the server, but that seems like a simple fix.
Can you explain a bit more what I'm missing here? I'm a bit confused.
I agree it's possible in theory to reuse the API key for end-to-end encryption, but I don't think that's the right design. This is in part because the encryption algorithm may, now or in the future, have different requirements for format, key size, etc. But more importantly, a cryptographic weakness in the encryption algorithm should not allow an attacker to stealing API keys.
It might be best to move this conversation to #backend in chat.zulip.org :).
If anyone wants to help, I'd love links to privacy pages that folks really like and/or lists of questions that readers would like to see such a page clearly answer.
Turns out, it's a bit of both, and trying to use it as any single one of them, will probably make it harder to use.
Basically, I treat it as a live chat where it's easier to participate in different topics at the same time, Slack on steroids where everything is a "Thread" basically.
Alas, Flowdock is dead now and I've learned to embrace Slack as the better remaining alternative over Teams or Google Chat.
Other than that... It worked fine. I kinda liked it but slack is more robust for me (even if slow).
https://zulip.readthedocs.io/en/latest/subsystems/full-text-...
(It's only available when self-hosting at present.)
Simple things like stream messages only being displayed halfway down the viewport with no scrollbar making it seem like there are no earlier messages.
I know I can find answers in google but I want them from someone with hands-on experience