When a rewrite isn’t: rebuilding Slack on the desktop
medium.com
medium.com
Speaking as someone who has done a fair number of rewrites as well as watching rewrites fail, conventional wisdom is somewhat wrong.
1. Do a rewrite. Don't try to add features, just replace the existing functionality. Avoid a moving target.
2. Rewrite the same project. Don't redesign the database schema at the same time you are rewriting. Try to keep the friction down to a manageable level.
3. Incremental rewrites are best. Pick part of the project, rewrite and release that, then get feedback while you work on rewriting the next chunk.
100%. rewriting all layers simultaneously is the first step to never finishing.
Some developers prefer to leave the company during the re-write (which usually happens after 3 years exciting hyper-growth).
Edit: Clarified quote of original rule.
It is more along the line of "don't try to do a big-bang release of everything" when you are initially writing an app.
I can’t believe FogBugz didn’t share any code with CityDesk.
(I think this article should be the new classic.)
> My takeaway from these stories is this: Once you’ve learned enough that there’s a certain distance between the current version of your product and the best version of that product you can imagine, then the right approach is not to replace your software with a new version, but to build something new next to it — without throwing away what you have.
Trello isn’t a rewrite of FogBugz. It’s a brand new product.
- remove one bolt and raise in the air
- slide out rev 30 "parts" from beneath bolt
- slide in rev 40 "upgrade parts" (fuselage, wings, engines, etc) underneath the bolt
- fly "upgraded plane" without a lot of pesky 'new plane" studies
Additionally, in parts of california, how to build a house:
- find existing house
- pick a wall
- remodel everything except for that wall
- reap tax benefits in your 99.9% new house.
Gradually re-written from Java (JBOSS) to Python over about 10 years. Basically the Python side knew what URLs it could handle and proxied the rest over to Javaland. We ended up shutting down the last Java bits early last year.
At the time the re-write started, Django hadn't yet seen a public release, Rails barely existed (and had zero traction, and Ruby-the-runtime was horrible), dotnet was barely a thing, Clojure and Go didn't exist yet, dotnet barely existed. 2005 was a weird time.
Realistically the only options at the time were Python, PHP (4, not 5), or Perl.
I didn't come in until much later, but I don't really think there was a better option at the time.
do you actually know what that would mean?
I would argue that if you already have a lot of in-house knowledge with Java then you might as well stick with it (and just not make whatever mistakes you made last time around) but Python seems like a reasonable alternative to me.
Partial rewrites based on Domain/Model/REST endpoint and just proxying to both webapps based on which was new already.
No breaking changes from the outside, either share the parts of the code that are still good (most of the business logic might be) or fork them (and then refactor and maybe you need to fix bugs twice) and after a while you can switch off the old part.
Works like a charm with added benefits of being in the same language to avoid wasting time. But the base idea works even if you use different languages.
It's also not for web apps, for example Apache Storm can do Bolts? (been a while) in several languages so you can also easily rewrite parts, if you can serialize your data in and out of it.
If your code is trying to get updated, you can usually refactor. If your code is having a paradigm shift, it's likely any incremental rewrite will take more time and have more duplication without benefit for far too long to be successful.
You can rewrite portions of the app and embed the older components in the new app until you rewrite them as well.
Refactoring is recoding a component while maintaining it's external interface so that external interactions with the component are unaffected.
Rewrites, incremental or otherwise, do not necessary preserve external interfaces (an incremental rewrite doesn't start clean slate, but instead progresses by gradual replacement of existing code.
The two ideas are not closely connected.
Chances are, that's the problem though. You have to re-write because your design is bad enough that it's necessary. The only time you'd re-write without changing the design is when changing languages or platforms.
Then, when that's ready and stable and working with the existing schema, it's way easier to do a refactor and adapt to the new schema too.
Attempting a design re-write without changing the data structures will just result in the same design. And you can't change the data design without then having to change the code to match.
I can't imagine doing that as an independent step towards a rewrite.
Sure you can. You decouple your data structures from your schema with an abstraction layer, facade, etc.
On the other hand, if you need to refactor the database schema, don't rewrite the code. Just do the minimum modifications to keep the app working.
Just don't do both at once.
That's perfectly compatible with the conventional wisdom you claims wrong, which, again, was “you should never rewrite your code from scratch” [emphasis added]
Rewrites are ALL about the schema.
Look, over time, your initial requirements will change. The initial schema will 100% not be what is ideal.
On a long enough time scale EVERYTHING becomes many to many.
I've found any rewrite of software without restructuring the schema has always been a waste of time.
Usually a quick glance at the schema will shine a bright light on top of all sorts of badness - denormalizations, excessive metadata-driven-ness etc. Rewriting code on top of crud like that would be epensive, and result in a lot of ugly code, which would need to be rewritten again.
They make little sense as soon as a project is completed, even if you rewrite, the software hasn't had time to point you in a direction, a 'nice' rewrite of the same wrong solution is just as useless as a bad implementation of the wrong solution.
They may make sense after a product has matured, since the actual scope of the software will be more defined. The 'correct' schema is starting to reveal itself, but usually it's rare that business wise it's worth rewriting at this stage, adding features is the better choice.
They definitely make sense once a product is fully matured, starting to gather legacy issues - since fighting around all the cruft and incorrect schema restrictions is wasting more time than redesigning it from ground up using the existing implementation as the guide.
In the first case, you have a system that mostly does what you want, but it has architectural issues that make it hard to change. You plan a rewrite. You get buy in from management. But the problem is that the business still needs those changes. If your rewrite takes more than a few months, then you're still going to have to do the changes.
Over time, you end up with 2 groups: legacy and greenfield. The legacy group is usually made up of crufty old salts that don't care what they work on. They just grind it out. The greenfield group contains usually younger people who what to do something new. They want to "get it right" this time. They care deeply about what they work on and so there is usually a fair amount of conflict on the team about how to approach just about anything.
The legacy team grinds along, slowly solving business problems and the greenfield team provides no business value until they are finished (while constantly promising that it will all be amazing as soon as they are done). But the business types see only that the legacy product is doing what they want and that the team is quietly plugging away. The greenfield product does not do what they want and the team seems to be doing a lot of excited talking, but it still isn't finished yet.
So they say, "Wouldn't it be better to move the greenfield team back on the legacy product so that we can get things done faster? After all, they said they couldn't move forward with the old architecture, but it's moving forward just fine. And with more resources it will go along a lot faster. Besides we have these business emergencies that we have to deal with, so let's just put the greenfield project on hold for a while until we can sort out exactly what's best". And the greenfield project is effectively buried.
Now, it doesn't have to turn out this way, but powerful forces point you in that direction and you have to be very careful not to have it happen to you.
The other main problem is that when people think about doing a rewrite the requirements document usually looks like "Do exactly the same thing as the last system". But the problem is nobody actually knows what the last system does precisely while simultaneously they all think they know precisely how the old system works.
You get a lot of push back from the business when you start gathering requirements because the only thing they really want to say is "Just do it like the old system". Only, after months and months of development you end up realising that there were a lot of corner cases that you missed in the rewrite. Oh... and it's not possible in the new architecture to do that without jumping through a lot of hoops.
So, incremental rewrites are best, but only if you can get the business to actually use the greenfield project. Normally they will completely ignore it because their paycheque and their promotion and their happiness depends on being able to do their job at least as well as they could with the old system. And the new system doesn't do everything that it needs to do (because we've released incrementally). What's more the developers keep asking stupid questions like "What do you want it to do" and the business people are responding with "Why can't you listen to me? I've told you one hundred times to do it the same as the old system!"
And so the new system is shunned by the business. It becomes radio active. Unless some upper management type sends a mandate down to force the business to use the new system, nobody will touch it. The upper management, in the meantime are wondering, "Why are we doing a rewrite again? The old system seems to do what we want, while everyone is complaining about the old system".
Yep, it can be done, but it requires considerably amount of help from upper management. You should avoid it at all costs unless you are sure upper management is supporting you all the way.
But even when you are successful, it may just be the beginning of the end. You've managed to stave off cancellation. You've managed to keep user engagement and work through the requirements so that the new system is equivalent. But it turns out that there is some small detail or decision that makes the new product non-viable.
For examples of products I've been personally involved with: Word Perfect 5 was written in assembly code and was not based on Windows. Word Perfect 6 was rewritten in C++ and was based on Windows. The team that did the rewrite were very proud of their work. But they through out all the keybindings in WP 5 (because GUI is way better!). The new rewrite was also very, very slow compared to WP 5. This was the beginning of the end for WP, even though one could definitely say that a rewrite was necessary. They just made the wrong choices.
Similarly, I once worked for Nortel and they had a telephone switch that they sold for $10 million a pop. It had 31 million lines of code (and stinky, stinky, stinky code at that). They realised that they could rewrite it in a fraction of that amount of code. They had 3000 developers working on the rewrite and after a few years they succeeded. Only... it didn't work with all of the business equipment that the old switch worked on. And it turned out that nobody wanted it unless it worked with that equipment. And.. the new architecture was not conducive to making it work with the business equipment. In the end, they gave a few switches to China before they abandoned it.
Rewrites are hard even when you've made the right choice to do it.
That completely fits my experience. If the new code is being used early and can grow iteratively while being relevant and useful, its gonna work out fine.
Having the codebases running side by side somehow is the best choice all around, if it can be managed.
In fact I'm sure this comment will bring out native fanatics in force.
At any rate, good for them. Seems like they reengineered themselves to a happy medium. I can definitely see myself installing it again at some point.
Yes, getting a 5+ workspace client down from ~800+MB to 300MB should be applauded - but the 250MB floor is still too high.
I want to know what’s in the 250MB it’s still using.
Too high for what? Low end desktops and laptops today come with 8GB of ram. Using ~3% of that for a chat app doesn't seem like a big deal.
3% on an Electron chat app here, 5% on an Electron text editor there, and pretty soon you’ve managed to replace what could’ve been several apps with small memory footprints plus several gigs of efficient file caching in a users’ RAM with just a few bloated programs crowding out the cache, and made their entire machine considerably less responsive.
Ram is cheap. Usage at this scale just doesn't matter to the vast majority of users.
Obviously no PM is going to tell customers, "just spend $200 for another 8GB of RAM to run our chat software."
> Usage at this scale just doesn't matter to the vast majority of users.
The usage model is that it's on all the time. So if it's draining batteries, causing paging or otherwise slowing things down, people are going to quit Slack, thus defeating the immediacy of communication, thus defeating the point of running Slack.
And Slack becomes powerful when it can be required by a workplace, which is their business model. This means the "vast majority" isn't enough at any given company.
RAM is cheap...and not upgradeable on the majority of devices in the world.
“It only uses 8% of 8GB of RAM. If that’s an issue consumers can buy more RAM or throw away their devices and buy new ones” is soon phrased as “It only uses 8% of 16GB of RAM. If that’s an issue consumers can buy more RAM or throw away their devices and buy new ones” and soon after etc, etc
At some point, preferably before we start saying “it’s just half a terabyte of RAM to run a chat app”, we may want to step back and question this circular justification for ever increasing bloat and ask ourselves when enough is enough. There’s been no real gain in functionality in the past 10-20 years of chat app churn, just an enormous explosion in RAM and CPU requirements.
Perhaps you aren't giving credit to functionality that is important to people other than yourself?
The main, really only, features that Slack has that IRC doesn’t is easy account onboarding and (crucially, for business) offboarding, and persistent messaging. This has been discussed to death, publicly, by organizations like Mozilla. They were also smart enough to offer themselves for free to startups, which largely undermined hipchat, which had the same basic value proposition over IRC, and allowed them to build a hip reputation and network effects in the startup crowd.
Are we really going to pretend that those features are why the Slack client consumes tons of RAM and a fuckload of CPU cycles? They’re entirely server side, for christ’s sake.
ICQ could do all of the above on a 32 Meg Mac 20 years ago. The bloat isn’t related to the reasons it “won”, or why it continues to be popular. Network effects, the fact that if your company mandates it you have no alternative, simply insulates them from suffering many ill-effects of their bloated, laggy client software.
But sure, ok, so let’s hear your theory on why they got big and how it requires all of this client side RAM usage. So what is it?
Mostly what drove orgs I’ve been with from one to another has been pricing (Slack having a free tier ate Hipchat’s lunch, like I said), and “buzz”. Slack was the hip thing to get on in 2013.
And no, not even with 16GB will I tolerate what slack uses even with their new version.
Yes, I do consider upgrading the ram myself by replacing the soldered ram-chips. And do not for a second imagine I'm doing that so that I can send text messages@!
Lot's more options in PCs obviously, but 2 minutes of googling found a 16GB XPS13 $580 more than the 8GB version. Again, not $1000. I'm sure with slightly more searching there are cheaper options as well.
For reference I bought 64GB for my workstation for less than $400 a few years ago. And that's with 25% sales tax.
Desktop memory is cheaper, laptop manufacturers are rent seeking with upgrades normally, and with soldered memory they have free reign.
It’s true that often the upgrades are bundled, ie; can’t get 16GiB without an i7. But I think that’s fairly rare at least.
As for 16GiB, there’s a hard limit there for LPDDR3, and intel CPUs do not support LPDDR4 yet.
Oh, and another thing regarding cost, there were some issues a long time ago with the factories producing them, which is why the price is still high, maybe artificially.
https://www.vice.com/en_us/article/wnxb35/the-year-that-the-...
Well, the biggest point of soldered ram has always been to be able to recoup costs by denying people to buy ram from someone else.
Even in the old days it was much cheaper to buy 2 GB of ram and then upgrade to 4 GB than buying it with 4 GB in the first place.
Which makes perfect sense. If you're operating on more memory than should be necessary you're gonna be burning a lot more processor/battery than is reasonable, too.
I honestly think that most people us use phrases like "webtech junk" are offended on principle and not because of any measurable difference that matters to regular users.
I think it is because it expires caches left and right, loses track of events, or I have no idea what else could be going on on a PC with 16GB RAM (plenty of it free) and stable Internet connection.
I guess if my principle is that I like my computer to respond quickly and accurately to my input then yeah, I dislike webtech junk on principle.
[EDIT] as far as it not mattering, everyone I know, including plenty of non-techies, who has used Google Docs hates it, due to the UI lag. Typing with that large a delay between keypress and the letter appearing sucks. They may use it anyway because they have to, but they all complain about how laggy the UI is, responding to clicks and keypresses.
Try telegram on desktop and compare the UI performance. Slack definitely has UI lag. Not nearly as bad as Teams though.
The problem isn’t with Slack itself - as more and more desktop software run on Electron then with 8GB of RAM you can only have a handful of instances open simultaneously before you get performance- crushing disk paging.
Why should it be reasonable for displaying a few megs of text & maybe a dozen megs of images to take 250MB+ of RAM? What is the rest of that memory usage, and why is it loaded? CPUs don't like executing code that's not in cache, so hopefully it's not code. And if it's not code, what is it? Figure a few dozen MB for the graphic buffers, but where's the other 100MB+ gone?
While 250MB is a big chunk... I think it's actually okay, even in this extreme case.
Too high for an app that runs continuously.
The reasons:
- Cache. RAM may be >8GB but the CPU L1 cache is just 64kB. And L1 cache is about 100 times faster than RAM. From my experience, proper cache management is by far the most important low level optimization, and most of it can be archived by just using less RAM.
- What are you doing with all that RAM? 250MB in absolute terms is huge. The entire Harry Potter series is around 5MB uncompressed. For that size you can also have a 1080p fullscreen RGB image, uncompressed. What it means is that unless you are working with large datasets, these 250MB represent a huge amount of stuff you don't control. Lots of moving parts with potential performance bottlenecks, security issues, etc...
- Every byte your app is using is one less byte for your system caches. Wasting RAM makes your whole system go slower.
- Finally, an ethical argument. Your chat app will run continuously on millions of PCs. On that PC are libraries, kernel code and frameworks where people worked hard squeezing every bit of performance they could. Saving energy and making your apps run to their full potential. For me, a background app made by a company the size of Slack have a duty to be as light as possible, as not to interfere with the actual work people are doing on their machine.
What gives you that idea? Lots of apps have cruft, but that's nothing to do with Cocoa.
A "Hello, world!" in Cocoa uses around 13 MB, and the application bundle weighs in at 79 kB. The fact that Electron uses a bare minimum of over 100 MB just to get an application running is a source of some understandable consternation.
People have skewed ideas of what's doable in memory constraints in 2019 for the feature sets that customers tend to expect.
Compare a "media-heavy" slack with that Telegram, or compare with Telegram minimum.
To support that, you'd need far more than a "Hello World" stack.
(Gmail uses 250MB at runtime in the same Firefox instance, for comparison; I didn't test it in an Electron-like container.)
You pick Gmail as another example. Gmail is a memory hog and fairly slow. For comparison, Fastmail (which company I work for) tends to use 10–15MB in Firefox for slightly different functionality (no chat, and I estimate the Hangouts widget to be about half of Gmail’s bloat, but on the Fastmail side you can load the calendars, contacts and settings modules in the same document comfortably without breaking 20MB), and is a good deal snappier. It comes of careful engineering with performance as a priority, and a small team.
Slack is simply atrocious in these regards. Painfully slow, utterly greedy for memory. I often run it in Firefox rather than standalone to keep its memory footprint down. Whichever way I do it I regularly find it to have crept up over a gigabyte of memory, on my single workspace.
Personally I would have used those years of development time to go native but it is what it is.
I talked about Redux usage stats and comparison with other alternatives in my "State of Redux" talk at Reactathon earlier this year [0], and my post "Redux - Not Dead Yet!" also addresses some of these aspects [1].
Also, quick plug for our current focus. I'm working on some tutorials for our new Redux Starter Kit package [2], and then will be working on tackling additional functionality to push it towards a 1.0 release [3].
After that, we'll get started on a major revamp of the Redux docs [4] to improve the learning flow, better organize the content, cover more real-world usage topics, and teach using Redux Starter Kit as the "default way to write Redux".
[0] https://blog.isquaredsoftware.com/2019/03/presentation-state...
[1] https://blog.isquaredsoftware.com/2018/03/redux-not-dead-yet...
[2] https://redux-starter-kit.js.org/
[3] https://github.com/reduxjs/redux-starter-kit/issues/82
[4] https://github.com/reduxjs/redux/issues/3313#issuecomment-45...
Also aren't communication apps a perfect use case for Redux due to the need to have events from multiple sources happen in a single store in a linear order?
Lots of folks have used Redux because they were _told_ they "need" to use it. That's always been overkill. And, yes, there are plenty of other viable alternatives that overlap Redux's capabilities in various ways (Apollo, MobX, React context, etc).
So sure, it's not a "must-have", because you should _always_ evaluate tools and determine what's really appropriate for your use case rather than just blindly using something.
But, at the same time, Redux is still very widely used (~50% of React apps), and not going away. So, there's a big difference between "not an automatic must-have" (which is true) and "dead/dying" (which is not).
In reality, there should be a good balance between global and local state. I wrote a Redux FAQ entry that tries to give some rules of thumb for determining what state goes where [2].
When we do the docs revamp, hopefully we can make that kind of thinking more explicit.
I'd be very interested in hearing more details on any issues you may be having with React-Redux, and I'm happy to offer advice for you or anyone else on using it better. HN comments aren't a great place for that, but please file an issue for discussion or contact me in the Reactiflux chat channels on Discord in the #redux channel.
I'd also greatly appreciate any feedback folks would have on weaknesses with the current docs, so we can improve those issues with the docs revamp.
We've had open issues to cover writing a "Performance" docs page for a long time, but no one got around to contributing that, and there's been too many other higher priorities for us to write it. My React/Redux links list does have a section of articles on improving Redux perf [3], if that helps, and there's an FAQ section on that topic [4]. In general, connect more components [5], and use memoized selectors [6] to read data and do transforms.
[0] https://redux.js.org/introduction/three-principles
[1] https://redux.js.org/faq/organizing-state#should-i-put-form-...
[2] https://redux.js.org/faq/organizing-state#do-i-have-to-put-a...
[3] https://github.com/markerikson/react-redux-links/blob/master...
[4] https://redux.js.org/faq/performance
[5] https://redux.js.org/faq/react-redux#should-i-only-connect-m...
[6] https://blog.isquaredsoftware.com/2017/12/idiomatic-redux-us...
Rather than putting absolutely everything in the global DB, I now try to restrict that to things that are actually global, such as the user profile data and such.
After a couple of years working with Redux, I think my interpretation has settled down to:
Redux should be the single source of truth for the pieces of information you decide to put in there.
I am not super knowledgeable about all the exact architectures, but I thought mobile was just starting to adopt the redux architecture vs MVP, MVVM, etc
Under the hood it uses [cofx](https://github.com/neurosnap/cofx) which is a library I wrote to solve the need to write declarative side-effects without redux.
Something like Redux Saga, Redux Thunks, Redux Observable (my personal preference here), or what have you, are the higher order "toolsets" that Hooks alone aren't likely to replace yet. It may be the case that some of them are going to start targeting Hooks/Context more directly without "needing" an intermediary state engine like Redux, but it's also entirely possible that it will still remain a good reason to make a step change to a state engine like Redux when you find yourself needing higher-order coordination between components that libraries like Sagas, Thunks, or Observables can provide.
I use Sagas as well though, and Ive found them ok...really easy to test but feel very heavyweight and clunky; personally, I don't _really_ like the loading/error state to be handled by Redux: the more years I work on React/Redux app the more I prefer for a section of an app to be mounted -> triggering a fetch -> loading, show local loading UI -> {fails, show local UI|succeeds, dispatch success}. So Redux becomes a serializable normalized data cache that acts to store the results of disparate API calls and doesn't care at all much about the UI.
There's a tremendous amount of work out there for how to get redux to completely handle async and branching methods, and I think the best solution for most cases is to not have it handle any of that.
Keep your data fetching in normal everyday functions, have react components call those and funnel the data you need in the global store back to redux, and then have the components that need that data read it from the global store.
Edit: If you do need to synchronise multiple async updates to that state that happen in a very short space of time & in an unknown order in a controlled way, then yeah something like sagas is a solid choice to exert control over that. But very often the user of the UI is only going to make one request [or one batched set of requests] -- eg I am working on an RN app at the minute, and each screen does need to fetch data. So I fetch in the component, it shows loading etc, then populates the store. If the user cancels what they're doing (navigate away for example), the component used to fetch unloads and no dispatch is made.
I've got nothing against any of them, I've used almost every major one in my jobs over the last few years. But I (and the person who replied to me) both seem to feel the same way, that in many cases just fetching in React, in the component, is all that is needed, let the Redux application deal with marshalling the resultant state (which it is very good at)
but how do you put the result back in the store? how about displaying a default failure screen?
The API request is in the file that represents the point at which the API request is made by the user (previously in a loader component or, now, using a hook, in future using suspense), so I would say it involves jumping through 0 files, as opposed to several if I move it to redux (the dispatch occurs at the UI trigger point, then I have to drill through several other layers of abstraction). In practice there's normally some facade in front of the API to prefill values and also to make it easier to mock for testing, but if anything that just makes it simpler as the facade acts as a reference.
> how about displaying a default failure screen?
failure && <FailureComponent>{failure.message}</FailureComponent>
This is I guess down to a preferred philosophical approach but from my pov React is a UI library and it's _really_ good at showing conditional UI. Redux is a state container and is not great for this. I don't feel I want that in Redux, I want Redux purely to be a reference the state of the data in the app; loading/req/etc states are not that. Whenever I've moved error/request/loading/failure state to Redux (ie almost every app I've made over the past four years before this year) it involves a series of abstractions to track the current request. Whereas React can do this out of the box at the point the action is taken: make the request, show some state, if the request succeeds, dispatch the data to the store and [re]render the component that displays the data based on the state of the store.> but how do you put the result back in the store?
dispatch({ type: "foo", data: "bar"})
And have a component that renders based on props in the store, same as normal.In our main app we also use advanced features like what Slack is describing in their post, like having multiple Redux stores. We also combine stores from multiple sources (e.g. appStore = [...store1, ...store2, ...storeFromALibrary]. It's incredibly flexible.
There's nothing wrong with Redux; we just found that Redux based code tended to have a fair bit of boilerplate, and it wasn't as easy to maintain code using Redux as we would have liked. Every dev team has to find the right place to draw the line on implicit versus explicit, magic versus verbose, convention versus configuration. No real right or wrong answers.
> I thought mobile was just starting to adopt the redux architecture
I don't think so, no. The Redux architecture (ie, reducers, actions, etc.) is fairly specific and not universally loved. I would say that people are starting to really grasp the importance of solving the problems that Redux solves, however.
I think it's the best of both worlds.
Nothing is dead about event sourcing, other then the mob has moved forward in the hype cycle.
And yes, agree on needing more documentation:
https://github.com/reduxjs/redux-starter-kit/issues/59
I've got an example app I put together using RSK, our new React-Redux hooks, and TS, and I'm planning to turn that into a tutorial soon:
- The first iteration was largely powered by jQuery(!)
- Every workspace got its own isolated Electron process because they didn't anticipate multiple workspaces in their initial architecture
Honestly it's amazing they got as far as they did with the above limitations. Sounds like this rewrite was sorely needed after their phase of rapid growth. Glad they pulled it off.
There's no way of knowing, without knowing what they were doing previously. The whole point of React is to leverage the virtual DOM to avoid unnecessary (and very slow) DOM manipulations. Given the sheer number of things Slack has on any given page it isn't unreasonable to expect that React will make things quicker, even if it has a penalty at startup time.
I now do my best to keep my website apps running just in Firefox containers.
Electron feels like the last rush of "write once, deploy everywhere" days of flash and Adobe air. God save us all.
Even with the performance improved, the "stuff the whole app in one web view" implementation is still showing. Clearly this isn't an Electron limitation; VS Code handles multiple windows just as well as any other text editor.
Slack's desktop client is basically "It's a web browser except worse and with the URL bar hidden." Since chat requires an internet connection and it's not like Slack's desktop version launches particularly fast, I'm not sure that it does anything better than using a browser.
We'll see if they support multiple instances on iOS 13. If they do, maybe we can get a Mac version based on that for something closer to native desktop behavior.
I don't see how it could ever be worse than the webpage. And it gives you nice perks like notification badge integration, which for me personally is essential.
I usually don't need to do that, but when I do it's annoying that the desktop client prevents it.
It's fine that my phone doesn't let me open two text conversations at once since that's all the space it has, but on a computer with a 24+ inch screen and multiple virtual desktop spaces with different sets of tasks, it's a pointless limitation.
So "the downsides of Electron outweigh the downsides of native apps" when you're someone who patrols the task manager looking for things to be upset about. Your chat client - one of half a dozen applications you realistically have open - using 400MB of RAM on your 32GB workstation does not have a meaningful impact on your workflow. It's just offensive.
(I'm not against web APIs as a platform for desktop apps, but I'm happier using Webkit wrappers and I think this would be improved, cross-platform, if Chrome could effectively host the apps that currently ship with their own instance of Electron or CEF)
However, if your other applications are a terminal, other efficient text editor (e.g. Sublime), and other generally respectful apps, I've found having Slack open is the difference between having enough battery life to work all day, and not.
The upside that keeps getting paraded around is that it's a huge boon for product velocity. But that's just objectively false because Slack-the-product hasn't meaningfully changed in years.
If you had come to me 10 years ago and said "Microsoft will drop their browser and use Google's code," I absolutely would not have believed you.
I don't see how this has anything to do with the point here (browser engines are not normal applications…) or if it's even true.
I'm just saying that "native client" doesn't necessarily mean separate Mac, Windows and Linux ports; furthermore, for a billion dollar company, a native client of _some_ kind isn't an unrealistic demand.
I do wonder if the plugin/extensibility architecture — "Slack Apps" like Github and IMGUR and OpenTable — is the real answer. (Unless I'm mistake and these run on the native mobile apps.)
Name three major feature improvements to Slack in the last three years.
I can think of one, threads, which are a fucking usability disaster and should have been killed immediately. I can't think of a single other major feature.
I use it for incident updates internally too so the discussion is grouped. Much easier than paging through the channel.
[1] It's still strange to me that many people compare Slack to IRC and XMPP and complain that they were doing all the same things, why have people abandoned them. Even though they (and especially the clients) were doing (or capable of doing) just a small fraction of what Slack is doing.
The question is whether the shared part is written in C++ vs. JavaScript.
also, whether the shared part is 98% of the code or 50% of the code.
For years slack has been a resource pig on desktop and on the web. So what improvements are you imagining they’ve been working on? Chatting on the internet is hardly such a groundbreaking subject that it can’t be done efficiently.
the ones that are described in the blog post that you're currently commenting on
Thank you for making my point for me.
People seem to forget that C++ and opengl is cross-platform. And there are projects far bigger than slack, like ffmpeg and OpenCV that have existed for decades, always had very fast development cycles, with only a subset of the funding and money that Slack has, and stayed native and close to the metal forever.
A better answer would be, that Slack has deemed the benefits that would result from this as not that important. Or that the current team's expertise cannot handle the task. But they always looking into ways of improving the core product and experience.
There's no conspiracy or anything going on here, and you're right it's wrong to assume. That's the general sentiment of what I was trying to say though.
https://en.wikipedia.org/wiki/Glitch_(video_game)
That core team therefore had a lot of frontend web dev experience. It made sense that they'd stick to their strengths
Can you see how maybe other people might have valid concerns about the effects of a product, and ask for more rigor in the company's processes?
That's perhaps the most tortured analogy I've ever heard. I can't even begin to fathom how you think that's comparable or relevant.
Sounds like it got my point across. You just don’t like the point.
Slack is successful because of marketing, not technical superiority.
I worked at a small company that used Google Talk for office chat, and us developers complained because it was difficult to have a group conversation with. We started using Discord (we were familiar with it and liked the dark theme) and it caught on and we were quite productive with it. Our business types saw it and decided to "formalize" it, so they picked Slack and rolled it out company wide. Most people actually preferred Discord, but the business types preferred Slack (probably mostly because of marketing), and now we're all using Slack.
Now, Slack also didn't royally screw up as it scaled up (outages were rare and transparently reported), which is a credit to the developers and PR people. That being said, just because something is successful doesn't mean it's good; it could be, but that's tangential to the topic at hand.
OpenGL is dying. It's already deprecated on MacOS.
https://appleinsider.com/articles/18/06/04/opengl-opencl-dep...
OpenGL effectively should die... but it's not going to be in favor of raw Metal/Vulkan, except in rare cases. Most people already use engines that abstract most of that away, and Metal/Vulkan are in basically every way better for those engines.
It was posted a few months ago [2] and I’ve been using it since then, it works great.
I'm also a bit concerned of using a closed-source client with Slack.
"Slack connectivity is now available for testing. It's still pretty rough and missing a lot of features."
any further insights here?
could just try it myself, but slack is our key communication so getting more input would be great
The only "ideological battles" are religious in nature. I'm fine with keeping religion off HN, but you can't just call a contentious topic "ideological", especially if there is evidence and/or rational debate to be had. The fact that an insufficient number of people opt-in to discussing some topic in that way does not make it "ideological". Would you also consider conversation about climate change "ideological" and thus unworthy of HN?
In any event, the original post (interestingly) got unflagged, so this was superfluous anyway.
1. Slack is too restrictive, and its API is too poorly supported for anyone to create a port. I find this unlikely, given that the API is extensively documented and that an Emacs port already exists, but maybe there are problems I'm not aware of.
2. Good native ports are actually a lot harder to build and maintain than typical HN posters claim.
3. For all everyone complains about Electron, maybe the current app is good enough for pretty much everyone, including the complainers, and those complaints are largely just hot air.
or 4. A few good, Open Source native ports already exist, and people are just unaware of them.
Regardless, the HN crowd is made up of developers who know how to develop things and use experimental software -- most people on here know how to extract a login tokin, and a lot of people on here are willing to put in a lot of effort to solve problems. If there isn't a native port already, there's very likely a good reason for that.
If there isn't a good reason for it, and it's just that somehow nobody on HN has thought to sit down and build a native app yet, then I'd welcome one, I suppose. But obviously I'm not annoyed enough by Slack to do it myself, and I suspect that most other people posting here fall into that category as well.
Native GUI clients:
- Ripcord: https://cancel.fm/ripcord/
- Wey: https://github.com/yue/wey ("written in Node.js with native UI powered by the Yue library")
- Volt: https://volt-app.com
IRC bridges (allow using Slack from native IRC clients):
- wee-slack: https://github.com/wee-slack/wee-slack
- irc-slack: https://github.com/insomniacslk/irc-slack
- bitlbee: http://bitlbee.org (using libpurple)
libpurple plugin (allows using Slack from Pidgin, Adium, bitlbee):
- https://github.com/dylex/slack-libpurple
- Adium (native macOS app) plugin based on it: https://github.com/victori/slack4adium
CLI clients:
- https://github.com/erroneousboat/slack-term
- https://github.com/haskellcamargo/sclack
- the emacs one you mentioned
Most of these clients don't support 100% of Slack's features, aren't as pretty, and are generally not as 'polished' as the official client. But they're also mostly written by individuals in their spare time, as opposed to a team of full-time employees. So no, I don't think that native clients are 'actually a lot harder to build and maintain'.
Once Riot and Matrix pick up steam, I suspect the Electron debate will be even less of a real issue, since you can pretty safely build a native client for Matrix and know for sure that nobody on the corporate side is going to get annoyed and tell you stop.
The best compromise right now is to aggressively build pretty, heavily themed prebaked designs. Think Material Design, Cupertino etc. so native UI is somewhat good looking too even though it may not be as "rich" as a React one with all the drag and drop animation and flashy trinkets. Feature wise of course there's only marginal improvement to old school Java Swing, but it's certainly easier on the eyes.
Never build your future on someone else's platform.
See Twitter for a good example of where business interests and API stability intersect in negative ways. I would make the point that if you don't trust a business to keep their official API stable, you probably shouldn't be building a community around their tools in the first place, regardless of whether or not they have a native app.
If I'm going to put my resources into building a client for something, it's not going to be based on some closed source backed that could change at any time, it's going to be based on something open and forkable (worst case scenario, I can just maintain my own backend).
Slack bills itself as a complete solution, not a piece to a larger puzzle, so third party clients and plugins will always take the backseat. Who is going to try to build a serious competitor when they can't really rely on the backend staying consistent?
Building a decent frontend isn't particularly difficult. It's not trivial, but it has been done. For example, Pidgin, while a little ugly (I prefer "functional"), works well and has been around for years. I used weechat and irssi for IRC chat before we switched away from IRC, and it worked really well (didn't see pictures, but I think that's more a feature than anything, and links were more than sufficient).
The problem is that business types prefer hosted, "batteries included" solutions, and those types tend to not be very friendly with third party add-ons. If I end up being able to use Matrix/Riot, maybe I'll try my hand at a native, cross platform client. But until then, I'm not going to throw my effort toward something that can change at any moment.
It seems Slack has addressed the major issue with their current app with this rewrite — multiple workspaces.
That said, as much as I dislike Electron, Microsoft has done a fine job with VSCode, so there are ways to do Electron “better”, even if the “best” Electron apps are still mediocre compared to native apps
The economics of RAM has changed somewhat but 1990s me is still appalled.
I'm not saying they can't do better, but the app does do quite a lot.
Microsoft has done a fine job with VS Code considering it is built on Electron, but it's a freaking text editor ffs! It has basically zero need for a UI. If this were 1990 and there were no cross-platform filesystem, i/o, network, async, etc. APIs available for native applications, it might make sense. But a text editor is basically the very simplest cross-platform UI you could ever need to design (proper text metrics and rendering is the only gotcha, but if you just shell out to freetype and DirectX rather than doing it all yourself, it's not really a problem).
VS Code is only fast because people compare it to the likes of Atom (which has thankfully dwindled in usage), GMail, and Slack. It doesn't hold a candle to the responsiveness of, say, basic native text editors like Notepad or Text Edit.app, or even horribly written "native" software like Notepad++ back in its heyday (although they might have cleaned up that mess in the decade and a half since I last used it -- I see Scintilla has switched to C++17!). Sublime Text was originally written by just one person and doesn't use a pre-packaged cross-platform editor widget, yet Skinner managed to do it without bundling in the kitchen sink and it shows.
Microsoft has realized this of course, and since the initial release VS Code has shied away from actually using Electron and many features are not implemented as HTML elements/nodes and instead are written bypassing the abstraction entirely because it's literally not possible to get the sort of responsiveness users (sometimes, incredibly meekly and never if they have to actually pay something for it) demand from a code editor if you have to go through Chrome/Chromium's abstractions to do it.
* Syntax highlighting and code folding * Autocomplete * Integrated file browser * Integrated git client * Automatic formatting * File preview * Debugger * ...and a ton more features
Just looking at the screenshots from the release notes (https://code.visualstudio.com/updates/v1_36), I highly doubt this is "the very simplest cross-platform UI you could ever need to design".
Also, as far as I know, the vast majority of the editor is still HTML/CSS/JS, with only a few exceptions (the integrated terminal and project-wide search I think?)
And it also plays little explosion animations around my cursor while I type, which is neat.
But no doubt, Sublime Text exists in a different (native) world of speed altogether (and I do love Sublime Text)
The disappointment is strong.
On a side note, SBlack, a lightweight slack client is very impressive on memory usage [1].
Looks like it just wraps the website in this case.
Yes there are clients that use the API. Slack offers a http rest-like api as well as a websockets realtime api. For example, wee-slack[1] (weechat plugin for slack) and slack-term[2] (curses client in golang).
>Sblack works exactly like a browser with small tweaks. We inject the dark mode style at the end of the Slack’s html, and that’s it.
Another benefit of this is that Safari/WebKit does better with battery life than Chrome. I'll have to give this client a look.
Even though Slack is using react now, they are still using electron under the hood. Which means it can never match the snappiness of a native application like mIRC.
And you, without knowing all the product requirements and technical constraints, know this exactly how?
For you ... for me, mIRC is difficult to use, understand, and overall not nearly as aesthetically pleasing as slack.
Not to mention mIRC is missing a ton of features that slack has which specifically makes it better suited for businesses. Push notifications and chat history to name a few.
I've since moved to IRSSI and others, but you are correct about memory/CPU usage and functionality.
Every single '$new-thing' like Slack makes the interaction WORSE, instead of improving it. I wish more SV superstars were greybeard hackers.
Their first marketing bullet: "It's not made from a web browser" :D
As an older dev who's eye sight isn't as good as it was 30 years ago when I started, it's getting harder and harder to use native apps because the default fonts are so goddamn small.
Oh sure, you can dork around in the font preferences dialog and change some things, but there is no option to scale all of the UI (UI => Scale => 120%). It ends up being a hodgepodge of unreadable small text mixed with reasonably sized by but poorly styled and colored larger text that's nearly as hard to read because of the lack of space separating elements. Of course everything gets more crowded when you increase the size, instead of doing what it should do and also increasing the space between elements to let everything breath.
In an Electron app, I can Ctrl-+, Ctrl-+ and everything looks great. I have yet to find a single native app that I use regularly where this isn't an issue. The only ones that comes close (no surprise here) is Sublime Text.
These problems just don't happen with Electron apps. Electron technology may be slower than native, but it sure nails UI scaling. I'll take a UI I can actually see over one that runs more efficiently any day.
Honestly, I think it's an accessibility issue. Most devs are simply used to the small GUIs, so that's what they build. They don't give it any real thought or put any actual design effort into it. It's the same with accessibility, it's not my problem so it's not a problem. And while native GUI's help with some aspects of accessibility, they don't provide the tools to make scaling easy and obvious so it doesn't happen.
And that's why people throw fits about software being inaccessible, because it's literally unusable for them. I'm lucky, I just want everything to be 20% bigger by default because it makes my life a little easier. Others don't have that luxury. That 20% can be the difference between being able to use your software or not.
Even just focusing on the usability of zooming in a browser-- I wonder how much even the best Linux desktop environment (or even OSX) trails behind browsers. Off the top of my head:
* the obvious keybindings (even across browsers?) that are the same for tabs/windows regardless of the content inside them
* in FF the zoom percentage animates itself into existence when you go below or above 100%, then remains viewable and clickable to return to normal
* most of the popular web pages are designed for zooming so that text reflows in a readable manner when zoomed
* when zooming, fonts remain readable without the user fudging around with DPI or rendering settings that no mortal ought to understand
* zoom level and scroll position is remembered in case I click the back button
Well, the desktop doesn't even have a notion of the back button so I'd say we're already at least a few years ahead of the desktop.
It's an accessibility problem, not a resolution problem. Electron apps don't have this issue because you can resize each app individually to match your needs/screen settings. You can't do this with most native apps.
"Features
- Not made from a web browser"
Understanding on why Slack is unable to create a performant desktop-client for their users is beyond me. I also wasn't able to use Slack and Discord at the same time on my MacBook without it swapping on the SSD. So Ripcord seems to be a good lightweight alternative for me.
I just wish it supported Keybase Chat so I can ditch that Electron garbage too.
I suspect that "Why is Linux competitive with Windows" has many of the same answers, in the end.
There is hope though, React Native for Windows team did a presentation showing how bad Electron resource usage was versus their React Native for Windows prototype.
So maybe they could do some advocacy to the VSCode team and start a migration movement.
React Native for Windows is ready for production, it powers their new Skype. It's being rewritten right now from C# to C++ for better performance in their vnext branch.
You have the option to use react-native-web (Twitter's way) to run your React Native app on the web, or use ReactXP (Microsoft's way).
React Native Web forks React Native. ReactXP wraps React and React Native. Two different approaches. Personally I prefer ReactXP because React Native Web will always be too far behind React Native. It's easier for ReactXP to update. Both projects have major backers and power major products.
I was very happy when we switched from HipChat to slack.
Then they switched to QT using WebView (2013) [2].
Then they created Stride using Electron (2017) [3]
Finally they created a partnership with Slack (2018) [4].
[1] https://www.engadget.com/2013/02/14/hipchat-ditches-adobe-ai...
[2] https://techcrunch.com/2013/02/14/with-3500-paying-customers...
[3] https://techcrunch.com/2017/09/07/atlassian-launches-stride-...
I was using from 2017 -> Feb 2019 when we migrated to slack. It definitely had a webview feel similar to slack. I never particularly noted any memory issues. I haven't noticed any with slack either though.
For me my top memory consumers end up being:
1. Docker (web dev servers) 2. Chrome 3. Firefox 4. VSCode 5. 1Password 6. Spark 7. Sublime Text
Even at the bottom of the list they are using 200-300 MBs. Spark, 1Password, & Sublime Text are all "native" apps so I really don't understand what the whole fuss is about.
Spark is a native app right? If it’s not they definitely nailed the look and feel of a native app.
This seems to be the right update (4.0.0), but on Slack's website the release notes [1] show the release date to be July 15. Is this the same version as I'd get via the Mac update? If so, what's with the press push today (Verge and others), if this has been out for weeks? Also, why is it described as "a little faster" and "a wee bit faster overall" (in the web release notes)? Is it a wee bit faster, or much faster?
1: https://slack.com/intl/en-gb/release-notes/mac?geocode=en-gb
The fact that the web part can be updated separately is why you can hit ⌘R and have new features appear in your workspace.
In my case, I quit and relaunched the (same version of the) desktop app and suddenly my sidebar looked different and I had a popup telling me that Slack was now faster.
The only one that doesn't make sense to me is _2 teams_. Unless they had some specific hack to reuse processes for a 2nd team, how can the incremental memory usage of that 2nd team be almost nothing?
Thinking about it, though- separate teams were lazily loaded and the usage behaviors of people with different numbers of teams probably varies substantially enough to cause these artifacts. e.g. I imagine this is "3-5 teams logged in" not "3-5 teams actively loaded".
(disclaimer: I don't work for Slack, just speculating.)
Before VS Code everyone just accepted Electron's memory bloat, maybe slack even had plans to go fully native b/c no one thought it could be done properly.
Not how I'd describe a text editor that consumes 700MB+ of RAM at any given time.
For reference AIM 4.2 system requirements was 6mb of RAM and 6mb of disk space.
Likely the only reasoble way to have all this and keep it compatible with the browser version is to either build on browser or embed a browser in the native app.
You can argue that all those things are not needed in a group chat, but Slack clearly feels otherwise. They are building a platform, some sort of groupware - not just chat.
Check for example the Doodle bot demo: https://slack.com/apps/AFA5VQJKX-doodle-bot
Considering this took _two years_ it's kinda incredible they couldn't rewrite it natively in that amount of time or less. I'm guessing the only reason this took two years is due to the incremental approach they initially took.
Not sure they changed that but this is always at the back of my mind when I see someone use Grammarly via the browser extension:
https://techbeacon.com/security/grammarly-leaks-everything-y...
Is there a security-conscious Grammarly alternative?
Been trying Language Tool as well: https://languagetool.org/
Not nearly as good as Grammarly, though, but it works with more languages and even has a plugin for vim.
With that being said, can we please have dark mode now?
[1] https://bellard.org/quickjs/
P.S.: Yes, Fabrice Bellard did his thing again!
Would be cool to see a native Reason app from such a big player :)
I realize native apps on the three big platforms is difficult, but they're not using any native components what-so-ever. It's their web interface in a window. They could have used existing toolkits like QT, Gtk or something else, themed them to look like the Slack we're familiar with, and ditched the electron bloat.
I've tried to write a couple of prototypes in electron and have worked on some electron projects and I just don't see why people are flocking to it. Every tutorial is based around huge amounts of boilerplate or scaffolding, and unit testing always seems to require weird freaky hacks with modules.
I mean I guess if it works for them, good on them, but I'm still not going to take this as a sign to recommend or try new projects in Electron.
I fail to see the need for a slack desktop client.
No, js/html "applications" are not desktop apps, they're web pages. Doesn't matter how you hide it.
However, only a tiny fraction of Skia’s functionality is actually used, just for rasterizing lines and blitting images. Font rendering is done by using the underlying platform APIs to do the glyph rasterisation. The editor also exposes an ABI/API that plugins can use via Python scripts. C++ provides the cross-platform functionality, Skia provides the cross-platform UI, custom code was written to target specific parts of the UI in macOS (using Cocoa) and Linux (using GTK). A tiny web driver is used to render a super-minimal set of HTML tags for tooltips.
And yes, VS code is a web site by my definition. Not an application.
Both are only "native desktop applications" if you squint your eyes.
And code quality has a huge impact on user experience in the long run. Writing it off as a non-factor is laughably ignorant.
1. You'll first need the ability to open .asar archives. This can be done with 7-zip along with the following package: Asar7z (http://www.tc4shell.com/en/7zip/asar/). Follow the installation instructions on the site and keep in mind you need administrative privileges to copy files to Program Files.
2. Completely close down Slack via Task Manager → End Task (Simply closing the application is not enough, you must kill it in task manager)
3. Navigate to the following file in your user app data C:\Users\{usernamePlaceHolder}\AppData\Local\slack\app-4.0.0\resources\
4. Find app.asar and first create a backup
5. Right-click app.asar and select 7-zip→Open Archive
6. Navigate down to the dist folder, right-click the ssb-interop.js file, and select Edit:
7. Scroll to the bottom of the large block of unreadable javascript (it's minified) and paste the code snippet to the bottom of the file.
document.addEventListener('DOMContentLoaded', function() {
$.ajax({
// Note: change the URL to whatever css file you want. This one works well enough for me. I have it locked to a commit to prevent XSS
url: 'https://raw.githubusercontent.com/laCour/slack-night-mode/5e3b2ba1303cf98775c386994730d9f87e8e8218/css/raw/black.css',
success: function(css) {
$("<style></style>").appendTo('head').html(css);
}
});
});
8. Save the file and close the archive in 7-zip. Reboot Slack.Note: that you'll have to reapply this fix every time Slack pushes an updated
I mean, I agree, losing mass marketshare SHOULD lead to a major shakeup on trajectory, and competing on desktop user experience would be a logical next step. But... back to the first sentence of this comment.
Here are the first sentences of the first two paragraphs:
> Conventional wisdom holds that you should never rewrite your code from scratch, and that’s good advice.
> Still, software codebases have life spans.