Software bloat makes me sad
remarkablyrestrained.com
remarkablyrestrained.com
Even though my computer can run all applications without a hitch, it is still very wasteful to constantly use CPU power because of technology choices or plain laziness. As an example, Spotify and Slack are two applications that seem to use most of my CPU after Chrome. Spotify and Slack combined seems to hover around 5-15% of total CPU (a two year old i7). When there is a lot of traffic in Slack I have seen it using 15-20% by itself, with multiple processes running and memory use going above 200 megs.
Both applications work smoothly, but should they really use that much resources? A chat application? A music player? With modern CPU's I would expect them to be at the bottom of the process list when sorted by CPU usage. I used IRC on my Pentium 75Mhz and it ran fine. When simple applications are made so poorly that they use that much resources, what is the worldwide impact of that power use? And what about the users that don't have powerful and expensive CPU's?
I guess it would be a nice incentive for developers if OSX could notify you that an app that is on background is using a lot of energy at the moment, maybe even with a quit-button when on battery power. At least I wouldn't want my app to end up on that kind of a popup.
A 15" Macbook Pro has a 99.5Wh battery and lasts about 8h; that's 12.5W. A program that decreases battery life by 20% would only mean an increase of 2-3W. Even if the machine and program are running 24/7 and all the energy comes from coal, that means an increase of less than 0.03T of CO2 per year. For comparison, the average carbon footprint of an US citizen is 20 Tons per year.
The problem is that our tools for making multiplatform native GUI's suck so bad we'd rather just embed an entire Web browser into everything.
I think the main reason that node-webkit and what-not are popular, is because of web developers moving to native app development. It's really easy to get started that way, and you can even share code with your web app. Where something like QT has a really huge learning curve for programmers transitioning from Javascript.
About poor algorithms. I actually worked on optimizing a well known web browser for a couple of years. And most of the stuff we did, was because of really bad Javascript code. Even though it seems gluttonous to embed a web browser in applications, and even insecure, it doesn't have to be as bad as it is, especially with a simple application like Spotify. This is going on a bit of a tangent, but every frontend programmer should at least learn how the browser actually works, a nice site for that is http://jankfree.org/
There's more to non-nativeness than just look and feel, which the more mature cross-platform GUI toolkits can ape fairly well. Another concern is accessibility for people with disabilities, e.g. blind people using screen readers. Qt, for example, is kind of accessible on Windows, Linux, and Mac, but not at all on mobile platforms. Not sure about the status of JavaFX. At least a single-page rich web app can be made accessible using the ARIA extensions to HTML, and if you use one of the big four web rendering engines, you can be sure they've done the hard work of implementing the underlying OS accessibility APIs well. Of course, many (most?) web developers don't implement ARIA for their custom UIs.
wxWidgets applications work like native applications because they are in fact using the native toolkits, and some successful applications, like Audacity, are written using wxWidgets.
And it runs fast, as it is C++, as expected. Why it is not used more fits perfectly with the content of this article.
It sounds like solution (for spotify at least) is better methods for each platform of creating a GUI around web services. Microsoft have sort-of had a go at this.
Browsers are already closer to this than most people realise. By the time you've included websockets, openGL, webaudio, and bunch of other stuff, you're 90% of the way to a useful OS.
The problem is that the current state of the web "OS" is a random collection of ad hoc APIs hacked together for a completely different job, and it's very badly designed for what it's trying to do now.
In a perfect world the FOSS world would collaborate to thrash out a new spec, and also design a new browser to follow the spec which was backwards compatible with standard HTML etc, but also included new and more efficient APIs for much faster performance.
Something like this has already happened on servers - see also, containerised VMs running RoR or Node or whatever you want - and it's only a matter of time before it happens in browsers too.
The problem is that the current plan seems to be to build a VM layer on top of the existing DOM/js/etc layer, which will kill performance even further. It should really replace it and emulate it.
On the one hand, I wonder how much of the wasted CPU cycles can be attributed to carelessness that could easily be avoided, or caught early, if we developers deliberately used underpowered hardware for our own machines. Speaking for myself, my main workstation, where I also do most of the testing on my desktop apps, has a Core i7-4770 processor and 32 GB of RAM. Maybe if I used something more modest, like an ultrabook with only 8 GB of RAM, I'd be more likely to notice when I'm carelessly writing inefficient code.
On the other hand, as others have argued both on this thread and elsewhere, there's a trade-off between machine efficiency and developer productivity. We may argue that it's wrong to waste machine resources when the machines in question don't belong to us, but then again, developer productivity means we can crank out more features that drive sales and make users happy.
So, looking at this rationally, consider your favorite lightweight window manager or desktop environment for Linux. Is it fully internationalized, including support for CJK input methods? Is it fully accessible to users with disabilities, e.g. blind people and people with mobility impairments? Does it auto-mount USB thumb drives? Does it do all of the other things I'm not even aware of that are required for a fully usable desktop environment covering a wide variety of machines, users, and use cases? AFAIK, the only Linux desktop environments that come close are GNOME and Unity.
Basically, the world is complicated, so real-world software has to be complicated too.
It sounds like Atom could still use some optimization though.
i18n is an orthogonal service that the DE or any other program may call into.
Is it fully accessible to users with disabilities, e.g. blind people and people with mobility impairments?
Screen readers and screen magnifiers are orthogonal services that the DE may make accommodations for, though some DEs may be lesser fit for users with disabilities in general.
Does it auto-mount USB thumb drives?
This isn't even the job of the desktop environment, it's the job of a dedicated device node manager, which the DE may call into. It's completely orthogonal and can be accomplished even without so much as a display server, because it's a different layer.
NFS in the classic cross-network *nix mount. And depending on available software (FUSE and related) you can mount the likes of SMB/CIFS shares, FTP servers, even SSH/FISH can be used via a mount.
Unix systems really aren't good at abstracting local and remote mounts and binds into a singular resource, but the parent poster's scenario was implied to be a local user hotplug, which is what device managers largely do (if even that, udev these days just maintains the /dev/disk/* symlinks and handles query requests for the hardware database).
Who does this, though? Are there reliable adoption numbers? NFS semantics are wrong in well-known, familiar ways and it ships everywhere, making it still quite popular.
There's much better network file systems but because of many factors (boat/complexity being the biggest) their adoption is minimal.
Once you move beyond single-user/simple read/write permissions there's really no easy to avoid complexity. It's just the nature of the beast that is sharing files over the network in a way that appears native to the user.
I'm the project lead of LXQt. All those things you listed can be supported without introducing much bloat at all. In DEs, most bloat comes from pure technical debt. Unnecessary libraries, duplicated code/libs, etc.
Feature bloat is another matter and is more subjective. A lot of it is down to UX.
Fun fact: KDE 4.x depends on ruby in most distros. Do you know why? Dolphin ships with a ruby script to update some folders (don't remember what it does). Nobody runs it. That's it.
Why did they allow an additional dependency on Ruby in? It seems like perl or python would be the more logical choice, given they'd already be on the system (heck.. maybe bash?)
A script that is compatible with POSIX Bourne shell (/bin/sh) would be more appropriate. FreeBSD, for example, does not come pre-installed with bash. If you want cross-platform portability, this is a better default.
For example, suppose you're developing an application, and you want to use a lightweight GUI toolkit, or roll your own as Spotify apparently used to do. Unless the toolkit is just a thin wrapper over the underlying OS's native GUi facilities, like wxWidgets or SWT, the resulting application is very likely to have accessibility problems, unless you do the hard work of implementing the accessibility APIs for your target platform(s).
The usual counter is that "optimising" takes time. And all of the time that you're spending trying to optimise before release is time that no one is able to use your software at all.
And the reality is that a lot of the bottlenecks can't be predicted in advance - who knows how may files the average user might want to load into your system, or whether your website is going to attract 100 or a million users a month? And many expected bottlenecks become largely irrelevant as technology moves on.
Take his Ubuntu example - for what percentage of users does the fact that it doesn't fit on an actual CD really matter any more? Is it worth spending weeks or months fine-tuning the distro to a point that it can be fitted onto a medium that most people probably aren't even using anymore?
Or the failure of a large team to properly manage software complexity.
It's using your time to do something else, such as improving security, or adding a new feature. Some people seem to be talking as if it automatically takes the same amount of time to create efficient code as inefficient code, and the idiot developer has simply forgotten to add the -run_faster option on the compiler.
I'm involved in a project at the moment where a team of developers and engineers have spent much of the last month trying to get response times in some very specific circumstances in an application down to an acceptable level. That's not because they are either stupid or lazy. It's because these things are hard - there's an awful lot of moving parts, and an awful lot of things that can go wrong in each of them.
When they get it down to an acceptable level, they'll ship it to the users. It won't be blazingly fast, but it'll be good enough. They could definitely spend longer tuning it further and getting it going even faster. But every day they do that will be a day longer until the features that the users want are actually available to them.
The reason there's lots of moving parts is because modern computers, OSs, languages, application servers, libraries, databases etc are built to do a lot of very complicated things.
We could have written the entire application from scratch, not using any 3rd party elements so that we had full control over every element. But that would have taken several orders of magnitude longer to deliver actual useful working software to the users.
And rewriting them, along with modifying the rest of the application to use them, takes time and can lead to further unexpected behaviour.
Here's the bits that aren't robust enough: any open source that isn't either used by thousands, or used in exactly the way you will be using it.
That 95% would still need to interact with my new 5%, and as it was never designed to do that, it could easily introduce new unexpected behaviour. Imagine trying to rip out (for example) the indexing code out of a database because it's performing badly and then rewriting it.
It's a 3rd party commercial platform for developing your own applications on. To rewrite individual bits that aren't working efficiently would be a nightmare.
As for imagining having full control over the behaviour of my code - I had that quarter of a century ago or more when I started coding, and spent weeks or months building windowing systems and low-level networking code in assembler that I could achieve in 30 seconds with any off the shelf language these days.
Yes, my code was extremely efficient (it had to be, trying to do graphical comms software on 8086 machines didn't give you much of an option to do otherwise), but the use of my time wasn't.
The third party product that we're using is the result of many years of a large team of dedicated developers. Rewriting that from scratch in order to make sure we've got full knowledge of what's going on is an utter ludicrous idea.
Then use the 10 year old version! Unless, of course, one of those few "actually useful features" is something you can't live without.
A lot of people use Word for a lot of use cases. The value of an added feature that someone needs always trumps the performance cost of adding it until the performance becomes so bad that it becomes the reason that other people stop using it.
It's weird how new trends in UX design try to make a first-time user become a genius immediately after double-clicking on the program icon. The only way you can do that is by dumbing down the software to the point it can actually be comprehended this way - which makes it much less usable and effective as a tool.
Maybe yours was faster, but I think a lot of people have rose-tinted memories of the speed of applications in the old days.
If you really need a lightweight word processor, there's plenty out there. They just probably won't have a lot of the features that the big hitters have now. And if you really want a lightweight OS, there's a fair few Linux distros aimed at that market.
What do you want your code to optimised for? Fast startup times? Functionality? Extensibility? Stability? Compatibility? Security?
None of those things come magically for free, and every day that a developer spends on one is a day that they aren't spending on another. And if they are waiting until any, or all, of those are optimised for any possible scenario, you'll be waiting a long time for any software at all.
As Blaise Pascal (or possibly one of many other possible candidates) once said - "If I had more time, I would have written a shorter letter." The challenge of optimisation v delivery is not a new one.
Don't complain that you don't need those features: it is on you to choose the right tool for the right job. If all you need is a light spell check and text editing, notepad++, vim or Emacs will do nicely. Plenty of people need the more advanced features.
That's only one thing among many mentioned, to prove a point.
The fact that it doesn't run well with only 2GB of RAM certainly affects plenty of actual and potential users. So the point stands. Not about the CD per se, but some optimizations are general and apply everywhere.
And every single Ubuntu user has complained at one point that pressing the Win/Super key does not bring the menu fast enough, or that the search results are not fast enough.
So, there's bloat.
Of course all software could be faster. It could also be more secure, more feature-rich, more user friendly, smaller and "better" in any number of other ways. But developers don't have infinite time to focus on all of them.
So they have to make tradeoffs, and the fact that most people default to the more feature-rich OSs rather than the lightweight ones, suggests that this is the place that the average user would rather the devs spend most of their time.
“wouldn’t it have been easier not to create the problem in the first place?” It would seem obvious that the answer to this is indeed “yes”.
It doesn't seem obvious at all, especially considering the implicit assumptions about whether or not bloat is actually a problem. There are three classic examples highlighted in the article of writing about this tradeoff: "Make it work, make it right, make it fast", "Premature optimization is the root of all evil", and "Bloatware and the 80/20 Myth" – but the argument about why these don't apply is a bit weak: "bloated software simply makes me sad".
To be honest, I don't find we've got a problem, generally. My experience with my computer is that I can do many more things much faster than I could in the past. And I don't think that counts as bloat.
Bloat is an invisible problem, but that doesn't mean it doesn't exist. Less bloat is more storage space, more performance, more battery life, etc. And the problem is you can't judge "bloat" standalone in one piece of software, you need to judge it as the sum of all the software the user is running. In the first equation, your numbers will always tell you "it's not worth it", unless things are really bad. In the latter, not so much.
Maybe if the Bonbon Squash Series game I have on my android phone didn't link to liballthesocialthings, I'd shave 1MB off a game which is 50MB large. But maybe if every app on my phone did the same thing, I'd have 200MB more on my phone. My old HTC desire doesn't even have half that much memory total.
You say your computer can do "more things much faster" than before - just imagine how many more things, how much faster you could do if the software didn't bloat itself up almost at the same pace as the hardware gets faster.
And you see, I can understand abstraction layers. I can excuse coding for function first, speed second. But then there's the kind of bloat that can only be explained by programmers having no clue about what their software is doing. There's literally no reason why an article should lag in your browser, or Spotify should take 10+% of CPU on standby. There seems to be a lot of pointless code that's being executed, and nobody cares, because developers often have powerful machines. Not so the general population who then has to endure such creations.
Not everyone is cut out for high performance software development. If you're not happy with what they've produced, don't support them.
Part of it is not code reviewed by senior developers. Another part is but too late in the cycle and goes in anyway because features.
The bloat is technical debt accrued in favor of shipping.
It probably happens to both types you described, but shipping is a business decision.
Honestly, an engineer that cares about the performance of their code has several options available: - design the project first, then code later. - design the code for the architecture (arrays of objects vs. objects of arrays). - use tools that can optimize out inefficiencies. - hand optimize after writing a functional slice. - document the APIs and interfaces to explain the performance expected as part of their contract.
Unfortunately, it seems as though we've gotten into a bit of a rut when it comes to this -- we rely on the awful tooling we have to optimize the code we write (yes, LLVM and friends are light years better at this than GCC et. al., but it's still not enough), we never document performance characteristics of the libraries and APIs we create, and we (at least in the OSS world) almost never design our projects to the level of detail necessary to fully understand it.
Honestly, I'm not sure how we can solve these problems if the engineers have a lack of discipline. It's a hobby, after all, right? We do this in our spare time, and nobody wants to write designs or document because that's the least interesting part of coding. See also GTK's autodocs, which for /years/ lay around as nothing more than a simple enumeration of which functions were in the libraries. Even in self documenting languages we fail at this (I'm looking at you, Common Lisp).
Optimizing afterward is a simple, if lazy, way out but post-facto optimization can only take you so far if the design is simply broken.
Since we never document or design, the problem ends up falling to our tooling. Things like oprofile and so on work, but are usually so coarse grained that digging through the code to instrument it effectively becomes a grind nobody wants to do. Maybe we need companies to develop free and open tooling for understanding the performance characteristics of the code we write. After all, the "modern" tooling we use for languages like C and C++ hasn't changed since the 80s, and it's often actually hostile to the end user, which just makes the job harder. Even our linkers often fail to do their jobs of optimizing out useless dependencies or symbols unless we explicitly tell them to do so.
Ultimately, the blame lies with us, the engineers who work on this code. New engineers joining our projects need to understand the code itself through and through. Our projects have become so large that without documentation no-one can make a clear, clean decision on what should or shouldn't be written anymore.
What I want to see is tooling that can semantically understand the code I write, and give me characterizations of the function calls I make as I write them. When I need to examine source to better understand it, I want tooling that tells me how often the functions are called, where they are called, and how. Raw text is great for writing prose, but we're not writing prose -- we're writing code. Our tools should reflect this, and give us as much help as possible.
The point you make about Linux systems being bags of Lego is interesting. That's exactly how it is supposed to be. Unfortunately, the projects we're talking about don't compose well -- they're more like Duplo than Lego because often times they subsume whole feature sets into one huge monolithic block. Huge lumps of code exist in GNOME and KDE and systemd and dbus that just simply have no business being in there.
Dbus and systemd especially fail at the whole design angle.
At the time of X's creation, DEC and IBM was looking for a GUI for their unix offerings.
And dbus came out of trying to make it easier to get the various Linux GUI toolkits to talk to each other, under the stewardship of Freedesktop (where also Systemd was housed until very recently). Thing is that RH was a big backer of that, and was out to win against the big unixes (A recent video interview with various employees there pretty much proclaim "we won the unix war(s)").
However, I'm optimistic. For one, I think that the tools we have today are superior in absolute terms (convenience, speed, ease of use etc.) than they ever were. Of course they are not exponentially better (like the hardware they run on), but they are better. Also, occasionally I see some gems here and there. There are still many developers who take pride in what they do and I enjoy finding that diamond once in a while when searching for an alternative technology to use in a new project.
But we often use different libraries that do essentially the same thing or different versions of the same library, so instead of 1 copy of libfoo.jar, I have 2 copies of libbar.jar and 4 copies of libfoo.jar that may all do essentially the same thing. Then I have essentially the same functionality in C++ (some libraries that wrap collections), Python (where maybe one of they python versions wraps one of the C++ libraries, but a different version). And of course I have a version installed in each ruby environment. Add to that their dependencies, and the dependency's dependencies, and you have a perfect storm of craptastic. So libfoo.jar version 1.2.3 depends on libbaz.jar 2.3.4 which depends on libqux 1.5.7. Let's say each one is 250k, and all I ever used was some list sorting utility in libfoo.
But I don't know what we could really do about it. You can't force everyone to program in C++ or limit them to a set of blessed libraries. I think maybe developers could be more judicious about when they could add a few lines of code and when they actually need to bring in a hard dependency an an external library. And it happens with commercial software as well. Maybe this is just the way the world will be.
That was actually a talk called "Bloat: How and Why UNIX Grew Up (and Out)": https://www.youtube.com/watch?v=Nbv9L-WIu0s
Well worth a viewing.
Downloading the depedencies for Ghost, the node blogging platform with the explicit goal of simplicity and minimalism, takes me minutes.
Compare this with the status quo when writing programs in C, where you might link to 4 libraries total, one of which pulls in 2 others as dependencies.
I've come to suspect that the super convenient package managers that all the "modern" languages have are at fault for this.
This would probably be quite tricky in Java land where reflection does add new entry points, but it could be used to solve the problem of "I only need this one function from this library, don't compile in anything else".
I was personally quite surprised when I ported from code from Node to Java/Groovy and the resulting shaded JAR was > 70MB, I think at some point it peaked above 110MB. I don't know what I changed, but it's down to 35MB now. But the code that we've written in house on that codebase boils down to 1MB. But besides figuring out that I don't want to make local builds I scp to staging (because scp is terrible), these numbers are all completely equivalent for writing server-side software that runs on dedicated machines.
We could certainly make it more efficient, but there's exactly zero business case for it.
How much of that was Java's fault and how much Groovy's, I wonder?
Every normal compiler implements simple (i.e. module level) dead-code elimination already.
EDIT: Of course you could use static libs, which does pull in only used symbols, but then you cannot share them across apps and update independently.
I implemented a tree shaker for my lisp and was very happy with it, esp. for delivery. Like Go does it nowadays.
I feel like our computing infrastructure has gotten to the point that dynamically linked libraries are no longer a good choice. I think dynamic linking has only caused us problems at work (devs install Node deps on the staging server, forget to tell ops, service crashes when deployed in prod), and the memory/disk/transfer overhead are practically irrelevant at this point. The only remaining reason to have dynamic libs is the idea that they can be updated without help from upstream, but that really only works if the software is compatible with the latest libraries, which isn't always true.
Supposedly ProGuard has some cross-module dead code elimination for JARs, but I haven't tried it: http://proguard.sourceforge.net/
If you only use one or two features out of a lib, why are you dynamically linking them in? If you do a static link, the linker can at least remove most of the bloat you don't use.
It's the editor which has frozen, not the whole computer. And the plugin in question is a fairly complex autocompletion / analysis plugin for python. There's a bug listed in that thread, to do with it scanning all the files millions of times, but the big problem actually isn't bloat, in this instance, it's the non-async nature of vim. neovim should, I hope, solve the 'totally unresponsive' problem, and hopefully someone fixes the rope/vim interaction re-scanning all the files bug.
But it's not bloat, per se.
If your whole computer is locking up due to this, then that's a scheduling issue, and you could try switching scheduler, or lowering the 'nice' priority of that program.
Not that we should have to care about such things - that's pretty terrible.
It isn't about "feeling big" at all. You can have large, feature-packed programs that still respond quickly and don't fragment.
There's tons of code in Mozilla that isn't necessary to do the one job the browser was built for.
I don't need code that manages bookmarks. At all. I don't need code that stores history for years. Honestly, I don't /want/ cookies persisted /at all/. I don't need themes to browse the web, and a reorganizeable UI is /never/ what I'm looking for in a browser.
I don't need WebGL. I don't need Canvas. Hell, I definitely don't need syncing of all this extra bloated data around to services somewhere in the 'net, either. Frames? What about floating frames?
Half this stuff is better left to tools that are built for it, the other half is junk I don't need, or is junk that actually creates more problems than it solves.
https://en.wikipedia.org/wiki/List_of_web_browsers
Based on your stated requirements, perhaps you'll like Dillo.
I actually /do/ use other browsers like Midori, w3m, and surf. The problem doesn't disappear, though, because it's up to all of us writing code to be better at it.
Bookmarks, for instance, can be handled in a much cleaner and portable way than isolating them to the specific browser implementation. The same can be said for history, authentication, and so on.
Also, to be fair, I wasn't arguing for minimal codebases -- that was your point. I was providing examples for the GP.
Actually there is a whole community around this http://suckless.org/
Configuration files introduce a new point of failure (file not present, file corrupted, file not readable), a point of slowdown (how long does it take to read the file from disk, how long does it take to parse and load the file), and a point of complexity (parsing even ini files is pretty complicated, if you want to support the myriad of ways we write them).
All of this for values which change once every... how long? Once a year? Once an install? Or for most configuration values, never?
I still use configuration files, because I'm OK with including third party libraries... but I can certainly understand why some people may not be.
Suckless is about minimalism and speed, and if you want speed, you don't want extra junk like config files clogging up your code's mainline.
It's certainly easier for the programmer to just leave some of the work to the compiler, but it's at the cost of being a complete pain in the ass to packagers and users.
Do you even have to do much work yourself? .Xresources is already parsed and loaded for you by xrdb if you can't afford the cost of an extra 100 microseconds doing it yourself. Is the API to interact with that mindblowing horrid or something?
> All of this for values which change once every... how long?
When I'm configuring software to taste, several times a minute. And considering this is likely to be my first exposure to the software, it better not suck completely.
But if suckless maybe takes their fight against bloat too far, it's only because hardly anyone else seems to care at all.
Free open source software bloat makes me sad.
However, that bloat is winning on its own merits. For instance, GNU/Linux distributions which are minimal are not popular. The popular ones tend to be the bloated ones.
Vim has gotten a lot bigger in 20 years, yet I'm not going to jump ship to a smaller vi implementation.
FOSS makes us confront the fact that we actually seem to like bloat. We inflict bloat upon ourselves --- there is no monopolistic software vendor there to blame.
A minimalist XWindow installation was possible with a 40 megabyte hard drive and 4 megs of RAM over a 386 CPU. (That's similar specs to what 68K-based Sun workstations had perhaps a decade before that.)
You make a name for yourself in SW development by adding features that others deem useful. You get money (in commercial setting) or prestige (in freely distributed software) if you seem to be producing something useful. It's trivially easy to recognize a contribution that adds code, but it's hard to recognize a contribution that means absence of something (e.g. absence of bloat, absence of memory hogging, ...). You get inclusion in newer, larger or more important projects easier the more you make yourself recognized.
So any useful software gains more contributors that add things than those who remove things. Commercial code can and does gain developers if it earns money, open source software gains developers if it is deemed useful by programmers.
This is a force that shapes both the group of involved developers and the resulting software in a process akin to evolution. In both cases the selection is biased towards adders of code. There's also always possible to add improvements that help some use cases and audiences. On the other hand, arguing for removing something or limiting might make you unpopular (you are seen as an obstacle to everlasting progress) and removed from the group of developers. Rare people have enough recognition and clout to prevent inclusion of something (Linus Torvalds is one of them - he can get away with rejecting patches to Linux kernel and he can play the role of the kernel guardian).
On the whole, software expands until it fills the resources available.
Not writing a compiler for your scripting language saves you years.
Not choosing carefully what features to include in a program, or what packages in a system image: design time saved again.
It also saves considerable time if a user wants the program to do something, and by golly, "look, they thought of that already: there is a feature for it!" Seconds later, the user is just doing whatever they need instead of surfing the web for workarounds or calling support.
Having all the features also increases the flexibility of customization: you have more choices about what you can remove to create smaller images which have specific feature sets. You can hardly remove anything from a minimal image to make a specialized custom image.
I.e. by not making a program as minimal as possible, compact as possible or fast as possible, you are making some kind of trade off. Though the program is worse in some regard, there exist parameters (of something, not necessarily the program) which have gotten better: if you think about it a little, you can identify what those parameters are. The worsening in some of those parameters in a "bad trade" is what makes premature optimization "evil".
The axiom is only valid in those cases because if you've designed your system well, the points for optimization are already obvious and the code can be written mostly optimized the first time round.
While I really like the language itself, statically linking everything is creating exactly what this article describes.
Burn karma, burn. I don't care.
Ditto, static linking leads to a lot more optimization opportunities, both in terms of performance and in terms of size. I would be very surprised if that 13 x 60% of a 2MB library was actually anywhere near 15.6MB.
a) You are failing to account for the bloat caused by the presence of the dynamic linker, the dynamic loader, all the code that must operate under the assumptions of a dynamically linked environment, and the various auxiliaries used to treat shared library hell issues like WinSxS and libtool.
b) Most modern static linking implementations do things like sharing of text segments across processes and other dedup techniques, so they're not actually that bloated at all.
Are you saying that two Golang projects sharing the same (large, for the sake of the argument) dependency would not take (size of the dependency * 2) on i.e. the Ubuntu live CD (as was the example in the fine article)?
X uses 1 symbol l1 from L, Y use another symbol l2 from L.
With dynamic linking you need the whole library, no matter how much of it you use. With static linking (and --gc-sections -fdata-sections) you have parts of L compiled into X and Y binaries, but just the functions used (and their dependencies). It's possible that l1 and l2 share dependencies - and these will be duplicated. But still static linking is likely to be net win.
Dynamic linking is mostly about maintanance.
I would not be surprised if those two Golang projects take less than the size of the dependency. 2x is a degenerate worst-case.
With static linking, you only need to pull in things you actually need (and you can optimize from there!), whereas with dynamic linking you need to pull in the entire dependency regardless. (You only need to pull in one copy regardless of how many things are using the library, true. But you need to pull in one full copy always.)
There are very few cases where you actually use the entire library in a project - and if you are indeed using an entire library in a project your project is necessarily large enough compared to the library that it's not much overhead percentage-wise.
Thanks for the insight, I now understand why my comment was misguided.
One of the best features I found in Golang was the statically linkage concept. There is just to much trouble with tools that come either in dozens of versions for different OS versions (Ubuntu, Debian, Mint, CentOS .... all Linux, but no Linux is the same Linux ..., it goes on: Ubuntu 12.04, 12.10, 14.x, ..., Debian Sid, Debian with and without systemd ...)
While statically linking might add some bloat, I don't see, that the real bloat-factor is because of this. The real bloat-factor for example comes from programs that rely to much on template libraries (yes, I mean C++), that just unroll everything into new statements and because many even simple tasks have a multitude of solutions in the code ... each aside each other. When hundreds of developers are developing on one application, many of them invent the wheel again ...
Wouldn't that actually be better?
Then I don't have 5 versions of a huge dynamic lib for 5 programs that all use slightly incompatible version and that only use 1 or 2 methods.
No, not if it works like it does in C and C++.
Lisp used to do that, back when Symbolics still existed; it was called tree-shaking, as in "shake the call graph tree to see what falls out". The main function calls functions FOO, BAR, BAZ, which call FROTZ and QUUX, which call... and remove everything not in that tree.
Of course, this gets to be equivalent to solving the Halting Problem the moment the EVAL function is called, so if the tree-shaker found any mention of EVAL anywhere in the code (probably after it had removed the provably dead code) it punted and linked in the whole Lisp runtime. It was the only viable solution, really, especially given how creative Lispers can get...
Go uses the Plan9 compiler architecture, which means that its linker knows more about the code and is responsible for more optimizations than its Unix equivalent, so it quite effectively eliminates dead code.
Of course, libraries are often highly coupled within themselves, so not much code can be eliminated, and then they might pull in additional dependencies, etc., etc.
See for example the LispWorks documentation: 4.6 How Delivery makes an image smaller, http://www.lispworks.com/documentation/lw70/DV/html/delivery...
Well, writing a text editor is far more complicated than it seems at first place.In order to support big files (50mb logs for instance), you need to make all your code rely on complex memory management, streaming and caching. Maybe js is not the right tool for the job in order to implement these features. I always thought that atom should be build in Qt with an api that can interact with JS,a bit like Adobe products for instance. That way you still allow JS plugins, just like Photoshop while getting a max of perfs thanks to C++.
Atom does have C++ code, since is essentially Chromium. However, try a naive implementation in Assembly and you'll find that the Emacs Lisp buffer manipulation code will outperform it.
In fact, if you try to use C strings (null-terminated), you're doomed from the beginning.
I would rather see bloat and new ideas, than only experts working in a problem domain. My best ideas come from a marriage of understanding a single problem space, and a wonder what I can do in another. Bring on the bloat.
Not perfect (especially as writing things as plugins tend to introduce a certain amount of bloat regardless), but it can be better regardless.
I get annoyed by having to reboot my TV or radio when they freeze up, or by having delays to my channel changes, or having to navigate complex menus to do things that would have just been a physical button click before.
(this is all part of the same problem as bloat IMO).
This is where the likes of Apple or Dyson do well (or OK) I think. It's not necessarily that they're brilliant at what they do, it's just that the competition is pretty awful.
A major reason for 1-2-3 becoming dominant was its speed and memory efficiency; it was written in assembler. If you had a 640K machine, 1-2-3 let you write bigger spreadsheets than anybody else could.
But 1-2-3 was killed by Excel because it lost the feature race. It's origins in assembly language gave it a significant disadvantage in this race.
Yet the winner was written in C++ for many years, while the world was distracted with the less inefficient VB.
Now some parts are .Net, but the crucial parts are still C++.
And in many cases, it's slow, CPU hungry, and memory intensive. Those three characteristics are the very definition of bloat to me.
Isn't that what bloat is? By definition, something that's shipped with "basic functionality" should not consume so much.
So ti's a tradeoff. On the margin, many more people are likely to write a halfway decent Atom extension than learn elisp and write one for emacs.
Wow. Talk about a self-refuting argument. If it's hard to figure out where to optimize a RUNNING system, how is it supposed to be easier to do so when the system is being created?
Arguing "bloat" is "bad for performance"? No problem - graphs or GTFO. Otherwise all you're left with is the emotional "OMG NUMBERS ARE HIGHER NAOW THAN BEFORE" drivel.
Windows went from 7MB required (95) to 6GB required (Vista), 1000x bigger, in just 10 years. It's true in file formats as well and verbosity is one of the reasons XML was/is so hated.
There is an irresitable urge to make things bigger bigger bigger, and huge cpu, mem, and disk resources are essentially free.
Is using more memory resources for caching "bloat"? It irks me when someone pulls a task manager screenshot, orders by memory usage, and picks one application out of it. "See, program X is using XXXMBs of memory! Bloated!" Well, perhaps it is, perhaps it isn't. You are still using it, so it is probably doing its job well. Perhaps that "wasted" memory is being used for caching.
This is specially true when people compare OS memory consumption. If an OS is not using all your memory, you are wasting it. It should be using it for something, be it caching, eager loading, whatever. As long as you can quickly reclaim it when needed.
Same goes for CPU usage. "It's using 100% of my CPU!". Well, did you tell it do anything? If so, isn't that what you want? You want the task to complete quickly so it can go back to idle. Now, if it is supposed to be idling, and using a lot of CPU, that's a problem (Slack, I'm looking at you).
From the article:
> The same CD that hasn’t been able to fit Ubuntu since 2011 still fits approximately 150,000 pages of unformatted English text without any compression.
Well, yeah. But one-dimensional metrics are useless. What about the number of packages? Did it increase? Is Ubuntu now packing high-resolution artwork, to be used with our 5k displays? What else has changed? I'm certain that the CD is not all source code, so the English text comparison is meaningless.
>The burden of selecting software that is not bloated is entirely on the user. The default is bloated, if you want the unbloated version, you’ll have to work (search) for it yourself. And in many cases (e.g. anything that needs a web browser) such a search may not even be fruitful.
Sometimes, the opposite is true. Take Vim and Atom, which are mentioned in the article. You can add packages to them, so increasing the perceived "bloat" is entirely up to the user. Unless the user takes the easy way and installs a "vim-full" package. Which, if we have memory and cpu to spare, isn't usually a problem. We are talking about VIM in the age of laptops with 16GB of RAM.
> There are very little tools available to to help the user select unbloated software. Very few packages make any claims about their storage and runtime charactaristics at all
Now, here I agree. It is something difficult to measure.
> Over time, the battle against bloat is always lost. Even Ubuntu, which has traditionally presented itself (besides other things) as a method to extract a few extra life-years out of old hardware, is mentioned in the list above. In other words: it’s only less bloated than the alternatives.
Yes. And yet it is inching closer and closer of being "ready for the desktop". It has to cater to a lot of people, which means bundling lots of features. Still less "bloated" than other commercial operating systems. Considering the number of available packages, Ubuntu offers a very good deal.
> or even simply the introduction of flash screens.
Oh, now we are in agreement. See, if you are waiting for a splash screen, you can't do your job. The software is not doing its job. Therefore, the amount of time being spent at the splash screen should be reduced, so that we can eliminate the need for one. In the bargain, eliminating the code and artwork for the splash screen, thus reducing bloat.
Still, in some cases, this is unavoidable. Take games. They will often present splash screens when loading a level (if they have such a concept). Some of them will even go as far as present animated 3D geometry, using the GPU (and CPU). Which are mostly idle anyway, waiting for I/O. Sometimes, one can devise a better way of packing the files, to reduce loading times. Even more so when the actual serialization mechanism is inefficient(hi, Kerbal Space Program). But what they are usually trading off is a lag-free game play, in exchange of a loading screen (think of it as warming a huge cache). Is that bloat? The assets are huge, because we want them to be.
Are you developing for an embedded platform? No, mobile doesn't count, they are effectively shrinked PCs from a few years ago (with non-mechanical storage even). If so, then worry about every CPU cycle you are using, as well as storage. Build your own linux distribution if you have to, compile with the exact flags for your platform so you can generate optimal code.
Running in a battery powered device (no matter the size)? Then try not to use too much CPU, and certainly not constantly. Batch stuff, run in bursts, get it over quickly. Disable any non-essential tasks.
Is a background task? Try not to disturb the rest of the system, please.
A foreground, interactive desktop application? You are likely the focus of the user's attention, do whatever it takes to minimize the latency! The only reason not to gobble all RAM is that the user may be running other stuff too. And the reason for minimizing CPU usage is heat and power. Other than that, I bet a user will rather have an application that is making use of all resources available, if it means getting work done quickly. And feeling... "snappy"!
If the user is not being impacted, then I don't see the problem. My issue with Slack, for instance, is that it uses a lot of system resources and doesn't feel fast in return.