Dependency
xkcd.com
xkcd.com
Here's `pdf-redact-tools` from Homebrew: https://github.com/kach/tower-of-power/blob/master/gallery/p...
Here's the CS core at Stanford (dependency = prereq): https://github.com/kach/tower-of-power/blob/master/gallery/c...
Edit: partly because I also know the original author was denied a job at Google. Imagine the productivity gain this team has given to a company the size of google.
https://www.propublica.org/article/the-worlds-email-encrypti...
But I also remember the takedown about no one actually relying on email encryption for security:
https://news.ycombinator.com/item?id=3080437
Voluntarily maintained by a single person for decades until hit by a big lawsuit and ICANN stepped up to help.
Yes, it's an important piece of software. But it's not "required" for any services itself.
digraph G {
a -> {x y z}
b -> {x y z}
c -> {x y z}
}
Suppose we have two components-- X and Y-- which depend on two other lower-level components-- A and B. How would you draw this as a block dependency diagram without repetition?
If both X and Y depend on B, then X and Y must be above B, and both touch B. If X and Y are orthogonal rectangles, then the entire X-Y edge is obscured from below. This means that another lower-level component A must can only interact with either X or Y. You'd need to allow X or Y to no longer be orthogonal rectangles to overcome this, or need to give up on the vertical organization of dependencies.
1->2; 1->4; 2->3; 3->4
Unless 4 is slanted.
Probably, however, the graph would horrify me as well.
https://npm.anvaka.com/#!/view/2d/webpack
And yes, the dependency chains become absolutely insane for some projects.
Click "Raw" to see the image
That's a nice instance of literate programming. I don't come across those very often.
https://medium.com/@behdadesfahbod/if-you-read-one-thing-fro...
His tweet: https://twitter.com/behdadesfahbod/status/129532694517805465...
The library: https://github.com/harfbuzz/harfbuzz
This is a hard and terrifying read. I really feel for him.
"It took me weeks to regain access to my accounts, which were disabled after my employer (Facebook) was notified and embarked on disabling them."
I guess they've disabled his account to prevent access of the intelligence services to any sensitive data.
> Someday ImageMagick will finally break for good and we'll have a long period of scrambling as we try to reassemble civilization from the rubble.
EDIT: don't forget to always hover over the image. Half of XKCD's fun comes from those tidbits.
That is, should graphicsmagick account for quirks introduced by imagemagick? If it's meant to truly be a drop-in replacement versus one with some strings attached, I would say the answer is yes. Otherwise, you can't easily just drop it into a system and let it run.
Systems built with libraries around long enough as imagemagick likely have all sorts of workarounds built in over the years for that kind of behavior and any deviation from imagemagick might break things more than one expects. We've all been bitten by libraries that claim backwards compatibility with previous versions, only to find out there's breaking changes in some non-trivial use cases the library devs didn't consider. Add those complications on top of switching to a library that is supposed to be a drop in replacement and it can get pretty messy for what seemed like a trivial task.
The last time I made heavy use of ImageMagick/GraphicsMagick, I was suprised how often I would find a feature present in one and not the other (often things like drawing gradients).
And it's not like one has a superset of the other's features: I once painted myself into a corner and created a project that depended on both ImageMagick and GraphicsMagick.
consider: most state-run web apps that need an image upload for personal ID images, or anything else. Ugh, this happens frequently for me and just frustrates me to no end.
EDIT: I suppose this is why you're saying ImageMagick is king right now.
> GraphicsMagick is originally derived from ImageMagick 5.5.2 as of November 2002 but has been completely independent of the ImageMagick project since then.
More info on the reason for the fork (at the time)
That's leaving a pretty big loophole there and in the case of SQLite you'll need that. To re-create something that solid will take a lot of effort.
Then, there are major incompatibilities between versions 6.9.x (which is what is even in the latest Fedora, likely other distros as well) and 7.x (which is probably what's sane to use in this day and age). Which means that to use 7.x, you're likely to have to roll your own and tuck it in its own LD_LIBRARY_PATH. That hurts.
The cherry on top in my particular case is PerlMagick. It's not installable in any way other than with the ImageMagick itself. You cannot have just Perl modules, which you "perl Makefile.PL && make && make install".
When you see separate packages for ImageMagick and PerlMagick in your distro, it means that the maintainer has compiled the whole ImageMagick, then torn it apart. A corollary to that is that PerlMagick is tied to binaries it came with real tight, and updating one means updating the other most of the time.
Then again, it's very likely that you use some other bindings to ImageMagick and my pain is not like your pain.
But then again we also use groupware/chat programs these days that occupy more RAM on the workstation OS than the entire amount of physical RAM in all of the BSD/Linux servers of a mid-sized ISP 18 years ago.
It doesn’t use intermediate disk store for frames AFAIK so you’d better have enough RAM.
In fact, you’d better just resize with ffmpeg and then convert to gif there.
Lanczos is much less easy to write, and is even harder to make it run fast.
And there are things like subpixel positioning (it is way too easy to make resizer that shift image by 0.5 pixel)
One might argue that it's because it's more popular than other similar projects, but it doesn't happen as often with, say, Pillow. And I don't quite remember PIL/Pillow having holes as big as ImageMagick.
Also, try actually programming against it. Whatever language you choose, it's going to feel clunky and alien. It feels alien in C. It feels alien in Perl. Even in PHP, the match made in hell, it feels thoroughly un-idiomatic. I'm not even surprised they have to patch it up now and then.
While "rewrite it in Rust" is a cliche, I think ImageMagick is a good candidate!
[ tech spouse takes a look at the image set ... rotated, various sizes, even different formats ... reaches for convert(1) and wonders why non-tech spouse can't do this easily ]
Additionally, if you have user CSS in your browser, you can force it to display the title text even in the official xkcd web pages (CSS has a command to display the text of HTML attributes).
[0]: https://gist.github.com/ldd/9b576bb6f0ac99a0a7895eaf1aae7802
Netpbm also does this; you can use the "pamdice" program (together with "pngtopnm" and "pnmtopng" to decode and encode PNG). However, pamdice requires creating intermediate files, while ff-dice doesn't require that.
So, you have two more things to try if ImageMagick won't work.
Ironically, trying to good for `ff-dice` or `pngff` sends me back to this thread!
(Although there are a lot of files, you will only need to compile the files that you need, and you can delete the files that you don't need.)
Props to ffprobe too - that’s part of our file upload + identification + sanitation process.
In summary:
H.264 has the widest adoption of all modern video formats.
H.265 is Microsoft + Apple.
WebM is Google and Firefox.
H.266 came out a few weeks ago.
Are there any other solo devs who get "dependency anxiety" and don't want to start working on a project because you worry that you'll have wasted your time if one of the hundreds of dependencies breaks something, and there is nobody to fix it (like a crypto library for example)?
Finally, the beauty of open source software is that you can fix things yourself! Found a package from 10 years ago that works 99% of the time, but you just need to fix one thing or add one tiny feature? Not a problem, you fork the repo and add what you need.
Don't you need proper domain knowledge to hack into the some obscure error for svg conversion inside of cairo? All you were trying to do is to display the company logo.svg from your graphics department who exported it on Illustrator version 5.0 because our company is too poor to go to subscription model for Adobe. May be something goofy in the svg data.
You spend next days fixing this open source package, ranting on stackoverflow and finally have to concede and file an bug report on Github.
It gets rejected because you have not gathered all the necessary details to file a bug report including proper stack traces and operating system information.
Time -- yes.
Usually any ecosystem has a core set of dependencies that are heavily relied on by most or a lot of projects as the go-to solution for a given problem. Then there are often sets of dependencies where there are several popular well maintained choices to pick from depending on your needs.
Where it gets a little trickier is things that are a bit more niche. That being said, most software development just wouldn't be possible without standing on the shoulders of giants so to speak cos it's just so time consuming.
They're far more of a benefit than a detriment but you have to learn to assess them and make choices based on your individual considerations.
(a) run it in an environment of ten years ago (otherwise other platform dependencies may be unavailable, or platform behaviour may have changed);
(b) ensure that environment is already fully set-up (otherwise internet systems from which it fetches other dependencies may be unavailable, e.g. due to a service being taken down or certificate changes); and
(c) are using strictly offline-capable software (e.g. if a commercial library does an activation/licensing server check, that may fail).
In other words, to be confident, you need to freeze a snapshot of the entire system, which is obviously terrible for security.
If these conditions are not met, things may or may not work. Some platforms will be more robust over time than others. Scripting languages are generally atrocious, with an installability half-life well under ten years (I’m going to guess something around five years). Compiled languages are a mixed bag. Rust is one of the best options out there, so long as it’s pure Rust code only and doesn’t include any bindings on more build-fragile code.
I shall give a few examples of the sorts of problems that can occur and have occurred to me in the last decade.
• In 2012 I wanted to install IE6 on a Windows 2000 machine (don’t ask). But the underlying infrastructure at Microsoft was gone, so even if I had the full installer, it failed to install.
• A few years ago I wanted to install something that depended (transitively) on some key foundational Perl package, but due to recent Linux kernel changes that package’s tests no longer passed. (Perl defaults to running the tests when you install. You can disable that, but clearly the software isn’t running as expected.) Fortunately the package was maintained, so I just had to install the newer version, overriding the stated dependencies.
• You could be trying to install something with Node.js and it doesn’t work beyond Node 0.12 or 4 or something, so you’ll need to figure out how to get that old version of Node before you have any hope of running the project. And that probably means “figure out how to build it from source”, which is harder than you might hope, because it probably depends on old versions of libraries that used to be easy to get from your OS’s package manager, but are now unavailable, so you’ll have to build them from scratch too, recursively.
• I used to use Hyde to generate my website. A couple of years ago, five years or so after it was abandoned, it had become very tedious at best to install anew, due to being part of the declining Python 2 ecosystem. A few years ago I think I tried setting it up on Windows and gave up after a while, though that’s hardly a fair comparison on its own. On my server, I had used https://aur.archlinux.org/packages/hyde-git to install it, but that’s no longer installable, because it depended on packages in the community repository that have been removed and have not resurfaced in AUR. This is probably not insurmountable, as you can probably install the package in other ways, but it indicates that the way that I had installed it has broken.
I had two or three more examples in mind, but I’ve just been called for lunch.
So just one more note: use lockfiles, they’ll normally make things more robust, but be ready to need to update some of those dependencies in order to get things to work at all a few years down the track, and you’ll be practising trial and error over which.
It just requires strong responsibility and good practices.
I’ve been doing software dev for close to 20 years. I’ve been watching various frameworks come and go for years. Somehow React gained precedence in web land, but based on the last 6 years of dealing with that hot mess it’s just an even deeper level of dependency hell but go a single framework.
The recent work I’ve been doing has been entirely back to basics. It’s extremely satisfying to write web software again that’s fast, easy to deploy, and easy to debug.
And if a dependency breaks, _you_ can be the one to fix it. Just, you know, don't depend on closed source code.
Also, some things are going to be around for the foreseeable future, like POSIX, Win32, or PostgreSQL. Debian Linux isn't going away any time soon, nor is TCP/IP. Too much of our economy depends on TLS for it to be horribly abandoned for long. After a decade or so you will get a feel for what things you can assume will be there for you and what things are ephemeral.
The net effect is that there isn't dependency creep where any team can just continuously add dependencies to a project without their ongoing justification and visibility.
For personal projects, I'm very liberal with using dependencies that can get something in my mind functioning quickly before my motivation dries up. I think I shared the OP's anxiety regarding at least the longevity of what I write until I came to peace with the fact that my software will not function forever, and likely not for the rest of the decade. Relevant Tom Scott vid: https://www.youtube.com/watch?v=BxV14h0kFs0
Interestingly some of my Win32 code broke this week after the latest update was released. It was in a fairly obscure part of the system and I was pushing at the border of what was possible. None the less I have 500 customers running around with their hair on fire at the moment and it's making me wonder why I am continuing in this business.
That being said, just start. Build something. Make it work. Learn. You can figure out problems with dependencies going out of support when you get there.
Another huge factor has changed my thinking on dependencies too. My employer has legal and security requirements around all dependencies. Anytime a new dependency is considered, it has to have the license reviewed which takes time, and known CVE's have to be addressed (patched or mitigated otherwise). I consider these good since it helps in the sense that the costs of dependencies forward, but certainly a pain.
So I guess my perspective change is simply one of trying to understand the trade-offs. Some are well worth the cost because they're cheap, you won't get much benefit of your own, and will get benefit from a common one for everyone developing in it (testing libraries are one example). If a UI library is applicable to your project, of course a modern common one is a good choice, though you will pay for a lot of dependencies. Then there are other options which are more context-sensitive. Examples that come to mind are date/time libs, math libs, collection helper libs, etc. Sometimes they make sense to have, but oftentimes today can be substituted for a simpler function, though this can change with time too.
This really just turned into a ramble. So I guess I'll conclude with this: if you're getting started and learning, maybe try and skip libraries until you really feel the pain and then introduce one when you need it. It may help to build up piece-by-piece and reduce the anxiety you feel about it.
0 1
Wish you the best.
I think the dependency anxiety comes from a mixture of knowing that solving the problem would be to use a language like Java or Go, for their rich frameworks and simplicity. What holds you back and creates the anxiety is the downside that the tech stack stops being sexy and interesting.
On crypto libraries, you've got options here. you can not do crypto in your app and push that into infrastructure and use some other app as a proxy to handle that work. This gives you freedom from issues because you can manage that layer by swapping in and out different ssl proviers/clients.
One way to avoid this is to choose a programming language (e.g. Python or Go) that has a "batteries included" standard library. If you choose your dependencies from that standard library whenever possible, it's likely that they will be maintained for as long as the language itself.
Using the standard library may not be the fastest way to solve your problem, and it won't be the "coolest", but it probably will be the approach with maximum longevity.
Unfortunately this solution doesn't work quite as well if you're using a language with a poor standard library - for example when writing JavaScript code to run in the browser. I wrote one project using an open-source HTML canvas library, only to see the library abandoned shortly after. Since then, I've avoided those libraries in favor of using the raw canvas API - it's significantly more work, but it will be supported for decades instead of just a couple of years. I've decided to avoid Bootstrap and its "long-term support means 3 years" in the same way.
> Even having someone paid to do the work doesn't seem like it matters anymore
In fact, I think commercial solutions may be more likely to be abandoned than free ones. I'd trust Python's community of enthusiastic volunteers to maintain their language for longer than I trust Google to maintain Go.
golang is quite notorious for its dependencies pulling in tens or even hundreds of other dependencies
The reason I recommend Python or Go to people like the OP, who do mind, is that their strong standard libraries allow you to mitigate the problem by minimizing, or even eliminating, those direct third-party dependencies.
Actually at my current job one of the tech leads decisions for choosing Go was exactly this.
Depends on what those "other languages" are. For example. golang has nothing on the Java standard library (e.g. what does golang have to even remotely compare with `java.util.concurrent`?)
Plain wrong. Just stop trolling against Go.
1. The point is that you don't need as much third lib deps with Go or Python due to the huge and useful standard libs.
2. For exaclty this reason the third party dependencies don't need as much other third party dependencies.
Please assume good faith when responding to people: https://news.ycombinator.com/newsguidelines.html
However, there are a few outliers. Most famously, the Kubernetes client libraries are made of hundreds of packages because of how they're structured and autogenerated (due to lack of language-level generics).
(An unpopular opinion - The web is a hierarchy of hacks and patches because the web browser was never made to do what it's being asked to do.)
Edit: here's some YouTube courses that are very introductory https://www.youtube.com/watch?v=wsGpj-FVjGs&list=PLZZqoiUyRB...
When installing React, I received errors and depreciation warnings in the latest build available. What kind of confidence is that supposed to give me when something distributed by one of the largest tech company in the world seems unreliable?
Even later, I've faced numerous issues where I had to manually find a fix online since a random dependency disappeared or new version was incompatible with the rest of my entire dev tree.
If one of those base level things break, everyone who depend on it, will come together and replace/fix it instantly.
https://en.wikipedia.org/wiki/Heartbleed#Root_causes,_possib...
The industry's collective response to the crisis was the Core Infrastructure Initiative, a multimillion-dollar project announced by the Linux Foundation on April 24, 2014 to provide funds to critical elements of the global information infrastructure.[192] The initiative intends to allow lead developers to work full-time on their projects and to pay for security audits, hardware and software infrastructure, travel, and other expenses.[193] OpenSSL is a candidate to become the first recipient of the initiative's funding.[192]
After the discovery Google established Project Zero which is tasked with finding zero-day vulnerabilities to help secure the Web and society.[194]
Look at the recent security issue with lodash. lodash is a dependency of a huge number of javascript and node projects. For people who would say "don't use tons of random libraries", lodash was a great choice; it was almost the equivalent of a standard library for JS before ES6. Problem was, for all intents and purposes it was abandoned about a year ago. Couple months ago a security issue was reported, it wasn't fixed, and then 'npm audit' started failing builds with lodash as a dependency. It's like all of the sudden half the node ecosystem started failing.
The problem was not that the bug wasn't fixed, it's that the original author wasn't really involved any more, and it took a long while for other people with commit rights to figure out just how to get the build working. But the problem with any reasonably complex JS project is that there could easily be thousands of references to lodash. NPM doesn't make it easy to essentially say "I'm changing the namespace of lodash to mean this instead of that."
Dependencies are pretty unavoidable, and I think the software engineering community will actually start to get better at handling what happens when a widespread dependency needs to go on life support.
I don't see how this is causally related to that dependency being thanklessly maintained by a single person. Maybe I misunderstood what you are trying to say.
On the other hand, in market capitalism, economy of scale is a powerful force for conglomeration that sometimes results in an anti-competitive reduction in consumer choice.
And then of course, manually approving each hours/month estimate above a certain amount i.e. 10 hours per month.
Not only is this a horrible metric of value, it’s ridiculously easy to game.
It’s pretty hard to measure objective value of software. I think the closest is letting the market set a price and that has lots of flaws.
One of the upsides of OSS is that it bypasses that whole aspect.
I think the best approach is to let projects choose their own license according to their goals. If they want to charge, let them. If they want to be free, let them.
- allowed the code to be visible and available as a general rule
- allowed an individual user to use it and modify it for non-commercial use
- disallowed redistributing modified versions (distributing patches would be fine)
- disallowed commercial use
Basically, a license that would allow free experimentation and grassroots adoption and all kinds of open-source goodness for individuals, while not giving FAANG profiteers a handout for RMS' ideological reasons.
Even simple stuff like note taking where it’s 95% personal, I still want to use at work.
I’d rather seek out an open source tool.
I’m not sure what the motivation is to prevent FAANG’s use. All of those companies contribute lots of open source.
Would you rather be back in the 90s where the giant tech companies kept everything proprietary and didn’t contribute anything back. Early 90s Microsoft, IBM, Oracle, Sun sucked and I’d rather have FAANG contributing their open source stuff nowadays than make them pay for projects and stop contributing.
Serious question. If you are devoting your working life (that is, time that would otherwise be dedicated to remunerative work) to OSS without compensation, then by definition you probably don't need the money, right? Perhaps because you are already wealthy, or have a family that will support you. There's nothing wrong with this of course, many of us would contribute to OSS for free if we didn't have mortgages and families to pay for. It's basically like doing volunteer work after retirement.
Put another way, are there many people devoting their entire working lives to OSS with no other compensation or resources, essentially going bankrupt to write OSS? I'd imagine very few people are in that category, or if they are, can't sustain it for very long.
I guess because I was not living hand-to-mouth, I didn't "need" the money? But I wound up nearly going into debt, because it took another year to get a job after that--turns out that maintaining an open-source project is not an asset for employers but a liability. I had an offer from a HugeCo for decent money, but I turned it down because it would basically have precluded me from working further on this project which is only now starting to walk on its own.
It's not my entire working life, by any stretch, but those 2 years were 5% of it, and I figure that cost me $200k+ in "lost" wages. And that does not count all the "free" time I've spent since, fixing bugs and helping people use it, because I love the thing and I want other people to love it too. I'm glad I created it, it's very useful to me and other people who do data exploration in the terminal, but now I have to accept that I probably won't ever recover any significant amount of that value, because...people don't pay for open-source software.
More that you made the conscious choice to work without pay for 2 years, and you and any dependents didn't starve or end up homeless in the process. That's something few people can do.
Besides that, it's in extremely poor taste to marginalize someone's sacrifice of two years just because you think they're privileged. It's usually better to make much of someone's contributions than to minimize them.
I didn't marginalize it. I said the ability to do it is rare. Those are very different things.
I think it would be great to live in a society where people can do what GP did without going into debt, but that will require bigger structural changes.
Measuring usage in general I think will be tricky..
Example: Bram Moolenaar (ViM) asks users to :help uganda, not to pay him. And I speculate that he wouldn't be offended if you donated to help someone else in need instead.
We as the OpenSource community need to start moving towards a more sustainable approach to OpenSource development. First step: Stop calling it free.
I make christmas hampers for the poor, I call them free hampers. Of course they are not free, but to the end user they are.
And no, people are not going to start using the word libre, it is not an English word and it is not used widely enough outside of computing and it will probably not catch on. /endrant
We as the free software community need to show people who call it open source the door and have them escort out by security so they stop ruining the whole affair.
...what was that money thing you where rambling about?
Do civil engineers have the opportunity to create the bridge version of Linux as their masters project? I think the physical limitations of bridge building means that there isn't a great analog to Linux in the civil engineering field.
The only field I can come up with that could behave similar to software is art. Why don't artist release art "open source" for others to use and transform? I could certainly see compensation as a factor: artists are famously underpaid so they are not willing to work for free. Or maybe it's a cultural thing, where lots of people using your open source library translates into respect for the author, but lots of people ripping off an artist could have the opposite effect?
That may be true to build final product, but to even become interested or explore the field of engineering (and other professions) has a lot of roadblocks and expenses that software developers don't have to deal with. Imagine having to pay $2000 for a Windows Server + MS SQL license to develop your small website. Whoops! I forgot the $500 license for Visual Studio just to get your code to compile.
The availability and power of free, open-source software greatly lowers the barrier to entry for anyone and everyone: hobbyists to make and release a project, startups to create their product on low/no budget, curious people interested in trying programming.
The fact that this free and open software rivals and often beats its paid equivalents is absolutely amazing. It's why despite the problems in our industry, I am so proud to be a part of it.
In fact—as you point out—the average company in most industries relies heavily on state provided infrastructure, that they get to use for free (or a really modest price).
If the Rust guys would understand this and put an async runtime, http server and client, crypto and some more essentials in the standard lib the adoption would be much higher. Than we would have a super robust language and a super robust ecosystem. I don't think the borrow checker is the issue with Rust slow adoption in the mainstream backend business.
The process is definitely frustratingly slow, but going faster has burned the Rust devs already in the past (just look at how much stuff in std::error::Error is deprecated).
Tokio is fantastic for web services, but is inappropriate for my microcontroller.
EDIT>> "Can you do that?" meaning, "Can you sell an open source project?" Not "Can I sell to a scammer?", for people accusing me of planning to commit a crime.
As for licensing: totally agreed. I would like to see more projects with protective licensing.
Too many projects are basically donations to Amazon.
This market might be one of the reasons why extension API powers have been scaled down.
Probably not. I can only think of two reasons someone would buy it: to do something not allowed by the license or to distribute malware.
If you have say copyright on an AGPL-licensed piece of software that a proprietary vendor wants to use/sell, maybe you have a shot at the former. I don't have any experience with this but it's an above board thing to do anyway.
Otherwise, I can't think of a legitimate reason for someone to be willing to buy it for a significant sum. They can always fork for free if they just want to make improvements and distribute them. If it's important to them to take over the existing name/url/credentials it's probably to put in something that users don't want...
If you don't want it and stop maintaining it, motivated people could presumably fork it and maintain it themselves rather than pay for it... but I guess you're also buying the community
The number of dependencies I've had fail in that manner is currently 0. Also, all my open source dependencies have their source-code reviewed regularly by distro maintiners (like debian's), and the last version's source code is available in source control, or at worst in a distro's src tarball mirrors.
However, on the flip side, the number of dependencies I've had break because they're maintained by a large company, and that company pivoted in some way, or reworked everything, or in some cases had the code calling into their proprietary backend via some api and broke their api... that number is in the dozens to hundreds.
Using a dependency maintained by an entire company vs by a single contributor doing it in their spare time... Well, the company-maintained ones have a much worse track record for me so far.
This made me want laugh, then cry. Both because it's often true.
(this actually happened to an ios app I had years ago - gas cubby - one day all the local data about your car - vin number, insurance info, mileage and more became cloud based overnight)
Also, each dependency you add to project is a security risk in itself. I really can’t believe how webdesigners casually add dependencies without a second thought especially with dependency management system like npm.
People give C++ lots of hate but thanks to not having an easy way of adding complex dependencies, it has less dependency driven bugs/security issues.
Oh, come on. The number of security issues to things like libssl alone is enormous.
Don't try and tell me C++ projects in general have more dependencies than high level languages.
https://nightly.ardour.org/i/A_Linux_i386/info.txt
That's the dependency stack for a 350k LOC cross-platform C++ project.
I'll leave it to your judgement if this is more or less than "high level languages".
Regarding security issues in C++ in general: ~51% of security issues are memory related, so you get at least half less security issues with a garbage collected language.
This doesn't stop people adding dependencies: most C++ software still has a complex dependency tree, just most of the labour of maintaining it falls on distro maintainers (and the poor souls which need to make windows builds) [0]
And I don't think C++ has any shortage of dependency driven bugs or security issues. Certainly if you asked me to pick whether I'd like to build something from npm or the average C++ project I'd pick npm every time: I expect to need to fix on average about 2 issues every time I attempt to run a non-trivial C++ project's build, each issue taking an hour or so to locate and fix or hack around. The nature of C++ dependency management means there's more likely to be vendored dependencies which are out of date and have security issues, perhaps introduced by the dependee project as patches (and ripping out this kind of vendoring and dealing with any required patches creates even more headaches for distro maintainers, though they are also quite capable of introducing their own bugs through their own patches).
In terms of ImageMagick, I used to use it but now maintain my own set of programs for dealing with picture files (almost entirely written in C, although the ones for converting to/from fax format are in PostScript; I may later write a version in C as well, so that you can use it if you do not have a PostScript interpreter). Like most software I write, I try to not use many dependencies if I can avoid it. (It doesn't use libpng either; it is using LodePNG, which is better in many ways, in my opinion.) (These set of programs I wrote also have many effects that I have not seen in other programs, such as automatic rearranging in a horizontal or vertical strip, removing duplicates from a vertical strip, and making the tensor product of two pictures. There are also file formats ImageMagick does not support.)
"Tech giants, chastened by Heartbleed, finally agree to fund OpenSSL" (2014):
https://arstechnica.com/information-technology/2014/04/tech-...
https://daniel.haxx.se/blog/2016/11/14/i-have-toyota-corola/
https://joshuakugler.com/an-interview-with-simon-kelley-the-...
Every time you think you need a custom file format for a given piece of software... you most likely don't. Just use SQLite. You can use the standard OS file Open/Save dialog box and users will never know the difference.
Yes, that means you now have to write SQL statements to manipulate your data. But that also means that you can get lots of complicated data structures on disk and can manipulate them easily, even outside your own software - just fire up the sqlite CLI and point to your file. There are GUIs as well.
Things like UNDO/REDO can also be had almost trivially (see https://www.sqlite.org/undoredo.html)
Sometimes this also means you don't need an external RDBMS even for web apps. I've seen so many apps which co-locate a small database in the same box that might as well have been a single sqlite file. I'm actually maintaining one right now that, although relatively important, will only ever be a single box. But sqlite wasn't 'enterprisey' enough, had to use PG. For a couple of tables.
https://www.sqlite.org/whentouse.html
Subscribe for my next rant on another underused piece of tech... Lua :)
For on-disk document storage I think Apple mostly uses a mix of plain folders with magic extensions (“packages”) and ZIP files nowadays, although there are definitely a lot of exceptions. SQLite isn’t that great for binary blob storage (relatively speaking) so a folder structure is still more useful there, IMO.
Consortia have been tried many times, but few as successful as the SQLite Consortium.
What a great model. Open source for the main thing, proprietary test suite.
This model does require building something everyone needs, years of patient care and feeding to make it wildly popular, and a fantastic test suite. So it's not exactly easy to pull off. But it is brilliant, if you can do it.
[0] Even just contributing is very difficult. Try to contribute anything other than a trivial bug fix, and you'll find that a) unless it's utterly trivial, the devs are not interested, b) they are dead serious about checking that the contributor has their employer's permission to place that contribution in the public domain.
Not only can you get a SQLite database (which can be used as a application file format, suitable for many uses), but there are functions such as sqlite3_mprintf() and sqlite3_str_new() and stuff like that; and furthermore, it can allow the user to query it too with SQL commands, even within the program, and to allow user customizability by the use of SQL codes, too, and it allows the user to deal with the file even without your software in some cases (not needing to write a separate decoder). Sometimes virtual tables are useful for doing some things. And, this is in addition to being able to use SQLite as a file format.
Core-js most recent commit was 4 days ago, and the repo owner stopped both committing and being the sole committer about 7 months ago with no extended interruption in activity (and there've been several patch releases since), so it looks like whatever happened was dealt with quickly.
https://www.infoworld.com/article/3144546/time-is-running-ou...
GnuPG is also maintained mostly by the same original guy, who was going broke a few years back so there was a chance he couldn't work on it anymore (https://www.propublica.org/article/the-worlds-email-encrypti...). Fortunately the community came to help and now the future of GPG is a little bit more secure
It's not really the same but ssh is literally everywhere. Absolutely none of our sysadmin work would be possible without it, and yet very few of us actually give back to the OpenBSD foundation. If we all gave a penny every time we installed it every year, the foundation would reach its annual goal (https://www.openbsdfoundation.org/campaign2019.html) but we obviously don't, so they still need to ask for money
Also not specifically in this way but pretty much anything that Daniel J. Bernstein does is so good that it becomes a crypto standard. It's kinda ok because the software he does are "finished" in the sense that no new features are neither needed nor wanted, and his work is more about the primitives than the actual libraries. Still, there's a lot of the upcoming standards that all depend on what this single guy did.
At some point, my library caused a bug or two in Docker, impacting how config was loaded.
bash (Chet Ramey)
https://www.sciencealert.com/how-a-programmer-almost-broke-t...
I don't know much about npm management, but I'm assuming it actually had nothing to do with caching? More like most companies have pipelines and breaking dependency changes won't reach prod.
It takes a strong open source advocacy effort to a reach out and a sophisticated organization to be reached, for such an organization to recognize when it's in their interest to participate in open source maintenance.
One such metric I found to be missing for npm dependencies was the install size, which is why I created https://packagephobia.com
We need more tools to help make this decision because its easy to add a new dependency but its often hard to remove a dependency.
You looked at the size and determined that it might not be worth bringing on a small dependency when you can implement it yourself (or rather use String.prototype.padStart).
It’s not just size alone but size of transitive dependencies.
Both examples you gave don’t have any dependencies (the publish size matches the install size).
Take a look at some popular dependencies like request or jest and notice the number of transitive dependencies.