Hacker News Reader

Comment by sergiotapia | Hacker News Reader

Yikes, what is it about node/npm/javascript that makes it feel like a house of cards?

Yikes, what is it about node/npm/javascript that makes it feel like a house of cards?

I think the (short) answer is "node, npm, and javascript".

The longer answer has something to do with the automatic installation of dependencies, and the common use of shell scripts downloaded directly off the internet and executed using the developer's or sysadmin's user account.

I used to use CPAN all the time. CPAN would check dependencies for you, but if you didn't have them already you'd get a warning and you'd have to install them yourself. It forced you to be aware of what you're installing, and it applied some pressure on CPAN authors to not go too crazy with dependencies (since they were just as annoyed by the installation process as everyone else.)

These days I use NuGet a lot. It does the dependency installation for you, but it asks for permission first. The dialogs could be better about letting you learn about the dependencies before saying they're ok. (In general, NuGet's dialogs could be a lot better about package details.)

> I used to use CPAN all the time...

CPAN has a setting that force-feeds you dependencies without asking, but I don't think it's on by default. Also, CPAN runs tests by default, which usually takes forever, so users get immediate feedback when packages go dependency-crazy. The modern Perl ecosystem is often stupidly dependency-heavy, but nothing like Node.

I had very limited contact with CPAN some years ago but I imagine it was slightly more sane in terms of granularity of dependencies.

Whereas a lot of npm modules are relatively small - some tiny - and have their own dependencies. So a simple "npm install blah" command can result in dozens of packages being installed. Dealing with that manually would, in fairness, be a giant chore.

Now of course there's a discussion to be had about whether thousands of weeny little modules is a good idea or not but, to be honest, that's a religious debate I'd rather steer clear of.

Whether it's a good idea or not, that's what JS' lack of stdlib produces.

> CPAN... forced you to be aware of what you're installing

I think CPAN is pretty sweet for variety/wide reach of packages available, but this is flat-out wrong.

CPAN is not a package manager; it is a file sprayer/script runner with a goal of dependency installation. That's perfectly sufficient for a lot of use cases, but to me "package manager" means "program that manages packages of software on my system", not the equivalent of "curl cpan.org/whatever | sh".

CPAN packages can (and do by very common convention) spray files all over the place on the target system. Then, those files are usually not tracked in a central place, so packages can't be uninstalled, packages that clobber other packages' files can't be detected, and "where did this file come from?"-type questions cannot be answered.

Whether CPAN or NPM "force you to be aware of what you're installing" seems like the least significant difference between the tools. When NPM tells you "I installed package 'foo'", it almost always means that the only changes it made to your system were in the "node_modules/foo" folder, global or not. When CPAN tells you "I installed package 'foo'" it means "I ran an install script that might have done anything that someone named 'foo'; hope that script gave you some verbose output and told you everything it was doing! Good luck removing/undoing its changes if you decide you don't want that package!"

There are ways around all of those issues with CPAN, and plenty of tools in Perl distribution utilities to address them, but they are far from universally taken advantage of. CPAN is extremely unlike, and often inferior to, NPM. Imagine if NPM packages did all of their installation logic inside a post-install hook; that's more like a CPAN distribution.

Also. In CPAN there was a culture of having comprehensive unit tests. If something broke, you would likely see it at installation.

I have recently taken over an Angular project (with a C# backend, thankfully) at my job. It took two hours to get it to even compile correctly because some dependencies were apparently outdated in package.json and it just ran on the other dev's machine by accident. I don't understand why I need over 100 dependencies for a simple Angular Single Page App that pulls JSON from the backend and pushes JSON back. Meanwhile, the C# backend (a huge, complicated behemoth of software) ran on the first click.

Thankfully we now have package-lock.json

Does that backend use nuget for dependencies?

No. It needs nothing but the .NET Framework (not .NET Core) because .NET is already providing everything we need.

Three developers on my team spent the last 4 years pushing for angular. Four years ago, I was 50/50 on it vs react, so whatever, but if my team's really for it, let's do it.

Fast forward to angular 2, and we're down to two developers who are still for it.

Fast forward to today, I'm down to one angular dev who's still for it, and two of the original three have left for react jobs. Meanwhile, I'm left with a bunch of angular 1 code that needs to be upgraded to angular 2, and a few testing-out-angular-2 projects that are dependency hell.

The only reason I ultimately embraced angular 1 to begin with (above reasons aside), was because it was so opinionated about everything, I could throw it at my weaker developers and say: "just learn the angular way to do it", and there was very little left they could meaningfully screw up. Angular proponents on the team would see it as a point of expertise to teach the "angular way" to more junior devs, and everyone left the day feeling good.

When it comes to Javascript 95% difficulty with writing good maintainable code is ensuring that your team is all writing to a very exact, and consistent quality and style, since there are so many different ways you can write js, and so many potential pitfalls. And if the team all wants to embrace Google's Angular standard, that works for me. Its far easier to be able to point to an ecosystem with an explicit, opinionated way of writing code, than it is to continuously train people on how to write maintainable code otherwise.

But with angular 2, if you haven't been drinking to cool-aid for a while now, it requires so much knowledge just to get running, I can't even have junior devs work on it, without a senior dev who's also an angular fanboy there to make sure everything is set up to begin with. Its absurd. And I'm supposed to sell to the business that we need to migrate all my Angular 1 code to this monstrosity? And then spend time again every 6 months making the necessary upgrades to stay up to date? Get real.

Curious, what would you rather do instead? Is there an opinionated React framework you could use?

>Curious, what would you rather do instead?

Moving away from SPAs seems like a dream at this point.

I don't understand. We've started a new Angular 2+ project and our junior developers managed to roll into quite easily. Our designers (who know jack about Javascript) got excited when they discovered that our project uses .scss and the results have been spectacular.

Seriously, I REALLY REALLY don't get this hate for Angular 2+

Just wait until Angular 2 hasn't been cool for a while and you can't find any JS developers who are interested in maintaining your software rather than rewriting it in xyz_latest_fad_framework.

The basic explanation is that the dependencies for the angular app are much smaller, but I’m not sure which bit is confusing you. You don’t understand why an incorrectly written program required work to run when a bigger but correctly written program was easy?

> incorrectly written program

In principle programs shouldn't stop working just because they are old.

Yes, no language completely realize this. But there's a world of difference between C's "it was written only 40 years ago, why did compilers break it?" to Python's "yes, you are expected to review your code every 3 or 4 years", and there is another world of difference to the faster Javascript frameworks that practice "your code is 6 weeks too old, your looser!"

No JS framework does that because they version things. Run the same versions and it works.

If you’re not pinning versions correctly that’s hardly JS’s fault.

This is a cultural thing, where developers will decide when to invest in developing their library against the old version and when for the new version. For stable languages like C, or distro supported packages it’s years - just check out Debian or Red Hat for an ecosystem that values stability.

Sure, and at the scale & purpose of Debian or Red Hat that’s extremely important.

But that’s nothing to do with bad versioning practices and everything to do with product priorities.

Also C versions are have the same issue. Try to build a C11 project with a C89 compiler. Hell I’ve had C89 code not work in clang...

Versions affect everything.

Mark, is that you?

Kidding - but we had exactly the same problem, except with a React app rather than an Angular one just before Christmas.

No joke on with this statement though: every time we have a time-consuming build issue to deal with it comes down to some npm dependency problem. Honestly, if there were a way we could realistically ditch npm (NO! YARN IS NOT ANY BETTER - to preempt that suggestion - it's simply an npm-a-like) I'd happily do so but sadly there isn't.

You are describing bad development practices.

Not sure why you’re stuck on the number of deps either - as long as they’re small who cares?

How about the idea that Node has been a hack from day one?

What hasn't?

Good point

Quite a few. E.g. ssh definitely was not, Rust was not, TeX was not. But these were mostly second-thought projects of the "let's now finally do everything right" kind.

They certainly weren't the cores of other projects, ripped out and made into a standalone thing.

Reminds me of this genius talk: https://www.destroyallsoftware.com/talks/the-birth-and-death...

Node was a very interesting thing back when it started. It was a hack, but a nice kind of hack. You could write some efficient servers with it. But then the community that formed around it, with it the project went berserk.

A bit like PHP in that way?

Well, kind of. Node was not a general purpose tool as conceived initially. You would write some I/O bound servers in it. And PHP too is not a general purpose tool, it is for writing interactive web pages (in its pre-Web2.0 sense) easily. Though Node.js was way more intellectually designed. I don't know much about PHP, but there's lots of literature (see https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/).

I really wish that people would stop referencing that "Fractal of Bad Design" article. It's outdated and mostly irrelevant now (April 2012, PHP was at 5.4 then, it's at 7.2 now). It's not that I want to defend PHP, I just think people should judge PHP for what it is now instead of what it was several major changes ago.

Besides, the author seems to misunderstand a great many things about PHP and languages in general. Here's a short rebuttal (also from April 2012): https://blog.ircmaxell.com/2012/04/php-sucks-but-i-like-it.h... that explains some of the misunderstandings.

Hmm are you sure? I've read the fractal of bad design many times.

Some issues might be "fixed" but could they fix the actual *fractal of bad design"?

Isn't it still a mix of c-style, java-style, inconcistent, left associative, horribly broken language it always was?

I always thought the bugs were anecdotal backing of the main point: php is badly designed, non programming language for non programmers, who suffer stockholm syndrome from all php abuse...

> Hmm are you sure? I've read the fractal of bad design many times.

Yeah, I'm sure. And so have I. Maybe you should stop reading it to reinforce your prejudice and instead take a look at PHP 7.2?

> non programming language for non programmers, who suffer stockholm syndrome from all php abuse...

Hating PHP is almost like a bad meme. Obviously it's doing something right otherwise it probably wouldn't be as popular as it is. (Same can be said for Javascript, I guess.)

Your personal feelings about the language are pretty much irrelevant. The Fractal of Bad Design article, however, is actually spreading misinformation yet people with an axe to grind keep referencing it because it fits their agenda, hence why I react whenever I see it referenced.

Here are just a couple of examples of where it's flat out wrong and/or completely outdated. There are plenty more.

He's left in things that were fixed long before he published the article — e.g. the new array syntax — but that doesn't stop him from saying stuff like "Despite that this is the language’s only data structure, there is no shortcut syntax for it; array(...) is shortcut syntax. (PHP 5.4 is bringing “literals”, [...].)" Keep in mind, 5.4 was already out when he wrote it...

Not to mention the whole section on "missing features" where he basically enumerates things that most certainly doesn't belong in a language's core but in separate libraries or part of a framework, and — surprise! — those are all available in both libraries, frameworks, extensions, etc.

"There is no threading support whatsoever." pthreads have been stable since 2013: http://pecl.php.net/package/pthreads

I wonder when people will stop quoting this 4+ year old article. Most of what are actually issues are long fixed. https://php.vrana.cz/php-a-fractal-of-not-so-bad-design.php

When it's invalid, maybe. The article you link says in the first three paragraphs:

---8<---

Whether you like PHP or not, go and read the article PHP: a fractal of bad design. It's well written by someone who really knows the language which is not true for most other articles about this topic. And there are numerous facts why PHP is badly designed on many levels. There is almost no FUD so it is also a great source for someone who wants to learn PHP really well (which is kind of sad).

I am surprised that I am able to live with PHP and even like it. Maybe I am badly designed too so that I am compatible with PHP. I was able to circumvent or mitigate most problems so the language doesn't bother me.

Anyway, there are several topics which are inaccurate or I don't agree with them. Here they are with no context so they probably wouldn't make much sense without reading the original article:

The npm repository is the largest package repository in the world. A lot of the major incidents they've could have happened to other ecosystems (e.g. PyPi allows a user to delete packages that other packages depend on), but they've either not happened or haven't had as large an impact. When npm breaks, everyone notices, because everyone either uses npm or knows someone who does.

Largely because Javacsript is so broken by default that it is almost required to depend on a whole slew of dependencies for functionality other languages contain in their built-in standard libraries. And furthermore, NPM dependencies are broken down into stupidly small units, versioned rapidly, and enforces very little consistency among transitive dependencies.

Other languages and package management systems don't encourage this kind of insanity.

I’m impressed. Not one thing you just said is accurate.

I'm impressed, because everything they said is accurate.

I know HN is basically an industry wide joke for JS discussion but no, none of that is accurate.

Most of it is plain wrong, though some of it is misapplied frustration to the wrong target.

But again, I guess I shouldn’t expect more from HN.

What exactly is wrong?

This in particular is a huge trust failure - working with mutable/replaceable libraries is like working with mutable/replaceable APIs.

Well they aren't mutable/replaceable, at least not since after the left-pad incident where npm announced new rules to prevent package unpublishing. It seems this was a operational bug at npm inc.

I wonder how much damage need to be done with JS/Node until seriously is put to rest the madness. Is absolutely necessary to break backward compatibility and rebuild JS from the start. With WebAssembly this is doable (without excuses!) and we already have a nice tag to declare what script language to use.

This is not possible, you ask?

In fact, JS/CSS is the most viable of all the stacks to move forward. Let's use the "advantage" that any JS library/ecosystem die fast and put enough hipster propaganda declaring the ultimate solution.

Is too hard? JS is so bad that fix it is too easy. You only need more than the week it originally take to build it.

As a counterpoint, couldn't any sufficiently complex structure be called a hack and a house of cards, when you really dig down into how it's put together? Mm, maybe not any - as some complex systems are well-tested with solid architecture - but just some, or most..

A "house of cards" implies that you don't have to dig to topple it. If you have to really dig down into how it's put together in order to start pulling it apart it isn't really a house of cards.

I don't use npm or node for anything serious, and i don't really have any knowledge of how NPM works, but this isn't the first time i've read this story of a whole bunch of packages disappearing and everybody's builds breaking. If everything is a house of cards, then why don't i hear the same stories about PyPI or gems or crates?

It reminds me of this article, Everything is Broken. Perhaps a house of cards becomes solid architecture through the test of time..?

https://medium.com/message/everything-is-broken-81e5f33a24e1

The issues with the NPM Registry are not technical but management issues and decisions not properly thought through.

As of this writing, aren't we still waiting to see what the problem was inside NPM that caused this user's packages to disappear?

It might well have been technical. It might well have been managerial. It very likely involved elements of both. But don't you think it's best to save the Monday morning quarterbacking for Monday morning, when all the facts are in?

Because when you’re on top, everyone loves seeing you fail.

A quick CVE search pulled up 18 vulnerabilities in RubyGems, including remote code execution.

> If everything is a house of cards, then why don't i hear the same stories about PyPI or gems or crates?

npm is roughly twice as big as PyPI, RubyGems and crates.io together.

Disappearing packages due to a deleted user is not an issue of scale.

I can't speak for PyPI, but I know Ruby gems has had vulnerabilities in the past. A quick DuckDuckGo will probably suffice to demonstrate that. I'm not saying NPM is a great system, but it does seem to me that most systems have flaws, and any system that is as heavily used as NPM is likely to have them surface faster than other systems.

"Have you ever noticed that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?" --George Carlin

I think the software version of this is: any system with more structure than your program is an over-engineered monstrosity, and any system with less structure than your program is a flakey hack.

Almost all problems I have with JS projects have to do with npm. It got better with lockfiles, but it seems they're inventing new problems...

Left-bad, I mean, the left-pad fiasco should have been the wake up call.

How do you not feel embarrassed using such low quality insults..?

It was just a bad joke, but in all seriousness, that was a big wake up call for a lot of people about the tangled web of npm dependencies.

Disagree, it was mainly a wake up call that npm shouldn't allow package deletion, a policy they changed as a result.

Every other 'these kids and their dependencies' opinion over the left-pad incident was highly subjective.

Because it is... It's mollochian complexity heaped on top of layers of excrement and ducktape, and we have collectively entered a state of mass Stockholm Syndrome about the situation.

I really would love to ditch web dev and all its myriad tendrils, and go back to native desktop software.

Somehow i imagine a native C-Desktopdev and a Webdeveloper meeting in No-Mans Land each party escaping from its own nightmare with that line on the lips, starting with a "Dont run into this direction-"

Eh, i like desktop development and i make desktop apps for 20+ years. Before i got Windows 95 i was even trying to make my own DE for DOS in Turbo Pascal and before that in GW-BASIC :-P. I love the desktop.

Web stuff on the other hand can die in a fiery death, as far as i am concerned together with mobile stuff they are the source of everything wrong with the desktop today :-P.

At my job we do native C and C++, some Java, some C#, scripting in Shell, Python, and Perl. When the left-pad incident happened someone said something to the room about it, we all looked it up, and spent a good 15 minutes mind-boggled, laughing and being grateful we weren't web devs. "Wait, you're telling me these people need NPM and GitHub to deploy? Seriously?"

>these people need [their package manager] and [their source code management tool] to deploy? Seriously?

Not really sure I understand what you're implying there

A public package manager and a public source code management tool, both of which are outside of your control. You should be able to deploy from a local [verified and audited] cache of your dependencies.

That's a good goal to strive for, but isn't necessary or practical for everyone. Maintaining local/hosted artifact caches, verifying them, and auditing them is a big hassle, and unless you make something (e.g. fintech, healthtech) that might need such an audit or emergency release, might not be worth the trouble.

Itty bitty company making a social website on a shoestring budget/runway with very few developers? Might just be worth postponing a release a day or two if NPM or GitHub are having issues.

virtualenv makes it trivial. It's not like it's strictly enterprise-grade tech.

How does vrtualenv make maintaining, auditing, and using a local mirror of dependencies trivial? Seems to me I can download a poisoned package into a venv cache just as easily as I can download it with wget, and unless I take the time to check, I’m none the wiser either way.

I was referring specifically to not being able to deploy due to a package manager being down. Of course there are still issues that can crop up with using virtualenv.

I'm not the poster you're replying to, but I think I understand it.

npm is not just their package management tool... the way most people use it, it depends on someone else's package registry/repository to deploy to your own servers.

And github is someone else's source code management tool/server.

As a matter of policy, if I can't have something on my own server (or one my org controls) I don't get to rely on it to deploy/run my application.

So I think I get the parent's comment... it's a really foreign situation, to me, to depend on the availability of stuff like this on servers I (or my org) don't control in order to deploy my application.

I'm sure the people who depend on these things look at me and say "Wait. You have to set up your own package repository and source control before you can deploy instead of using all this nice stuff that's available in the cloud? Seriously?"

Yeah. I've been on both sides of this coin. If I'm deploying cloud software (which I am, these days), then I have no problem relying on cloud software to make that deployment smoother. But if I ever go back to writing native applications, I sure as hell won't be reliant on the internet in order to manage intranet deployments. These are two different paradigms, and what works well in one doesn't make any sense in the other.

I haven’t dug too much, but I believe at my work, we run a server that hosts all our jars, and is the source of truth for all our builds. Nothing that’s been checked in goes straight to the Internet (you can add new dependencies to uncommitted code). And we’re only ~30 devs.

"Wait, you're telling me these people rely on their ISP and the telco infrastructure being operational to deploy? Seriously?"

"Wait, you're telling me these people need an internet-available Ubuntu mirror to install their development environments?"

"Wait, you're telling me these people need their users to have specific, updated browsers in order to run the deployed software?"

"Wait, you're telling me these people need their users to have a patched, up-to-date operating system in order to run your desktop app?"

"Wait, you're telling me these people just assume users won't switch off their computers before saving changes?"

"Wait, you're telling me these people depend on the power grid being available to deploy?"

"Wait, you're telling me these people assume their users have fast, low-latency internet connections to play their real-time multiplayer game?"

You get the idea.

And? Most of these are things you absolutely should be thinking about

"You should be thinking about" and "You need this? Seriously?" are very different statements. Of course you should be aware of dependencies.

And you should also be aware of what it takes to rebuild your stack, and have something in place if that disappears. If you think it's OK to rely on external tools like that to build your system, you deserve all the fallout you get when it fails.

Reply on news.ycombinator.com