Hacker News Reader

Part of it is that JS's standard library is terrible (eg. no built-in union/intersection when working with sets, bizarre default sorting behavior see (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...), groupBy (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...) is new and not well-supported, etc). You almost have to reach for some utility library or build your own ad-hoc one just to use the language.

I don't see that as the case. Modern JS can do a lot out of the box but the culture looks down on "vanilla" JavaScript. I've seen way too many libraries that are nothing but a thin wrapper around native functionality. When your first (and only) technique is to look for a library, this is where you end up.

I agree and am very pro vanilla JS FWIW. I just find myself reaching for something like lodash's `intersection`/`difference` functions when working with sets, `sortBy` to get more normal (and not in-place) sorting behavior, and `groupBy` to do group by.

> Bizarre default sorting behaviour

You are meant to pass an argument to [].sort; If you don't, it falls back to the most generic thing that makes sense -- which is turning everything into a string and sorting lexicographically.

Arrays in JS can have any type in them, such as [number, string, function]. That's obviously very unlikely, but the only thing in common between all types in JS is that they can all be explicitly turned into strings.

And yes, I agree that throwing an error here for no argument would be better here (a linter WILL enforce this), but this is hardly a critical shortcoming of JS's standard library, and you DEFINITELY do not need a utility library just to use the language (especially for your examples).

I hear what you're saying, but it still seems pretty bonkers to me that if you try to sort an array of numbers it will cast them to strings and sort alphabetically (!):

  > [1, 2, 10, 3].sort()
  [ 1, 10, 2, 3 ]

> And yes, I agree that throwing an error here for no argument would be better here (a linter WILL enforce this), but this is hardly a critical shortcoming of JS's standard library

I suppose "critical" is debatable but this seems very fundamental and very unexpected to me.

> you DEFINITELY do not need a utility library just to use the language (especially for your examples)

I hadn't actually considered using a linter to avoid these types of standard library footguns... that's actually a pretty great idea!

Since arrays can contain a mix of anything, I don't think it's that bonkers that the default impl chose to canonicalize values into consistent comparables with String(). It's just not useful for numbers.

But for example, they probably decided that it was more useful defaulting to having a sort order for things that otherwise aren't comparable:

    a = [undefined, null, undefined, null]
    a.sort()
    // [ null, null, undefined, undefined ]

Since null < undefined and undefined < null are both false, then a simple `a < b` comparator wouldn't sort them at all. Same for objects.

If you have an array that's a huge mix of random values, from null to undefined to [] to {} and you sort() it, all of those values will now be grouped together by type.

This doesn't seem like a bonkers trade-off.

There would be a significant performance impact if `.sort()` argumentless had to traverse the list and checked all inputs were numbers. It's better to pass a sort function -- that is how the API is meant to be used.

I agree that without an argument it should just be a fatal error, but if it were to have any sort of functionality, it should convert all to strings.

This is consistent among JS's apis, and pretty much the origin of all the 'js wut' moments on the internet. Everything in JS can be converted into a string. If you do something stupid with disparate types, it will likely turn operands into strings and compare them that way.

> You almost have to reach for some utility library or build your own ad-hoc one just to use the language.

Sure. But are hundreds of dependencies really required for this?

In Java you would use tools like Guava or Spring for general quality of life improvements, and there would be a few deps for them (under a dozen iirc).

The solution is for the “top tier” libraries and frameworks in the JS world to be designed with minimal dependencies. And where they do have a need for a dependency, they undertake serious consideration of the best option that minimises dependency hell.

Yeah, there's definitely a separate cultural/inexperience problem at play as well.

And those "top tier" libraries do exist for some stuff- especially if we're talking about the anemic standard library. The famous lodash library doesn't have any (non-lodash-umbrella) dependencies AFAIK.

What I think is happening: the JS ecosystem has been flooded by developers with minimal experience and/or education.

They learn engineering principles like "don't repeat yourself" and take that to mean installing an entire dependency to implement left pad is a good idea.

You are probably on to something with JavaScript being bug-prone as a factor in that.

JS as an ecosystem has a really big problem with developers not knowing the value of simplicity.

You can't have simplicity when every other week someone is shoving a new framework down your throat. And in fact you are made look like a looser if you dare do things in vanilla JS

This is a big problem in the tech industry, in general. Some weeks back I read a comment that described some behavior as "high intelligence, low wisdom". I believe that fits pretty well here.

People design new frameworks (presumably) because they see an array of problems with existing frameworks. In designing their new framework, they try to address the shortcomings that the existing frameworks have.

What they don't realize is that the problems in existing frameworks were _known tradeoffs_. Now, instead of the One True Perfect Framework, we have yet another framework with its own set of problems.

People think that every problem is solvable simultaneously, but that's simply not true. You can make tradeoffs. And this isn't just true in engineering, it's true in life generally.

Some tradeoffs make sense nearly always. Others only make sense in certain contexts.

An example here is the tradeoff between simplicity and high availability. It doesn't matter what you do -- the simplest high availability configuration for an app will ALWAYS be more complex than the simplest non-HA configuration. You're making a trade here. It's a trade that is absolutely sensible, but it's a trade nonetheless.

The lesson to be learned here is: stop thinking you can solve every problem at once. You can't.

One potentially contributing factor is that npm makes it very easy to avoid conflicts with duplicate dependencies, i.e. if one dependency has a transitive version on some other package, and another dependency also has a transitive dependency on that package, but on an incompatible version, that's not a problem in npm: it'll just store both on your disk. That might remove some pressure to minimise the number of dependencies on library authors.

And of course, there are just far more people working on it, I believe.

This reduces package-installing friction, but this is a good thing anyway -- there's no reason why you shouldn't be able to use foobar@1.0 and foobar@2.0 in the same project. For all intents and purposes they're different packages, viz. the version is just as important as the package name.

Yeah to be clear, I wasn't trying to say it was a bad feature, just that this might be one of the negative sides part of its trade-off.

This may not be the whole story, but one of the reasons is that JavaScript does not have a standard library. Corollaries are: several different module systems, application frameworks and bundlers exist.

Node does have a standard library.

I've seen this same weird line parroted by so many people in the comments here: it's very odd.

If you're talking about server & browser having different APIs that's equivalent to any other language having different platform apis.

I think that's a big part. The standard library isn't great, and progress is tied to browsers. Additionally, tiny packages became the norm early on. Is even, is odd, is negative zero, left pad, etc.

I don't know all the reasons for that, but I think part of it is that developers create them in order to put it on their resume. "My package is downloaded 500k times a week"

Simply because blast radius for Java is limited to a set of very high quality libraries -- in terms of code not functionality. These libraries come from Apache Foundation, Eclipse Foundation, Google, Facebook, Spring, etc. Literally every single Java application depends on something from Apache [ok I understand stuff like Log4Shell can still happen].

The same is not true for JS. The most mature libraries depend on absurdly vague libraries that no one has ever reviewed.

I was going to ask the obvious question of why the Java ecosystem ended up differently than the JavaScript ecosystem, but I think I know the answer.

It's a giant pain in the ass to publish a Java library. That's already weeding out a ton of low-effort projects. By itself, I wouldn't exactly call that a good thing, but it seems to have a silver lining...

I’ve mostly worked with JS in the browser and my understanding of node lacks nuance, but it seems like all of this would be mitigated drastically by building out the standard library. For comparison, here’s pythons list of standard modules:

https://docs.python.org/3/library/

They don’t make third party libraries obsolete— for example, I tried using the built-in IMAP library the other day and it’s definitely too low-level to make sense for most quick projects that check email, so I used a third party Library— but all of those modules are vetted, stable, and require no external dependencies.

I believe the node maintainers have staunchly opposed such measures. I don’t know what their reasoning is so I don’t have an opinion on whether or not it’s worth it.

I'm also pretty new to JS but totally agree. Half my Python projects have zero dependencies (except for development tools which don't get packaged with the app) because everything I want is already in the standard library.

It feels like a quarter of the time I spend working on projects that use npm is spent debugging my toolchain because of excessive complexity or weird problems in random dependencies. Doesn't seem like it should be too much to ask that I can spend most of my development time working on actual code.

The process of uploading packages onto Maven Central is... not modern. npm (arguably?) makes this a lot easier than any other language, therefore npm developers do more of it.

The interaction between a language and its culture is really complicated, partially because it's an ongoing iterative process, and thus chaotic, in both the English and mathematical senses of the term.

Not having a standard library made people accept needing libraries for even very small things that in most other languages developers would make at least some effort for using the standard library before reaching for something else.

I think another aspect is that the initial leadership of a language community sets the tone for a long time, but JS in a lot of ways didn't have that, not through any fault of any particular person but simply because Node grew so explosively at the beginning that the usage growth dominated the available leadership growth, and so there was a lot of very wild, woolly growth that got written into the earliest culture. This creates something a lot like a "seed crystal" that has outsized impacts on future paths for a long time.

Finally, I do think it is definitely an issue that JS is somewhere where you get a lot of people who are not "programmers" per se and they are making big decisions about code bases and libraries. They're young in the art. And while there's nothing intrinsically bad about that, people have to start somewhere, there's a lot of ways in which it's good that JS is relatively easy to get into, etc., it is also absolutely true that at scale, as the language and the libraries iterate on each other and seek out their stable points, that's going to affect the landscape. This is an "is", not an "ought". It is what it is. JS has also continued explosive growth, so even as someone who got into JS and programming for the first time in 2017 is now a 5-year "senior" (a crack about our industry terminology, not the dev here) developer who has learned and might be inclined to do things differently than they did 5 years ago, there's another 2.5 newbies "voting" in the community as well.

It's a hard problem, I salute the leaders in the JS and npm world working on it, I wis them the best and advocate people giving them grace working in a very hard situation. But I'm also glad not to be part of it.

I think the reason is JS doesn't have much of a standard library. Java, C#, Ruby, Python, Rust, Go, and many others come with a large library you can use to write non-trivial applications without ever needing to fetch an external dependency. JS, particularly outside of Node, doesn't have that. To get functionality most other languages/runtimes include out of the box, you need to write a bunch of code yourself or pull in a dependency to use someone else's implementation.

It may be more about the quality rather than the quantity, though. Rust's standard library isn't very big. People sometimes complain that we have to fetch dependencies that are so ubiquitous that they are effectively part of standard Rust: crates like rand, futures, bytes, etc.

But even JS's built in string stuff isn't so bad that it somehow justifies leftpad existing, so I don't know...

Can't find it now but I remember an interview or article by Ryan Dahl, describing his original vision of Node.js. He related it to childhood enjoyment of building and piecing things together, and used tinker toys as an analogy saying he wanted code to be the same.

In other words it has been an intentional design decision from the start.

My take is that the lack of experience for the average JavaScript developer is absolutely a factor here. I don't think it's the only factor though. Here are some of the other pieces of the puzzle.

JavaScript's standard library is so thin on the ground that there's already a culture of "reaching for a library" to accomplish tasks that many languages do out of the box.

The monoculture is wide enough that the language caters to lots of paradigms and schools of thought. If there's one library that uses classes and method chaining, you can be sure that another will pop-up to re-implement the same functionality in a pure functional style. One will focus on type safety and another will abuse the dynamic bits of the language to make the code you write as terse as possible.

Amount of code shipped has always been a more important metric for JS than other languages because the nature of the web means that users have to wait whilst the source code is downloaded before your page becomes interactive (for a huge class of applications). This encourages developers to favour smaller libraries that solve for narrower problem domains.

It's become very trendy to write a smaller, faster, better, smarter version of existing libraries. The JavaScript community loves the process of picking a catchy name, registering a domain, designing a logo, and publishing packages as though they were businesses. This creates an abundance of packages that look great on paper, but with no users, patchy/non-existent tests and maintainers that haven't ever used the code in a professional context.

Finally, I think JavaScript is a deceptively simple language. It doesn't take very long before people (mistakenly) think they're close to mastering the language. By comparison, contributing to an open source project in a meaningful way is quite difficult, so these developers assume that other libraries must be written badly if they find it hard to contribute. Then they write their own, because they believe they can do a better job.

The ecosystem as a whole sees a lot of innovation, and pays for that with a lot of churn and a lot of dependencies. From a theoretical standpoint, it's a fascinating corner of modern programming. In a professional context, it horrifies me and I wish I could sanely cut npm out of the chain.

I think it's the sheer amount of programmers using JS. It's also a very approachable language, so it makes it easy to learn the language before learning good practices.

> While I find projects in those other languages to also have too many dependencies, it's no where near what happens in JS apps. I'm thinking of projects I've recently worked on in Rust, PHP, and Java.

My experience with these new languages is such that this feels a bit unfair. It's like insisting that a disaster with 1000 fatalities is "much worse" than one with "only" 200. It's ... true ... I guess, but there's something uncomfortable about making the comparison. Something has gone badly wrong if the comparison even needs to happen in the first place.

What I'm getting at is that e.g. Rust has an enormous problem in this area. It's not uncommon for me to see Node projects with over a thousand transitive dependencies, but on the other hand, I very frequently see Rust projects with over a hundred. And the Node projects tend to be more complicated than the Rust ones; they do more.

Take the last Rust program I tried to use, tealdeer. [1] If you don't know, tldr is a project that provides alternative simplified man pages for commonly used programs that consist entirely of easy to understand examples for the program. [2] What a tldr client needs to do is simply to check a local cache for each lookup, and if necessary update the cache online. It's a trivial problem that can be, and has been! [3], solved in a few hundred lines of shell (if you're being extremely verbose). How many recursive dependencies would you guess tealdeer uses? Depends on how you count, of course, but as of today the answer is ~133 deduplicated dependencies! For a program that's a glorified wrapper around curl!

Or another Rust program I looked at recently, rua [4]. In Arch Linux, the AUR is a repository of user maintained scripts for building and installing software as native Arch packages. Official tools for building and installing software already exist for Arch, but it is common for users to use a wrapper around these tools that makes fetching and updating the software from the AUR easier. It's a relatively simple task that (once again) can be done with shell scripts. rua is such a wrapper. As of today it uses 137 deduplicated dependencies!

These Rust programs are simple terminal tools to do tasks that are almost trivial in nature. And yet they require hundreds of constantly updating dependencies! The situation may well be better than what you'll find for Node, but it's undeniably disastrous compared to either simpler languages without a built in package manager (like C) or more complicated batteries-included languages where best practices continue to prevail (like Python).

[1] https://github.com/dbrgn/tealdeer

[2] https://tldr.sh/

[3] https://github.com/raylee/tldr-sh-client/blob/main/tldr

[4] https://github.com/vn971/rua