Hacker News Reader

When I create a new rails 5.2.2 app, I see 79 dependencies in entire tree. Which is about what you said, and a lot less than 1014, sure.

There are various reasons other than "low standards" that the JS ecosystem has developed to encourage even more massive dependency trees.

One is that JS projects like this are delivered to the browser. If I want one function from, say, underscore (remember that?), but depend on all of it... do i end up shipping all of it to the browser to use one function? That would be unfortunate. Newer tools mean not necessarily, but it can be tricky, and some of this culture developed before those tools.

But from this can develop a community culture of why _shouldn't_ I minimize the weight of dependencies? If some people only want one function and others only another, shouldn't they be separate dependencies so they can do that? And not expose themselves to possible bugs or security problems in all that other code they don't want? If dependencies can be dangeorus... isn't it better to have _surgical_ dependencies including only exactly what you need so you can take less of them? (Makes sense at first, but of course when you have 1000 of those "surgical" dependencies, it kind of breaks down).

Another, like someone else said, is that JS in the browser has very little stdlib/built in functions.

Another is tooling. The dependency trees were getting unmanageable in ruby before bundler was created (which inspired advanced dependency management features in most of the rest subsequent). We probably couldn't have as many dependencies as even Rails has without bundler. Your dependency complexity is limited by tooling support; but then when tooling support comes, it gives you a whole new level of dependency management problems that come with the crazy things the tooling let you do.

These things all feed back on each other back and forth.

I'm not saying it isn't giving rise to very real problems. But it's not just an issue of people having low standards or something.

You are correct, sir: 79 dependencies. Need more coffee!

The influence of Bundler is just another example of Yehuda Katz doing something initially perceived as unpopular having a massive long-term impact on developer ecosystems. He is the Erdős of the web dev world. I wish he would sell options on his future endeavours like David Bowie did.

Anyhow, in trying to keep my initial comment relatively brief, I held back on several points; what I really and truly don't get about the 750k thing is how anyone can track the libraries available. You know... with their brains. In Rails, gems are big ideas: authentication, an ORM, the ability to address all of AWS, send tweets. The idea that someone would even think to publish a "left-pad" (which I understand is just missing from the also-missing stdlib) as something people should import is what seems crazy. What makes you stop and think... wait, I'm typing too many characters, I need to see if there's anything on Github that can insert spaces at the beginning of a string. How would you even know what it's called? Is there a module for concatenating two strings? When does it become silly?

How is it possible that finding a library to add padding which may or may not exist, doing even cursory code review and integrating it not take longer than just writing a few lines of code?

Using the example of the escape-string-regexp module mentioned in the whitepaper... this would be a deeply flawed thing to add to your project. It has a hard-coded error message that has zero affordances for localization strategies. It is, at best, a few random lines from someone's hobby app.

If every one of those 1014 modules in an empty project have weird, unknown-unknown failure modes, that sounds like a recipe for trouble.

Finally, 1014 packages getting frequently version bumped is way less reliable in terms of unfortunate conflicts. Your surface area of all things that can go wrong shoots up and to the right... all to save a few lines of code?

Here's an actual example I'm dealing with right now.

I am trying to write something in more-or-less "pure" browser Javascript, for Reasons. (Not JQuery, not react/vue/etc, maybe some polyfills).

I need a "debounce" function. There are a bunch around as copy-paste code. The one in underscore is famous. There are some blog posts saying they have taken the one from underscore, but what they paste doesn't match _current_ underscore code. Underscore has apparently changed with improvements or bug fixes. The literal code in underscore depends on other parts of underscore, so if I want to copy-paste current one, I've got to do some work to refactor it to be independent.

I don't want to use underscore itself, because I don't want all of underscore to be shipped to client browsers when I just need debounce.

I'm going to spend several hours or a day or more on this before I'm done. (that I am fairly new to 'modern' JS doesn't help)

If there was just a debounce dependency that seemed popular and trustworthy that I could just take, in a few seconds instead of spending a day on investigating it, so I could get back to actually developing my domain func -- you bet I'd take it. (Yeah, I spent some time looking for _that_ too, and surprisingly didn't find it, despite everyone using an independent 'leftpad', there isn't an independent 'debounce' (which is much less trivial code even though it's still only a few lines when done)? Or did I just not find it?)

There are "ecological" reasons that lead to the nature of the ecosystem, various pressures and barriers in the existing environment, not just an issue of developer judgement.

The fact that this kind of JS code is _shipped to browser_ makes the size of the built product matter a lot more, which changes approach to dependencies.

Those of us who use _Rails_, Rails _comes_ with so much (including apropos the current discussion, various text mangling utilities which aren't directly related to the web domain specifically), that you don't need to look elsewhere for it. But _that_ choice has been subject to much critisism too, with people thinking Rails does too much, which has it's own downsides. It's all trade-offs. And the trade-offs often effectively happen at the "community" level rather than the individual one.

If it wasn't obvious from my barely-contained bias, I'm old-stock Rails and I have been present for all of the heckling about ActiveSupport. Those criticisms happened but largely amounted to a tempest in a teapot.

Rails between 3.0 and 3.2 was a painful time, as the merger of Rails and Merb didn't go smoothly. Yehuda got everyone excited about pure modularity but didn't stick around long enough to do the hard, boring last 10%. Thank goodness for Tenderlove. Today, you really can disable ActiveSupport if it offends you, but in the field you rarely see a Rails project without ActiveSupport. It's modularity that nobody would benefit from taking advantage of. It's libraries like CoffeeScript, Turbolinks and now thankfully Sprockets that people are turning off. (I would strongly argue that people should use Turbolinks 5, but that's a different thread for a different day.)

The thing is that DHH's whole deal was that there's an "omakase" best-practices menu that you can sign off on and reap a huge number of benefits for free. What people don't seem to understand is that the biggest of these benefits was not any particular feature of Rails, but that it was a consistent vision throughout... both in terms of what was there and especially what was not there.

I think that's why the fuzzy size feeling I perceive from most gems is so practical and comforting to me. They feel like the right amount of stuff, and I can be reasonably confident that they will fit my other stuff.

Honestly, I feel really badly that most JS devs have never experienced anything like that. Whatever gains have been obtained by adopting JS-everywhere, the losses in developer ergonomics are not properly appreciated.

I don’t really disagree with anything you’ve said, just want to add.

It’s not really 750k packages you’re keeping in your head. Instead, there are swirling communities of standard convention. This happens because the js community is so large and the code syntax so “forgiving” that many different “dialects” of js exist. There’s a reason “Babel” is called “Babel.” Within your dialect, one package may be better suited than another package, which does the same thing. A great example of this is where ‘_ => result’ may look perfectly coherent in one sub-ecosystem, another sub-ecosystem would more readily understand ‘function AsyncCalculationProvider(callback: CompletionFunction) { return callback }’, both for perfectly legitimate reasons. That people wish to ‘standardize’ on one common way of doing a thing within their sub-ecosystem is the reason people accept dependencies for dumb little things. That people disagree on how those things should manifest in the context of their subculture is the reason there’s 10 different popular ways to do the same exact little thing.

This isn’t even just ECMAScripts fault. The Web APIs provide a lot of great fill-in as a standard library (for example, for Date localization), but even here we do not see consistency. For example, the “fetch” api for browsers is nice enough, but is not implemented in Nodejs. If you want to share a signature for HTML requests between nodejs and browsers, you’re going to use “axios” or maybe “node-fetch” (which replicates the Web api but is more verbose), or you’re going to end up rewriting a wrapper for that functionality, and why would you do that when there’s a community that will immediately see and understand what you mean when you import the “axios” module?

Big optional standard library needed, but minimized deliverable also needed. Recent tooling (eg tree-shaking and Babel), make this more reasonable today than it was in the past.

If historical lessons continue to apply, what will likely happen is a few more significant iterations of shakeout as the community coalesces around something closer to consensus. It's important to remember that the vast majority of developers in any ecosystem are simply trying to use a tool to do a job; they are far less likely to commit a significant amount to the ecosystem and that's okay. Wikipedia is very similar in this regards.

Perhaps the best comparison is the evolution of the Linux distribution ecosystem. Maybe React is Ubuntu? The story is still being written.

> that suggests that the standard for what constitutes a valuable quantity of functionality is 7.5X lower in the JS community.

Err, does it? The first thing this suggests is that the JS community is 7.5x larger (true for Ruby, even larger factor for Perl and not true for Pyhthon [1]). The second thing this suggests to me is that npm is X times more usable than those languages package managers, which from my experience true for Python (not for Ruby though, dunno about Perl).

1 - https://insights.stackoverflow.com/survey/2018/#technology

I'm not trying be argumentative, but I honestly don't see how you conclude that 7.5x more packages === 7.5x more usuable.

I don't have a huge amount of experience with pip or artisan, but they seemed roughly similar to both rpm and bundler in terms of concept and operation. You have a file with a list of modules to require, an executable tool that downloads all of those modules and their dependencies and finally a generated file that tracks the snapshot state of all dependencies.

I don't perceive npm bringing 7.5x game to that evaluation.

Don't get me wrong: I'm thrilled that it's working for so many people. Java was officially working for people when Rails became popular, too. However, popularity is a dangerous way to measure whether something is actually objectively better. Sometimes the most popular thing really is better. And sometimes Putin is super popular. It's complicated. :)

> If npm has 750k+ modules available, that suggests that the standard for what constitutes a valuable quantity of functionality is 7.5X lower in the JS community.

Or that JS lacks a decent stdlib! As evidenced by the left-pad fiasco, JS simply lacks things that are either available by default in other languages or easily include-able from the system libc (in c/c++).

Totally agree. If JS had only a decent standard lib for string, collections and math I bet the number of packages would be reduced by a lot. I wonder what the reception of such a thing in NPM would be. Would people appreciate having only one, but larger dependency?

There's an old engineering joke about how there's 13 incompatible power socket standards in use on Earth today. Sensing an opportunity to engineer a better solution, they created a power socket that is simpler, easier to use and cheaper to install than all current standards. Now there are 14 power socket standards.

I know that one but I don't think it applies here. In JS there really doesn't seem to be a comprehensive library. When I look at a lot of NPM packages they are simple 5-10 line functions. Consolidating them into a larger library would help a lo in my view.