Hacker News Reader

> I really would like to see a discussion on this, as most people seem to put a lot of blame on JS as a whole, while it's mostly a handful of popular projects generating all the complaints.

Discussion on what? You're right.

I know we joke about left-pad, but like you pointed out, a lot of big js hitters don't have many if any dependencies. That's true, but irrelevant.

Those dev-dependencies are still potential security threats, with all the minification and other crap, it's really hard to know what gets injected into the final runtime. And if not security, it's still development hell. Development, yes, but that's a if not the thing that programmers really care about.

And even if runtime dependencies are less common, there are a lot of developers that still do ascribe to using as many deps as possible especially because the web can be quite fragmented and they have to support a myriad of different target platforms. So even if it's a lesser issue, I think it's fair to talk about the js ecosystem as a whole when making criticisms about it's dependency disasters.

Development dependencies are especially terrifying because we generally don't use any sort of sandboxing. Any one of these dependencies could append to .bashrc to get your computer to run literally anything, and then hide the evidence.

And developer machines are particularly juicy targets because they often have ssh keys to production machines lying around.

and even scarier if on a corp buildbox server that typically firewalls inbound, not outbound/lateral

This is precisely why I created https://gitlab.com/mikecardwell/safernode - I can run "npm start" or "npm install" just like any other nodejs developer, but node is not installed on my host, and my .bashrc and any other files in my homedir are not at risk or being read or modified

> If those projects were able to get it together we wouldn't even be having this conversation. Solve this and you'll solve the biggest complaints people have about JS.

So the way to go would be

1. compile a list of opensource-dependencies by usage/complexity

2. go through their dependencies

3. find easy fixes and create PRs for the projects.

That actually sounds like a good idea, but you need step two-and-a-half: convincing the maintainers of those dependencies that using this strategy is hurting the ecosystem.

> Solve this and you'll solve the biggest complaints people have about JS.

Or, perhaps, we can just tell people that their complaints are nonsensical when most of the deps are not actually packaged into the final app or being used at runtime, instead of contorting ourselves to fit their silly worldview.

I'm not losing any money over this. Why should I care?

Not saying this is automatically a problem, but you will lose money over this. When a build dependency is unavailable or broken, you cannot create production artifacts. You'll lose money over this because broken builds lead to incidents, longer time-to-recovery during incidents, and slower development. All of these cost money.

Your exposure to defects in your build dependencies will depend on a lot of factors, but the exposure will always be there.

Then cache the dependencies you're using.

Deps aren't unavailable or broken if I cache and pin them, and I do, because not having done so would have cost me a lot of time and money a long time ago.

People did lose money when left-pad was not available. Sure, it was only a day and the whole class of issue solved now, but there are other modes of attack.

And dev dependency bloat might be nonsensical to you but it's not nonsensical to me. Different people have different priorities, that's it.

So yeah, maybe you can afford not to care, but that doesn't mean you should give us that can't a hard time about it :)

I'm pretty sure then that if you actually knew what was inside your C compiler, you might lose your mind. There is something akin to an entire LISP inside gcc. And it uses its own host of deps, which themselves have deps... https://packages.gentoo.org/packages/sys-devel/gcc/dependenc.... It's not like these compilers are large for no reason, there's definitely code in them.

What do you consider the preprocessor if not bloat? CMake or ninja or whatever Google's build system is, is that bloat?

Maybe you just haven't seen a fun enough pipeline. But the web is sufficiently complex that you need a sufficiently complex set of developer tools to build for it in an optimal way. You can try and skip steps there but you'll likely just end up reinventing what others have built.

If you know the leftpad class or issues is gone, why bring it up? Why not just start with the "other modes of attack"? Why do your priorities, whatever they are, matter?

I'm not the one giving others a hard time here, the people who maintain that it is fun and trendy to hate on a technology they think is "bloated" (whatever that means) are giving me the hard time.

Is there anything stopping a dev dependency from injecting malicious code into your app?

Yeah, it's called pinning. When you use yarn or npm, they generate a lock file that pins the exact state of your downloaded set of node modules, so you can't accidentally download a different set of artifacts that might be poisoned.

As for "but what if they hide it?" that's just a problem with compilers. A very "Reflections on Trusting Trust" sort of thing. I'm usually a few versions behind, so if anything like that were to exist it would have been caught by someone else.

For people who may want to read more about the " Reflections on Trusting Trust": https://www.win.tue.nl/~aeb/linux/hh/thompson/trust.html and https://wiki.c2.com/?TheKenThompsonHack.

> the ES6 standard library is not as bad as people claim.

I don't know how bad some people claim it is, but I claim it's pretty bad when it lacks basics such as string formatting, str{f,p}time, others that I can't bring to mind right now but I'm sure I'd find if I trawled through my old code.

So, assuming you're right, what might be a strategy to push these projects to a better place?

A few ideas of what we could do:

1. Since they're hurting the perception of the community, maybe Babel/Webpack/ESLint could admit that using multiple packages is the cause of most complaints against the Node.js/NPM community, and revert to single packages? Babel already use a monorepo, so I don't really see any disadvantage from their side. Most Webpack loaders are maintained by the same people who maintain the core packages. We need the whole community to push for this. I've done my part but I'm only a single person. We need more people reproducing my claim, seeing with their own eyes and complaining to those projects.

2. The community could rally around projects that don't use multiple packages. Sebastian McKenzie (writter of Babel) is working on a transpiler/bundler/linter/tester called Rome. There's also SWC written in Rust, which is a transpiler, but they're working on a bundler.

3. Using ES6 modules directly in the browser instead of using transpiled/bundled output. I'm in Europe, which is a big holdout on modern browser adoption, but even considering this, 99% of our customers are on evergreen browsers. With HTTP2 and gzip on the server you get the same advantages of using Webpack+Babel+minifiers without the bloat on node_modules.

I’ve been playing with ESBuild, and it has definitely influenced my thinking that dev tools should be a single binary if even remotely feasible. It’s just so much easier to deal with!

Your comments makes it sound like you don't realize Javascript runs on the server where large stacks of transitive runtime deps are actually common, and you're suggesting that because there's another problem (development deps) then the other problem (runtime deps) doesn't exist.

Any programming language runs on a huge stack of dependencies. For most languages it‘s just named „stl“ which makes it okay for most people, funnily enough.

Hell, when is the last time you saw a company C++ project without boost and at least a dozen other libraries?

I do run a lot of JS on the server and the number of dependencies we normally run is not even in the same ballpark of the number of dependencies we have on the frontend... except when the backend project also uses Webpack/Babel/ESLint/Jest/etc.

Great point.

What are your thoughts on RomeJS and trying to unify the entire toolchain?

https://github.com/romefrontend/rome

I think it's the future of tooling.

There's no reason to have Babel/ESLint/Terser live in different packages: they're just parsers with slightly different outputs. There's no reason to have to parse your code 3 times during compilation.

Also, a bundler operating in the AST is much more efficient, as has been proven by Rollup. But parsing a 4th time is also a waste, so let's put it together with the same tool.

And while you're also there, your test suite also needs a "bundler" of sorts inside itself to transform the code and run tests. Why isn't it the same tool? Jest and Webpack have so many small differences that makes life difficult. So put it in the same tool, too.

However... another thing that I hope to see becoming popular is not using a bundler/transformer at all. Except for special situations, or for some regions, you don't really need transpiling/bundling it at all, since 99% of your visitors are on evergreen browsers.

By Evergreen Browser I assume you mean that the browser will have latest support for language syntax? That does not get rid of the problem of bundling though. And modern SPAs are so complex that bundling your own JS would be a nightmare. This is why NextJS is so great, they split the JS into SPA "pages" and let you dynamically load it from the server. I don't know how you could build that into native JS comfortably, since modules are smaller slices by far.

I will die on this hill with you.

I would also add that we should be striving to release uncompiled JavaScript, since it is more auditable, more debuggable, and is naturally distributed. Dev dependencies are often basically compilers (Babel, React, TypeScript, Angular, etc), and JS compilers should be avoided if possible.

I still remember working for an old school .Net shop where we wrote raw es5 with one dependency: jQuery. We used discipline to get around the limitations of es5 which came naturally to a team of C++ developers. I probably have my rose tinted glasses on too tight but it felt pretty good and we were very productive.