Bundle Phobia: find the cost of adding an npm package to your bundle | Hacker News Reader

Bundle Phobia: find the cost of adding an npm package to your bundle

bundlephobia.com

100 pointsorf69 comments

The real problem with npm and JavaScript is not the package manager, it is the lack of a standard library bundled with the runtime that has such basic functions. To someone coming to JavaScript from a Java/C# world, the fact that a package like isArray actually exists, nevermind the ridiculous number of downloads it has, is mind boggling. Using a dependency like this on a Java project will get you crucified in any sane team.

If you want to get out of this dependency hell, bundle these small, essential functions into the runtime.

Depends on the team. I would not approve anything other than using a runtime library or importing 'core-js/fn/array/is-array' from the wonderfully modular core-js: https://github.com/zloirock/core-js

Although for this particular strawman I think `foo.constructor === Array` is probably enough.

What if foo is `null` or `undefined` ?

Array.isArray is available in the runtime, but it was a later addition so folks may be downloading it for compatibility. There is a LOT of that in the javascript ecosystem.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

They are bundled, it’s called ES6/ES2015. But for backwards compatibility, the userland packages are still used regularly.

> They are bundled, it’s called ES6/ES2015

I wish. See the `Set` implementation [1], for instance. It's lacking very basic stuff like union and intersection, forcing the user to copy/paste code or add another dependency.

Does someone know why they do that? Looking from the outside it seems that the committee in charge of this doesn't know what they're doing.

[1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Agree, Set is a disappointment.

I think the “Why” is because it’s easier to get everyone to sign off on small changes than large changes that can never be removed.

The web platform has to live for a long time and be backwards compatible.

But the `Set` was completely new. And things like union and intersection are basic features and without it the `Set` thing is almost useless, so it wouldn't make any sense to remove they in the future as much as it didn't make sense to not include them now.

I agree with you but there are ways to perform union and intersection with existing APIs.

http://2ality.com/2015/01/es6-set-operations.html

> Long-term, I expect JavaScript to have built-in functionality for this, e.g. via functions that operate on iterables

I agree with your general sentiment, but Array.isArray() has been standard since about 2010. It’s even in IE9. Anyone using an npm package for this feature deserves the beating.

IMO the problem with npm is that it makes no distinction between snippets, libraries, frameworks and scripted command line tools. An npm package can be any of these. I feel there perhaps should be separate namespaces and package managers for all four — e.g. an inlining minimal “node snippet manager”, a standardized framework installer for things like create-react-app, and so on.

Some packages ship with both importable module as well as a global CLI. So it’s hard to categorize packages on into a single category.

The library should be in a library package manager, and the CLI tool should be installed separately with a dependency on the library. There’s no benefit to mixing these together.

If you hand me a .framework and a .app for macOS, it’s immediately clear which is which. This isn’t an unreasonable standard for distributing programs.

The flipside is that the standard library from Java / C# are huge, large enough that there's basically one complete and proper implementation.

In JS land, you have multiple different implementations with different feature sets and behaviors. Code written for V8/chrome/node may behave differently in safari/jscore or IE/edge/chakra or firefox/spidermonkey.

By flipside are you suggesting that managing multiple feature sets/behaviors across implementations is a good thing?

This can be the case for big or difficult to implement functionality like async/await. But for smaller functions like isArray, isObject and leftpad this should not be the case.

C++ has a lot of different standards, compilers and implementations. There are differences in vc++ and gcc that can prevent compilation without flags or custom implementations. But I am sure that simple functions like printf or strcpy behave same across implementations.

Even in Java you have Oracle JVM and android's custom dalvik/ART. But the standard Java functions still work the same. Same with .net/mono.

Keeping the Node standard library small seems to be a deliberate approach. See this post for some elaboration: https://medium.com/the-node-js-collection/keeping-the-node-j...

I think perhaps the biggest problem is the lack of a flat dependency system. So not only does your project depend on isArray, it may depend on 5 different versions of it. The sheer number of deps a seemingly normal React app has is crazy. Especially with webpack, babel, etc.

Npm3 has flat dependency system.

It doesn’t. It has a deduping system, which may still leave you with multiple versions installed if there are incompatible version requirements in the dependency tree.

A flat system would mean there’s only ever a single version installed. Or at least, I think that’s what OP means, since that’s the only definition I’ve ever heard.

It's what was there with bower but that has a lot of problems as well. So many in fact that everyone has moved to npm

Perhaps WebAssembly will help with this, we could make a standard library in WebAssembly which is highly performant, and can be called from regular Javascript code.

Not likely. Calling from javascript to webassembly is slow, webassembly can't create js objects, and usually it is not much smaller than the equivalent js code, due to lower level of abstraction.

That's interesting - I see from some research that you are right in stating that calling from javascript to webassembly is slow[1].

Which is.. kind of surprising, I didn't realize that.

I sort of expected WebAssembly to have a performant JS API, since its origin is in asm.js, which was a peformant subset of javascript - but I see now that asm.js too had this same problem.

Which again surprises me, as I sort of expected it to be a bit like Macromedia Flash, which I recall had a Javascript API that seemed to perform well enough, although I suspect it wasn't used for high-performance scenarios, so maybe not.

It seems odd to me that the interfacing between these two technologies would be so separate that it would cause performance issues.. why would it not be something that is improved? Perhaps you'll have a flip where you write javascript and some other language, and the javascript runs as an interpreted language on the same VM with some compiled code, together...

Hmm.. an interpreted language interfacing with compiled bytecode efficiently... Python.. LISP can do it.. why not Javascript/WebAssembly?

[1] https://github.com/WebAssembly/design/issues/1126

If you were planning on writing modern Java and targeting JDK 1.3 you'd have just as many libraries.

> This thing lets you understand the performance cost of npm install ing a new npm package before actually adding it to your bundle.

Unless you are specifically concerned about the `npm install` performance in node, in which case the npm package size is relevant, you never include the whole NPM package in your bundle. Most universal packages include separate dist files and separate entry points for node usage, so most of the content is actually not included.

If you are truly interested in bloat, let your bundler give you a summary at the end. Webpack can give you detailed information about what files were included, how much each file costs, and why the costs were incurred.

Lodash is a good example of this. The whole package is pretty big, and includes a functional programming variant (which I suppose is mostly wrappers around the other stuff, but it still adds weight).

So in practice you'll pretty much never be using the full package on account of 1) going for either require('lodash') or require('lodash/fp'), but not both, and 2) requiring specific functions through require('lodash/each') or whatnot.

The only way to get an accurate picture is to analyse the bundled package, or I suppose as part of the pipeline of whatever packaging solution you use.

Author here. I made this because finding package sizes of front end libraries was a pain, especially after modular javascript caught on.

You can also see size results now when browsing a package on yarnpkg.com (look for "Size in browser") or as a sheild on some repos (WIP).

Suggestions welcome.

Wouldn't it make more sense to optinally add all dependencies?

If you're the author of `react-button`, it wouldn't make sense to add `react` + `react-dom` to account for it's size. In most cases, peer dependencies are already present in the user's projects.

Really cool site, material breaks it though. Can't find dependency "button" :)

Thanks for building!

Can you fix this?

    MissingDependencyError
    This package (or this version) uses `./lib-cov/http`, but does not specify them
    either as a dependency or a peer dependency

For input chai-http

Where are you from that 3G is "emerging"? I think we have like 90% LTE coverage for major providers, just a question of hardware supporting that.

Oh boy. Guys, should we tell him?

It's about emerging markets.

What are those? According to Wikipedia most countries have 60% 4G penetration. Not sure what it means and how accurate it is though.

Turns out 40% of several billion is a pretty large number.

Sorry, I think I broke it because I didn't understand it. I was really curious to see what angular-cli came in with in your tool, because my feeling is that it's very costly in terms of dependencies. The result never came back.. stuck at "Calculating File Sizes" was I not supposed to look at such types of packages?

Suggestion: also show the number of dependencies and sub-dependencies.

That's usually where I run in to bundle-phobia mentality (whether its warranted or not is another discussion).

good call, i'm a python guy who usually does science/infrastructure stuff but occasionally look at what is goin on with our front end libaries. Whenever i read thru our dep-chain i'm like holy shit guys, isArray is in here 4 times with 4 different versions.

All in all though cool project, looks great, works fast, provides good info. thanks and congrats

I don’t understand if you are the author or just use a suggestive writing style.

Wouldn't using something like Yarn resolve this issue?

No, because Yarn doesn't flatten dependencies that aren't in the same semantic version range.

> isArray is in here 4 times with 4 different versions.

just ... wow :D

I've used this NPM graph tool in the past to determine the cost of adding a dependency: http://npm.anvaka.com/#/

Love this tool. BundlePhobia should just merge its abilities into this thing (or vice-versa) they'd be way powerful together

Here's a great VS Code extension that shows package size inline next to your import/require statements:

https://marketplace.visualstudio.com/items?itemName=wix.vsco...

Leftpad: https://bundlephobia.com/result?p=leftpad@0.0.1 looks good, let's use it!

But seriously great service, I was thinking about a CLI tool like this to estimate dependency impact before using it.

left-pad only adds 7ms on edge, seems worth it. On the other hand, Jquery adds almost a second. And you hardly ever see pages that don't use it. (Unless they use some other heavy js library). But it's just soo convenient.

7ms is an eternity. You can do half of this [0] in 7ms or you can download one line of code. I’d be out of a job if I introduced 1/10 of that time for text padding.

[0] http://www.adriancourreges.com/blog/2017/12/15/mgs-v-graphic...

Its completely meaningless to compare the render time of a single frame of a AAA video game with the load time of web page over a network.

"completely meaningless" may well be putting it a bit harshly: there is good lessons to be learned by drawing on lessons learned across different fields of computing.

7ms is hardly anything but apply it to a list of only a 1000 elements and you have now slowed down the user experience by 7000ms (I know, this would all be buffered and hidden, etc. but this is just one tiny component and eventually all this stuff leads to real perceptible awful jankiness that could have been avoided if people allowed themselves to think a bit about performance).

I often find that most of the people who make the Jquery argument are the same people pushing Angular or React, both of which are bigger libraries...

I would love a version of this where you could paste in your package.json and it adds up all your dependencies and offers suggestions of popular dependencies that can be removed.

I didn't realise that others also actively avoid npm. I can't really place my finger on exactly why I do, but I guess it's somewhere between the cargo-cult mentality in Javascript (which makes me associate node packages with poor quality programming) and not seeing the point of having another package manager beyond what my distro provides.

There is a big distinction between the purpose of your distro package manager and the purpose of a languange dependency manager such as npm, pip, rubygems, cargo or composer.

While the first is for installing applications on your system (and also used for c libraries, because c has no official dependency manager), the second serves much more specialized purposes tailored for the language, such as

- os-agnostic (imagine the work in getting all them distros update just one package)

- compiles/postprocesses dependencies in deterministic ways

- etc.

Any projects we do where npm packages are a dependency inevitably end up with frequent transient build failures. npm install will kick out with some warning or error that's buried well past the 4MB log limit from CircleCI. Rebuilding the project always works.

I haven't been able to determine if this is an NPM issue or CircleCI issue, but short research periods seem to indicate the former, and I find the availability of packages in the manager to be 'unreliable'. That's not really something that I want to deal with in software that is managing my dependencies, but that's the state of the world. Apt repositories have a tendency to do the same thing, actually.

tried yarn?

Yup! It has been good, we're just not using it in production yet.

It doesn't look like TypeScript bindings work on this, but it'd be nice to see the overhead of @types/ packages.

There is no overhead for typings. Type information in TS is (mostly) gone in runtime.

I think type info in TS is 100% gone in runtime. Here's a blog post that explores why you can be confident of this: http://neugierig.org/software/blog/2016/04/typescript-types....

Seems to be broken. Keeps spinning.

>*"JavaScript bloat is more real today than it ever was. Sites continuously get bigger as more (often redundant) libraries are thrown to solve new problems."

Very true, pointed criticism, but i'm afraid part of the answer is avoiding npm altogether.

How does avoiding the use of a package manager fix bloat? Or do you mean to say everyone should be writing vanilla JS with zero dependencies?

Not true.

A massive majority of that code you download is from 3rd parties.

Ads, tracking, analytics, Instagram & Facebook embeds, etc...

Now you can suggest that we should move to not having any of these but from a business perspective that’s... very hard.

Trimming is possible, of course, but the reality is that npm is not the problem- sites were bloated with JS before it came along.

My company's entire business revolves around embedding tracking and analytics JS on our page and serving it to people and all of those plugins combined are still dwarfed by something like React or Angular by a factor of 2x.

The problem ends up being when you're embedding this stuff and the provider has their own dependencies they're loading. Hubspot's embedded web forms pulls in an entire React framework. Mess.

If you're serious about doing this stuff, you include the bare minimum JS possible (GA/GTM, usually), store GUIDs in your request logs and put that into your analytics data pipeline.

And keep telling the Google rep to take a hike when they put pressure on you to use AMP.

Sure, I’ll believe you and not what the browser reports.

AMP may be bullshit but it means a 5-10x increase in traffic so... again, missing the business case.

If eyeballs is what your metric is, that might be fine, but our traffic needs to convert into something (in our case a form fill, in others a sale). AMP dropped our conversion rate through the floor. And if you want analytics & tracking, you're very limited or you have to use all/only google services.

It's a great way for google to build a captive market of content providers, but once everyone is serving AMP pages, the benefit to brands (SERP) is gone.

Conversely - we see decent conversion rates (at worst no difference to regular) so... maybe it’s your approach?

I hate AMP with a passion but your specific objection hasn’t been true for us.

Reply on news.ycombinator.com