Hacker News Reader

Comment by PragmaticPulp | Hacker News Reader

732K downloads per month.

722K of those downloads come from this package by the same authors: https://www.npmjs.com/package/handlebars-helpers-ncc

The handlebars-helpers-ncc package contains 130 different utility dependencies including a few actually useful things like functions to convert Markdown to HTML, but also some weirdly trivial packages like is-even.

I suppose this was a brilliant way for the authors to generate staggeringly high NPM download counts for their packages: Repackage other people’s useful code into convenience functions and then include their own trivial package dependencies several layers deep to multiply their overall downloads count.

I wonder how many jobs they’ve applied to while bragging about their millions of monthly NPM downloads.

View on HN

This is a very well known trick that we don't talk about much in public, but this is how most people with hundreds of repos on NPM get to this number.

One very popular terminal progress bar spinner has over 30 dependencies. One with the list of colors, another to display colors, another to clean the line, another to check if emojis are supported, another one with a list of emojis... plus a whole lot of wheel reinvention.

It was visible that most code in the sub-packages were just functions copied almost verbatim from StackOverflow.

It was also extremely limited when I tried to use it, and had quite a few bugs (it only really worked well on Macs at the time), despite the extremely large number of dependencies. Reimplementing took about 20 lines.

This is what drives me insane about NPM. Libraries are useful, and even package splitting utils like underscore or babel make sense in some scenarios. But it only takes a few packages like these to explode the size of the dependency tree.

> But it only takes a few packages like these to explode the size of the dependency tree.

and create another future opportunity for a supply chain attack

Babel's splitting doesn't make much sense and is purely ideological. It's a monorepo, so split repos are not a development advantage. Plus, releases are often in sync. Plus, they recommend using presets rather than cherry-picking the specific plugins, making it kinda useless for end users. Plus, it's a CLI tool, so it's not really a necessary optimisation (and it would be a bad optimisation at that: having everything in a single package would make everything smaller and easier to compress).

For a long time, Babel plugins were entirely smoke-and-mirrors. If you looked under the hood at the source code of a "plugin" package, all it would contain was something like:

    require('babel').____secret____.enablePluginFoo = true;

There was no actual plugin API, just selectively-enabled logic hardcoded into the main packages. The whole thing was fake-it-til-you-make-it.

That's actually pretty clever, in a "all hail the dark lord Cthulu" kind of way.

For built-in syntax parsing, yes - those official plugins just told the parser to enable those specific bits of syntax.

Actual plugins for non-standard syntax were all real AST visitor-based modifications, though.

I found this out some years ago while investigating how I might write my own Babel plugin for some non-standard syntax. At the time there were no such facilities available, only the hidden built-in parser flags.

How long does it take to check the dependency tree of a package? If the package is trivial enough for it to not be worth to go through the dependency tree for 10-15 minutes, it wasn't worth it to include it in the first place. I'd rather have arbitrary depth of dependencies than some kind of governing system in NPM.

Wow, you just gave me a great idea for libc. Why not split it in libprintf, libscanf, libexit, libitoa, libstrbcpy etc. ? I'm sure the world will be better after this.

Ah, the infamous npm spammer jonschlinkert. This guy has even crazier packages like https://www.npmjs.com/package/ansi-red and I quote:

  Related projects
  ----------------
  ansi-reset
  ansi-bold
  ansi-dim
  ansi-italic
  ansi-underline
  ansi-inverse
  ansi-hidden
  ansi-strikethrough
  ansi-black
  ansi-red
  ansi-green
  ansi-yellow
  ansi-blue
  ansi-magenta
  ansi-cyan
  ansi-white
  ansi-gray
  ansi-grey
  ansi-bgblack
  ansi-bgred
  ansi-bggreen
  ansi-bgyellow
  ansi-bgblue
  ansi-bgmagenta
  ansi-bgcyan
  ansi-bgwhite

That fucking ansi-red "package" has 1.3M weekly downloads.

I have to actually check to make sure he doesn't have packages like regexp-left-parenthesis, regexp-dot, etc.

Oh, that's awful. It depends on ansi-wrap (another pretty much 1-liner package) with 2.6M weekly downloads.

Is there a site or code snippet where I can see which of the popular dependants was fooled into using this crap and is driving most of the downloads?

`npm ls <package_name>` or `yarn why <package_name>` is a good start for figuring out how a package ended up in your node_modules

Sure, I was looking for the opposite...which popular node package is pulling in ansi-wrap.

https://www.npmjs.com/browse/depended/ansi-wrap

Close, though sorted by popularity/downloads is really what I wanted...how to find where all the downloads are coming from.

I think that is sorted. If you go to the first result for `ansi-wrap` -> `ansi-gray` -> https://www.npmjs.com/package/fancy-log ("Log things, prefixed with a timestamp." by other creators: 2 398 259 weekly downloads)

Ah, yep...thanks. So we know where this particular bit of madness could be fixed.

The code of ansi-red for those interested:

    var wrap = require('ansi-wrap');
    
    module.exports = function red(message) {
      return wrap(31, 39, message);
    };

And ansi-wrap:

    module.exports = function(a, b, msg) {
      return '\u001b['+ a + 'm' + msg + '\u001b[' + b + 'm';
    };

Now imagine those things inside a real utility that gets adopted by some popular package.

But from the average programmer's perspective this is clearly something they want. I see this general trend as a symptom of the inadequacy of JS to out-of-the-box address the tasks it's used for, which (as we note here) opens an outlet for spamming and security liabilities.

Having one package for all of these — that's okay and something people might want. Intentionally spreading it out to 26 "packages" — just no.

But the package that's generating all these numbers, handlebar-helpers, does precisely that. Sure, the spammer gets internet points, but those don't matter.

Security matters. Performance matters. Hackability/forkability matters. This kind of practice reduces security and auditability of systems, and makes projects harder to hack and fork.

From a security perspective what issues do you see besides auditability? It seems like the surface area for supply chain attacks is expanded needlessly by these types of shenanigans but I'm not sure how that translates to concrete impact to a project. I suppose resources required to scan and detect a package takeover are needlessly consumed but that's more upstream/centralized.

But I'm with you, this is ridiculous and auditability alone is reason enough to avoid it. Even if you were trying to make a case for composable packages this is absurd to the point that I wonder if it's self aware criticism of npm or an inside joke.

> does precisely that

Precisely what? That's not one self-contained package I said is okay.

No. What programmers want are solutions to problems.

Nobody is asking for the madness of having proper-sized packages split into 100 smaller packages with zero advantages and actually a few disadvantages (difficult to audit, impossible to fork). This is what is being criticised here.

It would be perfectly fine to have a single `terminal-utils` package that contained all of this. And no, handlebar-helpers doesn't count.

> I see this general trend as a symptom of the inadequacy of JS to out-of-the-box address the tasks it's used for, which (as we note here) opens an outlet for spamming and security liabilities.

Two wrongs don't make a right. Package authors should not spam, regardless of the issues with of the ecosystem or language.

I get the ridiculousness of those tiny packages. On the other hand I also hate to have my codebase bloated by some kitchen-sink "util" package of which I'll use about 0.1% functionality. (while, if I were really strict, I'd also have to audit and track the remaining 99.9% of the code.)

Tree-shaking can help at least keeping the build artefact small, but it doesn't help with the codebase during development.

Of course there is still a difference between fine-grained dependencies and things that shouldn't be dependencies in the first place.

There is a sane middle ground between tiny and bloated, though. I don't see how using a false dichotomy justifies the issues we're complaining about.

Especially when most of those packages are almost never used by themselves, and are instead part of a meta-package with dozens of dependencies.

ansi-red is not a difficiency of JS. It's just stupid. It the equivalent of a package called add2 and another called add3. No language I know of has that built in and if it did it would arguably be a bad sign

It's ridiculous and amusing at the same time. However, I blame the people depending on libraries such this; not the opportunists enabling them. They either don't know what they are doing (incompetence) or can't be bothered to do anything about it (indifference). The combination of can't be bothered and wouldn't know how to is what leads to people using shit like this.

In the defense of javascript, dead code elimination is kind of hard because it is such a dynamic/messy language and that kind of pushes people in the direction of ridiculously fine-grained modularization. Languages with an actual type system and compiler tend to produce artifacts that only include code that is actually needed. That requires dead code elimination that actually works. Beyond what comes with the browser, there isn't much of a standard library. That's for the same reason. You'd either end up shipping the whole thing on every website or depending on some convoluted tooling in an attempt to strip it down to what you are actually using. Minification is of course a thing but it usually boils down to more obfuscating than actually removing dead code.

My strategy is to generally avoid the whole ecosystem as much as I can. I've been using Kotlin-js lately. Great libraries, runs in a browser, reactive styled components using fritz-2, web compose, or if you really insist react. You still get exposed to some of the madness (like webpack breaking between minor releases) but mostly you are shielded from that.

I don't think dead code elimination would be the end-all to that problem either.

Yes, it reduces the space cost of having dead code in your build artefact, but it doesn't solve the problem of having dead code lying around during development.

The more code is in your dependencies, the more effort you have in auditing - even if it's only to ensure that code is actually dead - and doesn't cause any unexpected side effects or other behaviour in your program.

Let me balance the chi by linking to a prolific open source contributor who makes actually-good repos: https://github.com/sindresorhus?tab=repositories&q=&type=&la...

umm... I just randomly checked and this person also seem to have packages which are of few lines. for e.g. I checked `is-fn`:

    export default function isFunction(value) {
      const type = Object.prototype.toString.call(value);

      return type === '[object Function]' ||
       type === '[object GeneratorFunction]' ||
       type === '[object AsyncFunction]';
     }

This has 120k weekly downloads: https://www.npmjs.com/package/is-fn

It's a polyfill for a buggy function, so it's not just arbitrary.

The real question is why JS can't provide the answer to "isFunction" in a builtin method

It can: JavaScript has a `typeof` operator that can be used for this. If you read the readme for is-fn, it points out that the package is only useful for code that needs to run in certain older browsers that have a buggy implementation of `typeof`.

Spammer is a polite definition. His background is in marketing, the business he links brandscale[.]com is broken/abandoned. How long until we see these packages used as trojan horses? Or sold to third parties?

It's very funny to me that these packages exist while one of his bigger projects (https://github.com/enquirer/enquirer) lists the following reason under "why use it":

> Lightweight - Only one dependency, the excellent ansi-colors by Brian Woodward.

> Several years ago I switched careers from sales, marketing and consulting[1]

The author's bio on his GitHub profile explains it: he's approaching development with a mindset of sales and marketing. It's also followed by his GitHub stats front and center.

[1] https://github.com/jonschlinkert

ansi-red, making the world a better place!

making the world a better place, one ascii color at a time

I think we know how he created more than "1000 open source projects".

s/projects/functions

> I wonder how many jobs they’ve applied to while bragging about their millions of monthly NPM downloads.

isEven claims to be a learning project. It depends on isOdd and isNumber, both of which were admitted (IIRC) to be explicit NPM download boosting projects. Such that the author of isOdd added it to major modules whenever they could

isNumber would make for a legit package. Is there a good way in javascript to determine confidently if an arbitrary input is a number?

e.g: [1, "one", true, false, 0, "zero", null, undefined]

edit: also ["1", "0", etc]

edit 2: also [NaN]

  “number" == typeof variable

Works for decimals, too.

Although maybe you meant to include

"1"

In your examples?

Unfortunately "not a number" is typeof "number"

    "number" === typeof NaN

Returns true.

That's a tricky one that I think would depend on use case. Yes, of course NaN isn't a number, but it's still kind of a number, depending on context.

NaN is a float according to IEEE754. All numbers in Javascript being floats, it makes sense that typeof NaN === "number".

Now, the fact that IEEE754 is insane is something else, but JS does nothing wrong (aside from not having not-floats)

> Now, the fact that IEEE754 is insane is something else, but JS does nothing wrong (aside from not having not-floats)

JS has BigInt - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

BigInt is relatively recent, all things considered. JS only having floats is a conception flaw.

I mean, it depends on context what “numbers” are acceptable. For an overly general solution:

  const allowNumbers = {
    all: () => true,
    notNaN: n => !Number.isNaN(n),
    finite: Number.isFinite,
    integer: Number.isInteger,
    smol: n => Math.abs(n) < Number.EPSILON,
    lorge: n => n > Number.MAX_SAFE_INTEGER || n < Number.MIN_SAFE_INTEGER,
  }

  function isNumber(n, allow = allowNumbers.all) {
    return ("number" === typeof n) && allow(n);
  }

This isn't bad, but for maintainability, it should be split into at least nine packages.

In the type of things that Javascript considers 'numbers', it's a value used to indicate that it's not of the type of things that math considers 'numbers'.

... typeof foo === 'number'

typeof x === 'number' && !Number.isNaN(x)

This seems like the best solution.

Any weird edge cases against this one?

+/- Infinity, better to use Number.isFinite.

Infinities.

Comparing it against the list of examples given in is-number, this solution does not properly handle numbers represented as strings, and it also recognizes Infinity as a number.

Because strings aren't numbers and Infinity is a number

Strings are the safest way to hold big numbers in JavaScript due to how JavaScript represents numbers. Also no, infinity is not a number. It's a concept.

Numbers is a concept. For some purposes it is useful to consider a number that is infinite, which is why concepts like the extended real number line have been invented.

    typeof v === "number"

The rest is parsing territory.

Looks like the author of these packages agrees. While the is-even and is-odd packages are under the i-voted-for-trump user with the description "This is a joke", the is-number package [1] is still kept under the main profile of the author, the repository is still active on GitHub and there are active issues.

[1] https://github.com/jonschlinkert/is-number

Number.isFinite() does very well in most cases

that is odd

The guy actually had a piece defending his practice making it to HN a few years back. It was not received well.

Is that the same guy who was talking about how much work it is maintaining 200 repos?

Yes. I actually don't mind someone doing this for fun, but to actually defend his actions and claim it's useful contribution is just straight up BS.

I wouldn't want to work with this person.

I would eagerly wait his PRs and annotations though

Would be a lot less work if he didn't do this package splitting nonsense

> I wonder how many jobs they’ve applied to while bragging about their millions of monthly NPM downloads.

I used to work for a company where it was ok (managers won't say anything, even if explicitly pointed at the situation) to remove company company code, create your own NPM package (on your own private NPM account) and add it as a dependency of the company codebase(s).

This and some other similar behaviours were the reason I left that company. But I'm the loser of the story here, as those guys are now working for even bigger companies with bigger salaries than me.

They must be quite the 10X programmers to be able to contribute to open source like that while still holding a full-time job. (/s)

Well... if the actual Open Source work you do is in your day job except that instead of committing to the internal codebase you commit in your own packages all day and then update the dependency in the project, I don't think you need to be 10x.

> But I'm the loser of the story here, as those guys are now working for even bigger companies with bigger salaries

Another way to look at this is that a lower-paid SWE is paying for their mental health and morale. In my books, that's a winner :)

With the assumption that those engineers whose high/er salary in big/ger companies is not achived through mastery (Netflix is certainly not going to hire the Jon Schlinkerts), they're likely going to work with, say, "moral peers", which is typically undesirable for mentally healthy people.

You must be kidding me. Color me old-fashioned, but software development for me is still a "serious" job - one that requires years of training, focus and patience to get through all the matter.

And you tell me these people don't give a single fuck?

> I wonder how many jobs they’ve applied to while bragging about their millions of monthly NPM downloads.

Do people really do this? Does this explain the dumpster fire ecosystem of NPM?

It is perfectly normal to see "N packages on NPM with N downloads" in resumés, and often a good sign.

...but interviewers often assume that they are all useful packages and that the numbers are not inflated.

Which is something that can be fixed instantly: simply stop showing download counts on NPM.

The author is a staunch anti-vaxer in addition to an open source huckster which seems to line up the fallacies of his “contributing to open source” with his willingness to spread lies about COVID and the vaccine.

Look at the guys Twitter: https://mobile.twitter.com/jonschlinkert

Code smell is true lol.

> I wonder how many jobs they’ve applied to while bragging about their millions of monthly NPM downloads.

The sad thing about this is that people will buy it too

What benefits do those authors accrue other just an increased download counter?

Patreon and Github Sponsorships are popular ones. But also being able to add to their Resumé, being popular and leveraging it for other activities like talks and interviews, etc.

I think you have the sell-side correct there.

And stretching the idea, people who download is-even for serious purposes are probably amongst the hopeless naive/inexperienced ones

I think the horseshoe is almost coming back around to the point where people criticizing NFTs are bringing them up more often and in more annoyingly-out-of-context ways than the crypto people themselves...

That's awesome. I would hire them on just the creativity, and they also understand the js package ecosystem as well.

That's great hire them, then it will have served it's purpose. They can repackage all their trivial number related stuff up into one generally useful module, deprecate the old trivial ones and everybody can get back to work (with a smaller number of dependencies).

That's one way to look at it, the other is that they will game whatever incentives you put in place at your company to your detriment.

My friend, you just described the strategy of almost every modern American corporation, except you forgot stock buy backs.

I mean the first guy who did it on the creativity, maybe the first 10, but nowadays that just means you can't think of anything else to boost your profile.

>> and they also understand the js package ecosystem as well.

You mean how broken it is, and how to take advantage of it? Depending on your business they could fit in really well.

Is this "regulatory arbitrage?"

Reply on news.ycombinator.com