Disabling npm's progress bar yields a 2x npm install speed improvement | Hacker News Reader

Disabling npm's progress bar yields a 2x npm install speed improvement

twitter.com

544 pointsishtu174 comments

Doing GUI calls is a notorious way to slow down apps. I sped up an app (major software from major large corporation) by a factor of 10x a few years back by removing GUI updates. What you have to do is create a thread that refreshes the GUI periodically and not let your main worker thread ever work on the GUI. This particular app was an Eclipse Plugin (a source control app) and the Eclipse SWT to update the console log window was being called after every file operation (of which there were 10s of thousands). All I did was move the GUI refreshing into it's own thread, and call it only every 250millis or so. I was hailed a hero, and deservedly so, for speeding up our app by 10X and was one of my most satisfying accomplishments in my software career.

When I gave the Nexus Mod Manager a try a little while ago, I wondered how it managed to take 18 hours to copy a few hundred files from one directory to another - on a SSD. Wouldn't be surprised if this was at least part of it.

Nexus Mod Manager is impressively bad. The interface is insanely laggy; select a mod, wait a second, list starts to refresh, wait a second, list finishes refreshing, wait a second, the mod info shows up. It's really easy to do things by mistake because of the lagginess (and probably just generally poor quality of programming). It has multithreaded downloads, but even these are messed up. Every time I tried them about 50% of the downloads got corrupted.

This strategy is also encouraged by best iOS development practices (main thread never works on GUI).

Edit: 'main' as in the one doing the one doing the work, in the same spirit of the original post. In truth, the true main thread that runs in main() is where all the iOS GUI work happens. Apologies for the confusing wording.

This is backward. Since UIKit is single threaded, the main thread must work on GUI: the best practice is that nothing except GUI work should happen there.

You're not being charitable enough to the comment... you two are in completely agreement with one another if you only changed the meaning of the word "main". It's clear the grandparent meant "the thread that's doing the work."

It's really difficult to get to that meaning, because "main thread" is what you call the thread where main() is invoked and which is where all the GUI calls must happen. You can't use a common term in context to mean something different from its common meaning and expect people to follow.

And yet it was perfectly obvious what was meant. If you just assume the person who made the comment is not stupid/rambling (difficult concept, I know...) it is not difficult to get the meaning at all.

I think it's just a "my environment" bias thing going. I don't work on iOS and I immediately understood what was meant by the comment. Of course, the "my environment" bias may also include a degree of the "other" is not as bright as me.

You say it was perfectly obvious, I say it's almost impossible for someone steeped in iOS development to understand any way other than the way I said. Even given a huge amount of charity to the poster, the best I can do is "I have no idea what they mean by this."

How come we have to give this person the benefit of the doubt to the point of understanding a term to mean its exact opposite, but you don't have to give us any of that?

Thank you for this. I see now how my wording was unclear, but this is exactly what I had meant. Sorry for the confusing word choice.

except on iOS GUI thread kinda has "main" attached to it, main_queue

Changing a fundamentally well-defined term (main thread, i.e. the one on which main() is called which is running the core event loop for the application process) to be an incorrect opposite is rather fundamental when one is explaining an iOS best practice.

This is not only completely wrong, it's dangerously wrong. It is in fact wildly unsafe to touch the GUI from anything except the main thread. It's all the non-UI stuff that you should shove onto another thread.

Depends on what you call "main thread" - maybe he's saying worker thread that's doing the "main code" shouldn't be doing GUI

In other languages / frameworks, "main thread" may have flexible semantics, but in the land of iOS, UIKit defines the term "main thread" to have one specific meaning.

So... GUI work goes on its own thread, however that works out

Nope, the term "main thread" is very explicitly defined by both iOS and OS X. Not only in all the frameworks (e.g. NSThread, NSRunLoop, etc. all explicitly define "main thread"), but even at the system level. OS X and iOS have a pthread function `int pthread_main_np(void);` that returns non-zero if the current thread is the main thread (meaning the initial thread that the application was launched with), and that is the thread that all UIKit (iOS UI) and AppKit (OS X UI) code must be running on.

Not just iOS. Pretty much all UI frameworks frown upon doing heavy work on the main thread.

Same was true of Java. Sun called it the "AWT event dispatch thread." Any process that does computation or blocks on IO should run in a separate thread and periodically post updates to the UI via an event queue.

https://docs.oracle.com/javase/7/docs/api/java/awt/EventQueu...

Years ago had a similar albeit smaller discovery. Loading a flash application with a progress bar and setting the quality option to the lowest quality made the bar go faster.

I remember having a few tricks like that–I vaguely remember moving bits of the GUI for some apps off screen so they wouldn't get drawn and thus run faster.

> What you have to do is create a thread [...]

A thread? But that'll never scale!

What do you want? a whole new process? That doesn't scale either. Come on.

What we need is to put this into a Docker container then parallelize the cloud instances using asynchronous message brokers enterprise

Enterprise-grade GUI

you dont need a cluster to display GUIs.

are you sure? it might be a really big screen!

You should think big. The cloud is the future of GUIs.

It's Eclipse. It'll never scale regardless.

> I sped up an app by a factor of 10x

Woah

> This particular app was an Eclipse Plugin

Ah ha ok so it just runs fairly slow now right? :)

Doing GUI calls is a notorious way to slow down apps.

Seriously. One notable Smalltalker sped up the compiler in VisualWorks this way. Yes, there is a compiler in Smalltalk, it just usually only compiles one method at a time, which had a lot to do with why no one noticed that printing notifications to the transcript took up most of the time. So for a bulk code load, turning off the notification sped up "compile time" a whole lot. In fact, the JIT-ed Smalltalk in VisualWorks actually outperformed the C/YACC implemented compiler in a different Smalltalk.

Someone needs to keep track of this stuff. Why do we keep making the same mistakes over the decades? (Insert famous Alan Kay quote here.)

Progress bar or not, npm has gotten extremely slow with v3, to the extent that someone is working on a performance-centered alternative called ied:

http://gugel.io/ied/

installs are literally the #1 thing npm needs to do well, and I really hope we see some improvement soon.

Awesome tool, but wish the url was a bit easier to spell/remember!

just gugel it

Heh, just "ied" will give you this: https://en.wikipedia.org/wiki/Improvised_explosive_device

"ied package manager" works, though.

Normally, I am not a fan of replacing words in the search engine with computer science stuff (think AJAX), but I can accept computer words replacing words that symbolize violence.

Your results, even when logged out, are heavily biased towards your previous query genres. For instance, I'm not logged in, but am at home with an IP address that hasn't changed in over a year despite being leased. When I search Google for the term "Ruby", the entire first page is for the Ruby programing language.

My father works with jewelry. Last week while visiting him, I did in fact search for the term "Ruby" on his computer in his normal browser, and even though he was also logged out, I wasn't able to find one result dealing with the programming language on the first two pages.

Google understands that Ruby can mean different things to different people, although I don't think it's gotten there with "ied" yet (and rightfully so, it's probably not searched for very often, except in the violent sense)

That sounds like just begging to be put on a list, though :)

I'm not sure "IED" is the best acronym to go for, either: https://en.wikipedia.org/wiki/Improvised_explosive_device

Didn't stop NUKE, PHP Nuke, DotNetNuke, etc.

That brings me way back. Any love for PostNuke out there anymore?

Since we're going back in time, let's not forget WWIV and WWIVnet.

"Free software projects with terrible names" is a long, long rabbit hole.

It gets brought up on a regular basis. Thought it was going to be renamed to `nom` but apparently not.

https://github.com/alexanderGugel/ied/pull/45#issuecomment-1...

> Thanks, but I'm not going to rename this project anytime soon.

That's unfortunate. Maybe sentiment on HN will finally get him to come to his senses.

But it's the bomb.

Even more important, npm doesn't reliably reproduce builds. Different runs of the same shrinkwrapped build could either fail or succeed -- which is odd given that builds should be deterministic. Additionally, shrinkwrap is also broken in the same way.

This is the first time I've heard of ied, a little competition could go a long way!

Could you expand on that? What causes it to be nondeterministic? I haven't had that experience but it's the second time I've read someone reference this behavior

I have a project that's shrinkwrapped. I check out a new copy of the project and run `npm install`. My co-worker does the exact same thing, his fails, yet mine successfully builds. An install of a shrink-wrapped should be deterinistic, i.e. either succeed or fail, but not both.

From that doc page it sounds like that should be deterministic? Even without shrinkwrapping, a fresh install from package.json with empty node_modules should be deterministic.

> The npm install command, when used exclusively to install packages from a package.json, will always produce the same tree. This is because install order from a package.json is always alphabetical. Same install order means that you will get the same tree.

> You can reliably get the same dependency tree by removing your node_modules directory and running npm install whenever you make a change to your package.json.

Maybe you're experiencing a bug, rather than some in-grained non-determinism in npm?

My understanding is that even when you fix your dependencies to exact versions, your dependencies probably haven't so without shrink-wrap you'll never know _exactly_ what gets installed.

What's the error on the build that fails?

Here are details from the docs: https://docs.npmjs.com/how-npm-works/npm3-nondet

> You can reliably get the same dependency tree by removing your node_modules directory and running npm install whenever you make a change to your package.json.

https://docs.npmjs.com/how-npm-works/npm3-nondet

That explains why I haven't experienced it -- our projects' npm scripts to build packages wipe out that folder before building.

That's frustrating, I got bitten by that a couple of times. I'm wondering when will they add a LOCKFILE like every other build system out there :'(

I agree, but I'm curious what other build systems have a lock file?

Ruby does through bundler, who else does this right? I can't think of any.

Mix, Erlang's package manager (used by Phoenix / Elixir)

Cargo, Rust's package manager

Meteor

> Mix, Erlang's package manager (used by Phoenix / Elixir)

Slight nitpick: Mix is part of Elixir, I've never seen a (non-elixir) erlang project that use it. Don't think erlang has an equivalent officially blessed tool, but the most popular one is rebar / rebar3.

(While I'm nitpicking: mix and rebar are more build & dependency managers; mix uses hex for package management. I believe rebar3 can also use hex. In ruby terms, mix ~ bundler (among other things); hex ~ rubygems).

Composer (PHP) too.

These come to mind: rebar3 (Erlang), mix (Elixir), composer (PHP), elm-package (via elm-stuff/exact-dependencies.json).

You get a similar effect for free with any language whose community defaults to non-floating transitive dependencies, e.g most JVM languages.

Indeed; a lockfile is just a workaround for the problem of "just give me whatever" dependency declarations. Solve the problem at the root rather than piling hacks on it.

I'm not 100% sure from your comment, but are you suggesting that by default you should vet every version exactly and freeze it at the point you defined it?

As someone writing a lot of ruby, "give me whatever" is analogous to "give me the latest unless I specify otherwise", which I consider to be a very good default. It keeps me up to date with security issues, and incompatibilities between libraries that the respective maintainers resolve amongst themselves with a minimum of manual work.

> are you suggesting that by default you should vet every version exactly and freeze it at the point you defined it?

Yes, this is the way that Guix and Leiningen and rebar3 and a bunch of other things work, and it is wonderful.

Pulling in new code without you asking is a fine idea for something like apt-get where you have a huge team doing QA on the entire system working together before it even hits your repositories, but for most package managers, the dev team is the one doing the testing, and upgrades should be done only with great care.

It does mean you have to watch for security updates, but this is true of all package managers.

If you want to do this in ruby you can just specify a version manually. If you don't, you still get versions frozen by default with Gemfile.lock. It doesn't pull in anything automatically—by default you get no updates, you can choose to update a single dependency, or the whole thing if you want to verify it works on the latest (useful for libraries for instance). I'm not sure I see the downside.

FWIW I have been doing ruby for over a decade now, and I hold up Bundler as one of the great success stories of open source, and it is one of the reasons I hold Yehuda Katz in high regard, in that he was able to solve a really big problem in the community and hammer it into shape aggressively over a period of two years with a lot of doubters and naysayers (even Rubygems core was against Bundler for a long time), until it finally got so solid for so many use cases (libraries vs apps, private vs public, development vs deployment, etc, etc) where it solved nearly everyone's problems in such a solid way that everyone adopted it.

> I hold up Bundler as one of the great success stories of open source

It's a great success in many ways, but the problem it solves is completely self-inflicted by rubygems. I've also been doing Ruby for over a decade, but I've also learned a lot from other library ecosystems, and I feel pretty confident saying that disallowing version ranges makes all those headaches completely evaporate.

I'm with you that version ranges cause problems, but my belief is that lockfiles are a better solution.

Generally I'd use semver ranges in libraries, and then fixed versions + lockfiles for transitive deps in applications.

I suppose this is roughly equivalent to doing `:pedantic :abort` in leiningen, except you wont't have as many warning to squash - either way you have to rely on the test suite to tell you if the versions you've pegged work.

I've also been on that journey and am starting to feel the same way about variance.

> Yes, this is the way that Guix and Leiningen and rebar3 and a bunch of other things work, and it is wonderful.

Even in Nix/Guix, it's still ideal for upstream projects to express their dependencies in terms of ranges (semver-wise), otherwise we run into the problem have either really large run-time dependency closures, or problems around e.g. wanting to use multiple (overly specified) versions of C libs within the same process.

As the current maintainer of Nixpkgs' Bundler-based build infrastructure, I've found the lockfile approach that Bundler uses to be quite frustrating - in part because Bundler's design is antithetical to packaging, but also due to the build times and sizes of the resulting packages, compared to C libraries. (People give C a hard time wrt productivity and security and such, but when it comes to packaging, C libs are usually so much easier to work with than most other higher level languages.)

I would love to see more adoption of semver, and possibly Haskell's PVP (https://wiki.haskell.org/Package_versioning_policy). Granted, dynamic programming languages don't have the benefit of making API breakage obvious at build time, so perhaps the best we can do in such cases -- if we want any certainty that packaged applications will actually function correctly -- is lock down every dependency version precisely per application...

We use Ruby extensively in testing and infrastructure and have lost months of cumulative developer time to this attitude. It works for a single user constantly making changes to a small piece of code and prepared to work out the solution to dependency change problems as soon as they come up. However, it doesn't scale to a team of people working on very large code bases.

One example of this is having a repeatable developer setup guide. If the dependencies might have changed by the time a new joiner starts your setup guide could very easily end up useless or misleading and that's without anything at all in your own codebase having changed.

Shrinkwrap fixes this, but always gets added after things went wrong several times already. It should be the default.

I've spent enough time in my life fighting with version mis-matches of slf4j. Hard defining your dependency versions just pushes the problem upstream.

Cabal (Haskell) too.

I try and push devs to do

    pip freeze > requirements.txt

Which has a similar effect.

This is a good start, but it has issues. You probably don't want all of your locally installed packages in a requirements.txt file. Instead, they should be curated. I've been using pip-compile [0] for a while now (see the author's blog post [1] for a detailed explanation) and have become a big fan of it. With this model, you should enumerate only what your application uses directly and let pip-compile convert this list to a full version-locked requirements.txt. (Shameless plug: I also wrote a bit about why this is the best currently available option for specifying Python dependencies [2].)

[0] https://github.com/nvie/pip-tools#example-usage-for-pip-comp...

[1] http://nvie.com/posts/better-package-management/

[2] https://www.jonafato.com/2015/12/15/Rethinking-requirements-...

If one uses virtualenv as a Pythonista should, then your keep point doesn't carry much weight.

That said, this looks cool and I'll check it out.

This should be used with virtualenv, not as an alternative to it. I have a bunch of libraries and tools in almost all of my virtual environments that are not application dependencies and have no business being installed in my production environment (e.g. bpython, tox, flake8, and neovim). This approach also handles things like a dependency being dropped gracefully (if some library no longer needs a dependency, it magically gets removed from your locked requirements.txt next time you compile requirements.in). Python's package management tools are making great strides (see pip's recent peep-style hash checking support), and pip-compile is a big win in that category in my opinion.

And it's so awesome when you discover devs have used lockfiles to leave you stuck on known-insecure gems instead of updating their fucking code to support the newer patched version.

Which is still better than having your server crash due to a bug in a dependency of a dependency that got updated without your knowledge when you deployed something trivial. It has happened to be more than once.

PHP have a package manager called Composer which have a lock file.

That's what npm-shrinkwrap.json is supposed to be. It's given me a lot of trouble though.

same issue here. this non-deterministic behavior is really fxxked up. there's one time that our building process suddenly begin to fail, spend a few hours on the issue, and it turned out to be one of the babel-core patch release is broken.

some of the very fundamental designs of npm is seriously wrong.

Speed and support for multiple module development that have cross dependencies. Npm link sucks and updating and publishing downstream dependencies is a chore I wouldn't wish on my worst enemy.

There is also JSPM and Systemjs - works great in production.

https://github.com/jspm/jspm-cli/issues/675

Here's a breakdown https://github.com/npm/npm/issues/11283#issuecomment-1752468...

I guess we're supposed to assume that V8 is smart enough to inline that sort of function definition, but it seems deeply weird to me that drawBar() is defined inside the "hot" method like that. Also, I would have expected that streams would have been used for this. Logging is a very common "simple example" on node streams how-to pages.

I'd also like to take the time to mention that the caching system when installing / other actions is extremely inefficient (for my times I have the progress bar turned off already and this is a project with around 80 modules shared between dev / prod):

https://github.com/npm/npm/issues/10890

I started on a very (I would like to emphasize very a thousand times over) basic proof-of-concept to show how much faster it could be in the order of magnitudes:

https://gist.github.com/nijikokun/2f1f16325f8ffe14b1b3

All this does is build a json of every package you currently have installed, and utilizes that as a lookup store the next time instead of rebuilding it every install; this was targeted towards installing / uninstalling existing packages. Not fresh installs.

Fresh installs would benefit from bulk lookups via the API imo.

Looks good. I wonder if it'd be too much scope creep to add extension points for LAN/proxy caching. I know of one small dev team who had a single ADSL connection shared between ~10 people - NPM downloads would have been painful.

I made an npm package that does this https://github.com/vvo/npm-pkgr

Some further details on the issue: https://github.com/npm/npm/issues/11283

Please refrain from adding unhelpful comments to the issue.

It is tremendously unfortunate that that needs to be said at all.

I guess it's GitHub's Eternal September. I'm all for more people having access to development tools, but GitHub really has to step up the tools available for managing issues/comments.

There is this open letter to github from open source maintainers: https://github.com/dear-github/dear-github

I think bug tracker comment forms for open source projects need some kind of question first (at least from people who aren't active contributors), like "Please select your type of comment":

* Nontechnical: I would like to express my eagerness that the bug be fixed or request its priority be increased, but don't have any additional technical information to add

* Technical: I have useful technical information to add, beyond what is already in the bug

Then "Nontechnical" type comments would be hidden from the developers by default (unless they specifically ask to see them), but their count could be used as a bug prioritisation mechanism (effectively leaving a "Nontechnical" type comment would be counted the same as a vote)

How is this unhelpful exactly? It's a link to another discussion that readers might be interested in.

Op was stating you shouldn't follow that link and go "Hey, me too" or "+1" on the Github Issue.

Not that the link to the issue was unhelpful.

GitHub badly needs an "approve" system that is external to comments, like bitbucket has.

Because I can't amend this comment:

I misunderstood the commenter above me. My sincerest apologies. Have a nice day!

Now that's what I call a bug report: https://github.com/npm/npm/issues/11283#issuecomment-1752468...

That's what I call a slideshow.

It's all good, Laurie fixed it - https://twitter.com/seldo/status/692192238445711360

Link to relevant commit?

https://github.com/iarna/gauge/commit/a7ab9c906bb72aa0ed8996...

Two years ago, PHP's package manager, Composer, was sped up by disabling the garbage collector:

http://blog.ircmaxell.com/2014/12/what-about-garbage.html

(Or more accurately the cycle collector. You can't turn off reference counting.)

I posted this below, but I'll post again here for visibility. What the heck is up with npm install on iTerm 2? I had no idea something was weird until recently, when I did an install with Termial.

Here's a side-by-side video: https://youtu.be/odoVfHHBYVM

Weird I'm seeing the left result in iterm 2 on my work mac

Check Syrup-tan's comment above for an explanation.

I had the same problem (and figured it was normal for a while), after some looking-into it I was able to fix it by unchecking "Treat ambiguous-width characters as double width" [0]

[0] https://denpa.moe/~syrup/Screen%20Shot%202016-01-26%20at%206...

That did the trick, thanks!

The funky progress bar is about the coolest thing with npm. They could let it take a minute if they wanted too and i would keep watching it with amazement.

npm 2.x user here. Does anybody want to share a video of the 3.x progress bar? (Easy with QuickTime or CloudApp, btw)

Seems like a perfect task for asciinema

https://asciinema.org/

https://www.youtube.com/watch?v=elWIVKWhWTk

This seems to make it go faster:

http://bennyhillthis.com/?v=elWIVKWhWTk

I see that with iTerm 2, but using Terminal gives me a very different experience.

Here's a side-by-side: https://youtu.be/odoVfHHBYVM

Until recently, I had no clue iTerm was doing something weird. I just figured npm install was just ugly and incomprehensible.

heh

My iTerm2 has the same behavior as your Terminal.app, so you might just have to change some settings in iTerm.

https://asciinema.org/a/9gt0i1kxbto41gj4796tjkdnt

Holy crap that was fast.

What's the color scheme you're using for terminal? I'm also rocking agnoster.

you mean the progress bar that takes up way too much screen real estate to the extent that it completely cuts off any useful information from the install process?

Yes, that one. I agree with you, by the way. I don't have a terminal in front of me, but the progress bar seems to leave something like 10 characters of less of rapidly-changing status updates in exchange for the vast majority of horizontal real estate. I bet if they had split the bar onto a separate line from the status, it would run faster

I work in China. Downloading from the outer Internet will never match the speed of writing to the console. Not too worried

Not to mention that when the binary you are downloading happens match particular keywords.

Oh, that's fine. You can just download it from one of those mirror sites that host their own copies of the files. You can trust that it's genuine because of that big green checkmark on the page.

You just have to wait for a security worker to manually edit them out

Could it be linked with the terminal being used?

Edit, reminds me of this, which is why I'm wondering http://stackoverflow.com/q/21947452/923847

This makes sense. It's like when I'm solving a project euler problem, I don't use verbose output because it runs much faster without having to print everything to the screen. (Python)

That's exactly my experience. Once I wrote some naive PE solution in nodejs and added some logging at each step. After few minutes and far from a solution I stopped it and disabled logging. I got the solution within seconds. The diff was few orders of magnitude, which was pretty surprising for me.

Sometimes writing to the terminal can be the slowest part because it's unbuffered by default in an interactive prompt, so even just redirecting stdout to /dev/null (or even a file) can be faster. Had that happen some ten years ago, so my explanation could be out of date or misremembered but the effect was real.

Your explanation is correct. That is the semantics of Unix stdio buffering as implemented by glibc. So across a wide variety of languages, you'll see a significant performance difference if you are doing lots of writes to the terminal.

On the other hand if you didn't do this, then interactive terminal programs would be entirely unusable.

I don't know the exact cause here, but I can say I'm thankful for my systems background that I've gleamed for years before moving into programming. Most sysadmin types know that displaying output on a filecopy slows down transfers greatly because the screen vsync holds up the operation.

When I write utilities today that do similar things, I only display output when writing and testing the program. Otherwise I'll write to memory then dump the results in a logfile at the end.

In this case since there is probably a less intensive way of providing status updates, and can probably be resolved by doing intermittent checks.

Someone with more UI development can comment here , but i always thought progress bars where more like a signal to the user that "something is being done" , and not a representation of how much is done and how much is left.

The whole point of a progressbar is to show work progress (as a fraction of total work). The issue is it's often difficult to map "work done" to a linear percentage, so either you do it on the cheap and your progressbar is just a souped-up spinner with no relation to wall-clock work time or you do lots of extra work to get more precise estimates but you end up taking more time overall.

I don't see this at all, what's more worrying is that npm 3.* is apparently way slower than npm 2.*

I did this test on a npm library I wrote, installs some test tools and also has to compile some gyp stuff

2.14.2

progress=false real 0m26.589s user 0m12.086s sys 0m3.362s

progress=true real 0m29.553s user 0m12.455s sys 0m3.486s

3.5.2

progress=false real 0m57.489s user 0m13.298s sys 0m3.421s

progress=true real 1m1.084s user 0m15.690s sys 0m3.644s

While I don't see any significant difference between progress bars or none for 3.x, what is 3.x doing that causes almost 30 seconds more churning?

It uses a very different dependency resolution algorithm, the biggest difference as I understand it is it will install dependencies maximally flat whereas npm 2 was much more nested.

I believe the reason it is slower is it can't do as much parallelism as npm 2. Also, npm 2 was susceptible to a lot of race conditions and non-determinism, especially with compiled dependencies.

I used to joke, in the context of virus scanning, that I could give you a more accurate progress bar, but the scan would take twice as long...

sbcl's compilation output is so verbose that the compilation instructions mention that it builds faster on xterm than gnome-terminal

Yes, but, which download feels longer?

Another product bitten by the dark side of progress indicators.

One of the most often underestimated areas of product development, though certainly not among the most important underestimates.

Easy to underestimate its cost and impact. After all, it's just a progress indicator.

...has he controlled for network speed? Unless he's set up a local NPM caching proxy, he's going over the 'net for these packages. It'll take more than 1 datum before I believe this hype.

Many people have confirmed a major speed improvement. I've repeated this with an empty cache, primed cache and in different orders many times and have always got the same results. Having said that, a good number of people have reported seeing no speed improvement too.

The network utilisation is much different between npm 2 and 3:

https://github.com/npm/npm/issues/10380

Also the second run probably used stuff that was put in the npm cache during the first run. Not sure how good a test this is.

A good read: "Programmers need to learn statistics or I will kill them all" http://zedshaw.com/archive/programmers-need-to-learn-statist...

It yields the same results in reverse order and also when the cache is primed for both and cleared for both

Screen IO is slow. A trick to speed up unzip speed, do unzip -qq. That skips all the screen output and make it flies.

That's the biggest issue I have with the new npm. I wonder how many developer-days it has wasted already.

Upon first seeing npm 3.x I immediately added `--progress false --color false` to my npm installs and never looked back. Color and terminal graphics are the work of the devil.

>Color and terminal graphics are the work of the devil.

Or you know, it's 2016, and it's npm who does it badly.

I don't even want color and terminal graphics done goodly. I code with two monochrome terminals on the screen using vim and the command line.

Not everyone is you. While I agree that flashy graphics are silly in the command line, I find color-based highlighting to be quite useful.

Thanks for sharing!

As an undergrad, I had a 14.4 kbps modem, and an amber WYSE terminal. With that, I was able to do contract work. It reminded me of the amber monitor I had on my Apple II ten years before. A solid phosphor CRT with no shadow mask is a fine thing.

Colors on a modern LCD are great too; give it a chance!

I have a soft spot for those short-lived X-terminals as well. The golden age of computing.

Colors are great for web browsing, don't get me wrong. Just not near my code!

I contracted at Nortel in the mid 1990's. In my cube I somehow acquired a Sun Sparcstation 20 that nobody was using, and found an NCD black-and-white (or was that one grayscale?) X terminal on a shelf somewhere, collecting dust. I found an image for it somewhere, and set it up to TFTP boot from the SS20 (which was running Red Hat Linux) to a login prompt (XDM). It was a nice little setup.

I bet it was. Well done.

A couple of years ago I was just about to throw out all my O'Reilly X Window/Motif books from 20 years ago and when I landed a contract to update a K&R C based Motif system running on 32-bit Solaris connected to - of course - Sybase. It felt like I travelled back in time.

Old code never dies.

I have X/Window/Motif O'Reilly's from more than 20 years ago which are still shrink wrapped together. I've just been carrying them from place to place.

Must be worth a fortune by now ...

You don't use syntax highlighting?

No, never had a need for it.

At the risk of getting downvoted myself, I wonder why all of your comments are getting downvoted. Is it such a terrible opinion that it doesn't even belong on HN? A lot of people turn off syntax highlighting. It can be legitimately distracting, and it messes up tooling.

Making vehement categorical statements about matters of taste is an old flamewar thing on programming message boards. I'm guessing the downvoters don't want that trope on HN. If so, I have to agree with them, because such discussion is basically never substantive.

I was asked a question and provided an answer. So I don't care for syntax highlighting or colors in my editor and command line - big deal. I am not imposing my view on others.

When working with a command line one wants npm or any command to run as quickly as possible. Any graphics that slow operation of the command should be an opt-in, not an opt-out.

Everyone is entitled to their opinion, but some people prefer to suppress others' opinion.

Well-designed server platform.

Don't even get me started on the number of reasons why nodejs should not be running on servers, I'll be here all day. We're currently deploying a NodeJS app and we'll never do it again - the amount of problems with that ecosystem is quite amazing. As I tweeted earlier today "JS ppl tend to have marketing & UI/UX experience so JS crapcode gets popularised, becomes widespread, then we have to deal with the fallout".

I'm genuinely interested. I've been using nodejs for hobby projects and scripts, so haven't seen too many issues. But I'd like to know how things are and what issues you've been seeing.

Howdy, we've had lots of issues with single threaded performance, poor memory management especially around garbage collection, we had to modify our SELinux policies as it likes to execute memory off the stack, npm has been very unreliable and often unreasonably slow, error handling of the node index app server seems to have some interesting behaviour but I haven't dug into that one too deeply yet and the list goes on. As an experienced systems engineer it's very clear that this is browser technology that was never designed not fit to run on servers.

Reply on news.ycombinator.com