How to write a command line application in Node.js | Hacker News Reader

How to write a command line application in Node.js

blog.liangzan.net

76 pointsdpmehta0250 comments

The number one reason to write CLI apps in node is npm. npm gets package management right, preferring local dependencies to global ones - this means no worrying about what version of a library a user has in their global environment.

You also get to bring node's parallel io-centric patterns to your scripting. Need to download a bunch of files from a remote host, process them, and write them to disk? Go for it.

But take it with a grain of salt: it's very much a use-what-you-know kind of thing.

Need to download a bunch of files from a remote host, process them, and write them to disk?

    cat urls.txt | while read -r url; do 
      base="$(basename "$url")"
      wget "$url" -O - -o "$base.log" | process > "$base" &
    done
    wait

For ad-hoc scripting, I tend to prefer bash, with anything remotely corresponding to heavy lifting assigned to the language with the best library / performance / whatever critical factor for it. Fork-join parallelism in bash is very easy.

I thought a lot of people discourage the use of static linking or any form of local dependency bundling, because when there's a security update every app needs to be updated individually. But it seems that with the emergence of Bundler and NPM, people are trending more and more towards local bundling. What happened?

One possible contrarian view: global dependencies actually cause a lot of those security updates to be necessary. Exploits that target a given version of a given library may be easier to propagate if you know that almost every application run by every user depends on that library version.

alternatively: maven managed to get dependency management right about 5 years ago.

Hahaha really? Have you seen a pom file on a real project its about 10 pages with plugins galore.

The verbosity of the XML pom is the problem. Alternative syntax using the same coordinate system are great and much easier on the eyes. The real value in maven (I think) is having that clean dependency chain and it works great. XML is what sucks (especially doing it by hand).

Yeah, pom files are disgusting, but leiningen (the clojure build tool) is built on top of maven and it's wonderful.

    [enlive "1.1.1"]

instead of

    <dependency>
      <groupId>enlive</groupId>
      <artifactId>enlive</artifactId>
      <version>1.1.1</version>
    </dependency>

really. if a page is about 60-80 lines, that's what, 1000 lines? and most IDE's help manage them. The makefile's to some open source projects get longer than that, and they only get worse when they're proprietary.

I didn't know that the bar is makefiles... welcome to 1977.

Is there a reason to build a command line application in Node.js? It doesn't really seem like Node is the right tool for the job...

It's fun, simple, fast, and has a good package manager. JS is a high level scripting language like python or ruby but gives significant speed benefits in a lot of cases (in particular, the startup time of CLI programs written in Ruby always bothers me).

If you want to write in C for performance reasons, go ahead. But if your other alternatives are Python or Ruby and you already know JS, give node a go -- it's pretty fun.

This is what happens when you let front-end developers loose in your infrastructure.

Don't knock it 'till you try it. Most developers would balk at the idea of creating a project in something that isn't their favorite language, especially when they've been burned by poor uses of that language before.

I have written plenty of JS, and absolutely hate it.

2 plus years ago there may have been merit to your implicit assumption that JavaScript is limited to front-end developers.

However, today there is more diversity in the JavaScript developer community. For example, Yehuda Katz is both a server-side (Rails) team member and client-side (JQuery, Ember) developer. Also, system engineers have begun working in JS. The Meteor team is a good example of system engineers working on JS.

Well, there's my daily laugh :)

If you're doing crazy heavy processing or something, it's not the right tool. But you wouldn't do that in JS. You'd shell out or call something native.

Node is best used managing parallel streams of stuff. Perhaps surprisingly, a command line application is a great example of that.

I would much rather build a nontrivial command line app in node than in anything else. Try it, you'll see.

There are tons of better options to choose from. If you're not making a web UI, literally any choice is better than Javascript, unless that is the only language you know.

Compared to most languages you'd use for a commandline application, node has a simple, usable async I/O interface, and the libraries you can get for it actually assume async.

If you need that, there are only a handful of choices and node is one of them.

Which scripting language doesn't have an async library available (most of which don't descend into the Chinese Hell of Being Called Back Ad Infinitum)? Granted, most of the other languages have lots of libraries that don't play well with async, but then again, that's mostly because they have lots of libraries. And it's not like there's a guarantee that a random Node package won't screw you up, either.

I'm not really arguing against using JavaScript for command-line interfaces. I'd much rather see that than the same in PHP, and if the programmers don't know Perl, Python or Ruby, then why not? We're talking about an area where there are still way to many hacks that consist of badly written bash scripts with lots of inline sed and awk...

concurrency != async

Brainfuck, then :).

People don't use node for the Javascript. They use it for the modules, and node is the glue that coheres them (and does an admirable job at that).

When you're comparing writing a CLI app in node to writing it in something else, it's a world of difference, and that difference is npm. With good modules, writing a CLI app has little to do with Javascript -- you're more specifying config and flow by passing some options and streams around, and you're done.

How about writing it in Perl and using CPAN instead of npm then? There are tons of modules in CPAN. But since Perl is not cool any more, there's always Python and pip.

I have yet to see any language repository that constrains quality in the way CPAN does with CPAN testers.

CPAN modules must provide POD documentation, must provide a regression test, and must run on several dozen platforms before PAUSE accepts to publish it on CPAN.

Python or Ruby modules, or my favorite Lua, are a bunch of junk compared to CPAN.

Yes.

If a person already knows JavaScript well, then it will be easier for them to write a CLI app/utility/script in JS than it would be to write a shell script.

Another reason to write in JS would be to gain code reuse. If the person has a web app that uses node, then they can share code between the web and CLI applications.

  > Another reason to write in JS would be to gain code reuse.
  > If the person has a web app that uses node, then they can
  > share code between the web and CLI applications.

I often see this argument presented for just about any server side js. I often wonder about actual real world reuse though.

I actually reuse the same code a lot, which is a very nice feature once you get used to it.

I share code between the client and server. For example, I have data type checks where I reuse the same library and the same application specific code on both the client and server.

I expose substantial portions of my server-side code via the command line. I also frequently call shell scripts from node as there are certain things I find easier to write in a shell script.

That said, I don't know if this amount of reuse is typical. And, code reuse is clearly not the only consideration.

Look at the top 2 depended on modules modules in the entire of the server side node.js ecosystem and guess what they were written for - https://npmjs.org/

I wrote a database specifically for web browsers and someone case along and write a relatively small pull request and it now works in node / on the server / with a http api - http://pouchdb.com

Depends on the job. I'm writing a command line app in Node right now to automate a couple thousand SQL queries for a db I'm working with. Node wasn't my first choice, but I looked at how to connect both Ruby and Python to MySQL and it seemed like more trouble than it was worth for a one-time use utility. Node was just 'npm install mysql'. Might not be the best way, but I only need to use it once.

Isn't Ruby just 'gem install mysql'?

I don't recall the instructions I looked at, but I know what I read was much more than a gem install. Correct me if I'm wrong, I actually wanted to use Ruby first.

Sorry if this is off topic, but I remember having this exact issue when I was learning Ruby, maybe I can provide some clarity.

For raw access to your MySQL database, the gem 'mysql2' is preferred (github with a simple tutorial: https://github.com/brianmario/mysql2).

mysql2 seems to have taken it's interface design from Perl's DBI, which is cool if you're used to that but might feel a little foreign.

If you'd like something that gives you a little more help, or there's substantial business logic in your script, take a look at ActiveRecord. This StackOverflow question (http://stackoverflow.com/questions/16683903/sinatra-mysql-an...) has a nice example of how you might set up your script to use AR.

My email's in my profile if you'd like any future help.

Thanks. Appreciate the info.

What is the memory footprint of node? What is the startup cost?

imho, node as command line makes even less sense then Smalltalk, Java or Mono. Those are heavy weight languages that have no place in interactive commandline scripting.

Better choices are the classical languages like Perl or Python, and Lua or awk has even less startup costs.

> What is the memory footprint of node? What is the startup cost?

It's fast enough, and low-overhead enough, that I don't notice any overhead when running my Node.js build scripts. And really, that's all that matters.

Out of curiosity, I wrote a little script to test it:

    #!/usr/local/bin/node
    console.log(process.memoryUsage());

And here's the output (memory usage is in bytes):

    jshore$ time ./deleteme.js 
    { rss: 12660736, heapTotal: 4083456, heapUsed: 2131864 }

    real   0m0.055s
    user   0m0.043s
    sys    0m0.011s

I don't think it has anything to do with the interface. Is your program mainly about handling input and output, tied together with some business logic? If yes, Node.js is probably a good choice. If not, Node.js might not be a very good choice.

I was thinking the same, does nodejs even has a complete linux api interface like python or ruby?

EDIT: Ok, just went through the nodejs api, they have cover a lot of ground now :)

This was also recently released. https://github.com/chjj/blessed

Definitely check out Inquirer.js if you're using prompts in a Node CLI application. It's the prompt utility used by Yeoman generators:

https://github.com/SBoudrias/Inquirer.js

Callbacks and CLI apps aren't a good fit in my opinion. I do like writing command line tools using Common Node though: https://github.com/olegp/common-node

Or you could just use http://silkjs.net

Something similar but published just last week: http://flippinawesome.org/2013/07/29/writing-a-command-line-...

I find the lack of a synchronous way to pass a command to the shell and get the result makes node much more difficult for the kinds of things I usual write in ruby.

in this day and age, if i had to create a command line app, from zero, i would definetly start with go.. it has all the functionality in standard libs and the package manager can handle github repos for third-parties (like mongo or mysql drivers).. you just compile it and go!

can be as simple than that? the right tool for the job..

if i would create a library that everyone could use and embed i would use C for that..

if i would like to build a VM, a compiler, a browser or a OS i would use c++ (is almost c, fast and gives you easy composability and control)

i would use javascript.. but only in the browser and because and dont have nowhere to go or to runaway for.. (no transpiling either please)

this is just the general experience of my years of coding and deling with every lang in the earth.. rsrs

ruby is pretty, eyecandy, bu is like a virgen that cant be touched.. python looks just like perl.. is a complete mess, and is like the mafia,i prefer not to touch or to have any involvement with them.. java is the frankenstein child of c++.. you can create big software with it.. but to many keywords and excessive use of vtables..

and for ui and web, network, etc.. i would use something like dart for this job, any day.. the code can grow and you can still understand your code

so these days i would use c, c++, go or dart.. in the case i can choose of course.. :)

In this day and age, drumming off single-sentence dismissals of major languages does not lend one much credibility.

The diversity of programming languages is due in part to the fact that developers enjoy looking at old problems from new angles. Doing so lets us to make creative re-use of libraries and practices that were invented to solve disparate problems, and good things invariably result.

Node.js is a staggering example, but every language you mentioned has its strengths.

im completelly aware that this is very subjective matter, and by saying touching language options, a lot of people would get offended..

experimentation is good! it make us have more of the good ideas.. sure! go and create a command line application in node.. nobody will forbid you to do that, its not a sin..

i would do it.. but theres a difference between whats fun and what works.. one thing i take my hat of for all dinamic languages like ruby, javascript and even python, is that they want to make the experience of creating software fun, and more pleaseant.. this is cool

but the answer its in the balance, thats something i respect about go and dart.. the idea to mix fun and pragmatism so we can have both.. maybe better things can come in the future.. and we should all thank dinamic language for that..

javascript did not achieve the nirvana yet.. its like a duck.. it can walk? sure. it can fly? sure. it can swim? yes.. but can do it good any of those things? nop

> the package manager can handle github repos

A package manager with no concept of package versioning, unlike every other major languages' managers.

This is a complete deal breaker for some / many (choose as appropriate).

The idea of "Just track original master or make your own fork if you want a specific version" is insane when all the others seem to be able to do dependency resolution.

> ruby is pretty, eyecandy, bu is like a virgen that cant be touched

Which, even compared to the dismissals of the other languages, makes no sense.

Bunch of broken URLs due to closing parentheses

github.com/dpweb/cli-node

Reply on news.ycombinator.com