Challenge: Implement CLI interface like that? | Hacker News Reader

Challenge: Implement CLI interface like that?

challenge.docopt.org

74 pointshalst46 comments

Imho this is a bad pitch. It took me a moment to realize that you have already solved the problem that you're proposing (and the "challenge" is only meant to pitch the solution).

Given the elegance of that very solution I think this marketing stunt is entirely unneeded and actually harmful; it's confusing and you miss out on getting a descriptive headline such as "Generate OptParser from code comment!".

That said... Brilliant idea!

I'm definitely trying out docopt for my next project.

And I have a counter-challenge for you:

How about making docopt work with nested commands? I.e. multiple methods could have a docopt-style comment and they would smartly merge to enable usages such as "mygit remote show --help", where 'show' is a subcommand of 'remote', with its own set of parameters and a matching help-page.

If you can work that one out then docopt could become the defacto standard for generating even complex OptParsers.

You can already have deeply nested subcommands in docopt, take this example: https://github.com/docopt/docopt/blob/master/examples/git_ex...

Making possible several help-screens for different commands is on it's way. The only stopper—deciding on API: https://github.com/docopt/docopt/issues/17

Ah, that's great!

But I'd strongly vote for making it possible to annotate multiple functions and merging their respective doc-strings smartly.

I don't think you'd want to express the entire syntax-tree of a reasonably complex program (e.g. 'git') in a single doc-string. Not only would that turn into a mess, but you'd also lose modularity. I.e. subcommands often need to be dynamically generated (e.g. depending on loaded modules) and not all subcommands may be available at all times.

I'm also a fan of idea of using function-docstrings for separate subcommand help-screens. Problems: (1) exact API, (2) how to do that in non-python implementations. You (and everyone) is very welcome to suggest APIs for that in issues: https://github.com/docopt/docopt/issues

BTW, about bad pitch: maybe :-)

I wanted to improve on presentation, since my last post: http://news.ycombinator.com/submitted?id=halst

For me, your new presentation worked brilliantly. The short while it took me to get what was going on, was exactly what caused me to keep reading. Granted, it is a bit of a bait and switch, but a pleasant one.

Great idea, great presentation!

I too think the pitch is brilliant - specially when you think about the target audience - folks who get turned on by the word "Challenge", er developers, and most of them must have felt the need for something like this at some point of their life("Some day I'll have to write clean & DRY way ...").

Only thing different might be instead of giving the solution on that page itself - a hyperlink to the solution which takes you to docopt.org landing page

I agree. I thought, interesting challenge, although not interesting enough for me to actually try it. I wonder if he has any example solutions that I could look through. Huh? Ooooh.

If I had just seen it presented straight out as, "Parse arguments based on docstrings," I doubt it would have been nearly as memorable.

I think it's a great pitch, because the pseudo-pitch appeals to exactly the people who would be interested in this. "Ha! I've got this licked, I'll just whip out argparse ... Wait ... Oh!"

The docopt documentation suggests that a lone "--", signifying the end of the command-line flags, is only supported if "[--]" is added explicitly to the syntax string. In my opinion this is a mistake, and may even break POSIX XBD §12 [1]. I would seriously recommend the authors support it by default.

[1]: http://pubs.opengroup.org/onlinepubs/009604499/basedefs/xbd_...

Thanks a lot for the link. Your suggestion will probably make into next version of docopt.

> How many LOC

Exactly one. Unless, of course, it's in some funny language that mixes syntax with presentation ;)

You don't need a funny language:

    import zlib;exec zlib.decompress("x\x9cmQ\xb1n\xc20\x10\xdd\xfd\x15\xd7t\x80HuB;\xa2\x90\xa9\xea\xd6v\xa8:!\x14\xb9\xe4\x82-\x92\xb3e\x9bP$>\xbe1&\xa5Ex\xf1\xbb{\xef\x9e\xee\xe9\xee\xef \xdf9\x9b\x7f)\xca\x91z0\x07/5\xb1$I\xdeD/Zx\x11\x1e3\xc6>\x9d\xd8\xe0\x9c\x01P\xe8V\xcd\xd0\x05'\x95\x01\xc2=\x14$:,\xb3,\xbb\xc1G\x0e:\xdd#\x14\xdf%\x14\x87\x12\x96\x9c;\x83X/\x8a-\x95\xab\x1bCNj\xedG\xf9\x7f\xbeS\x840u\xe8\x8f\x16\x83i\xfa\xd7\xb5\xd3\xdab}\xe4\xbc\xb6\xaa\xf1\x8a6W\xe6\\\xc2\x118\x97\xd8\x9a+\x82\xf7h\x9d\x1a\x82\xb3w\xe3\x87\xdf\x85\xac\x83<\x8a!\xbc\x0f\xa9\xf7\xe0\xa5r\xe0\xd6\x16\x91B\xda\xdf\xb9\x8b\xe2\xdc\x88\xec%\xe6\xc0\x06\x0c\x8a`K\xda;X\xd6\xd8\x88]\xeb\xe7\xf08[Eu\\\xffd\x05\xaf\x11O\x05\xade@\xe9)y\xd4\x8d\xe9\x82\xeey\xc4\x91\x0e\x87c\x8d\xd5\x1d\xd4z\xad\x8d\x07\xd5\x19m\xfd\xb9b\xccXEc5\xad\xaa\x01T\xd5\xc3\xb8\xf3br\xb99<e\xb3I\xfa\x03y\x8e\xad\x94")

All those numbers are the encoded Python solution... Do I win? :P

Never used much of the docopt module, maybe I should spend a couple hours with it. Last time I saw it, it didn't appealed to me for some reason...

Reminds me of apenwarr's option parsing in bup.

http://apenwarr.ca/log/?m=201111

Had you seen that before?

Also, I love it; it's taking DRY to its necessary conclusion.

Quite similar approach. But I think usage-pattern matching is where docopt shines.

Ouch, and I implemented https://github.com/andreyvit/dreamopt.js only a few months ago. Similar idea, but yours is better. (Not sure how you deal with type coercion and the like, but I guess it's not too hard to figure out.)

docopt will maybe borrow a few ideas from dreamopts.js :-) Would you like to join forces on [docopt.coffee](https://github.com/docopt/docopt.coffee)?

this seems analogous to the old Getopt::Declare in perl[0], maybe you can et some ideas from that

http://search.cpan.org/~fangly/Getopt-Declare-1.14/lib/Getop...

Getopt::Euclid from CPAN is also very similar.

I've found Plac[1] to be a suitably terse and DRY method of building command-line interfaces. Basically, you write a Python function where each argument to the function becomes either an option, flag, or positional argument on the command line. It's one "line" to define the function's arguments (which might be split on multiple lines due to length, but it's just the function's argument list), plus one "line" to annotate each argument with a type, single-letter abbreviation, docstring, etc. (which might be longer than one line if the docstring is extra long). The "type" of an argument is any function that takes a string and returns anything, so you can for example have a function that parses and returns a comma-separated list of numbers as a Python list of floats. I think Plac supports subcommands, but I haven't used it that way.

If you want to see some examples of real-world usage of Plac, here are some examples from my own projects:

https://github.com/DarwinAwardWinner/intemp/blob/master/inte...

https://github.com/DarwinAwardWinner/mergesam/blob/master/me...

https://github.com/DarwinAwardWinner/splitloci/blob/master/s...

https://github.com/DarwinAwardWinner/fastqident/blob/master/...

[1] http://plac.googlecode.com/hg/doc/plac.html

Docopt looks like a useful library, but I'd be interested to see how intelligent the "help" parsing is when faced with slight variations in the format.

Using Commons-CLI in Java actually goes the opposite direction and if can be reasonably terse (one line per parameter or option) but it's also got to be wrapped in a class and have the rest of the Java boilerplate (a main method to execute, etc).

Perhaps I'll try it on my next NodeJS project.

docopt is even more terse than 1 line per argument/option/command.

Take an example:

    Usage: quick_example.py tcp <host> <port> [--timeout=<seconds>]
           quick_example.py serial <port> [--baud=9600] [--timeout=<seconds>]
           quick_example.py -h | --help | --version

2 commands, 3 arguments, 5 options: in only 3 lines of DSL.

Take a look at more examples: https://github.com/docopt/docopt/tree/master/examples

When the DSL (http://docopt.org/) was designed, the main goal was just to formalise that pattern-language used for decades in man pages and `--help` screens. So variations that are conventional (such as `UPPER-CASE` for arguments or `<angular-brackets>` for arguments) are supported.

The question, of course, is whether it will scale to more complex cases with more elaborate option documentation (i.e., multiple paragraphs, formatting, short version for usage, extended version for man/html, etc).

For C++ there is the CLI compiler that implements a similar idea (i.e., uses a DSL) but instead of using the usage itself as a specification, it is based on the class-with-members abstraction. While the result is not as terse, it is quite a bit more flexible. Plus it allows you to specify option type (i.e., int, double, string, etc).

From a single interface specification CLI will generate C++ parsers, usage printing code, man pages, and html pages. Here is an example of a real-world interface that is handled with CLI:

http://codesynthesis.com/products/odb/doc/odb.xhtml

The project page is here:

http://codesynthesis.com/projects/cli/

One nitpick:

Note that there is a conflict between the subcommands in the example:

    naval_fate ship new <name>...
    naval_fate ship <name> move <x> <y> [--speed=<kn>]
    naval_fate ship shoot <x> <y>

Users can't name their ships 'new' or 'shoot' and you can't safely add more 'ship something' commands in new versions as you risk colliding with some user's ship name.

You could try to warn of such conflicts, but that sounds like a lot of work... :)

You can just change the second line to:

    naval_fate ship move <name> <x> <y> [--speed=<kn>]

to eliminate all the problems you named. Those problems are about specific interface, not docopt.

As I said I was nitpicking. :)

Very nice! And especially clever that the DSL is the usage help we all know and love. I'm never happy about the amount of code normally required to write these.

One nitpick is that there is still some repetition: "naval_fate", and well as the long options are repeated. Granted, there might be ways around that which I haven't discovered yet.

About long options repeated: you can either specify "[options]" shortcut in a pattern in order not to put all options in that pattern, or you can have all options listed in the pattern--then you can avoid having them in option description.

About repeating program's name (say "naval_fate"), you can do:

    """Usage: prog <bla> ...
              prog <bla> --bla
    """
    args = docopt(__doc__.replace('prog', 'naval_fate'))

If you are looking for something like this for .NET, "synoptic" can generate cli-usage help in a similar format based on your methods/classes complete with defaults, examples etc: https://github.com/bitdiff/synoptic (disclaimer: it's my companies' project)

For Ruby people wanting something similar, check out https://github.com/delano/drydock

I've been looking for an excuse to use it.

There is already a Ruby port at https://github.com/docopt/docopt.rb

But I must warn you: Ruby port (as of now) implements only a small subset of docopt language, but we're working on it.

I'm back from holiday now and should have some time to work on this - Alex

On Python I'm a fan of Opster: https://github.com/piranha/opster

Could this technically be ported to bash ?

One of the early versions of docopt (0.1) was ported to bash: https://github.com/colinta/bocopsh

It should be relatively easy to update it to work with 0.4; if you can make it work with 0.4 it would be great if you make a pull request to the project above.

I will try and we'll see if I have the skills then :)

Any plans to add Ruby and other programming languages?

See clap for an implementation, https://github.com/soveran/clap

I think Clap is good. But you can't beat the readability and succinctness of a DSL (docopt).

Right now we have two full-blown working implementations: - Python - CoffeeScript

We have just started to port to Ruby and Lua. All ports live under `docopt` organization on GitHub: https://github.com/docopt

You are very welcome to help us out with Ruby, Lua, and your favorite language :-).

You should checkout Clamp https://github.com/mdub/clamp

seems like per #1 & 2 you could either write a dsl or line-processing library and get it done in 1 line

That exactly what [docopt](http://docopt.org/) is—DSL that allows implement almost any interface in one line (having formal interface description).

ahh, cool. suppose I should have read past the rules

Reply on news.ycombinator.com