Tools I use: argparse builder | Hacker News Reader

Tools I use: argparse builder

blog.rfox.eu

109 pointszdw46 comments

I'd really love some kind of unified interface for program input.

Like, a Python program, a Ruby program, an online form, a REST API, and so on, all specify the input they need in ways specific to each tool (selecting from a predefined list is `CHOICES` param for argparse, a dropdown for a web form, etc). Why can't we just have some JSON spec for "input"? A list of params, each with a name, a description, a type, perhaps a list of possible choices -- the same sort of thing `argparse` exposes as its API.

Then, depending on the need, we can render that into a web page, or a parser, or whatever, and we can build libraries that can always expect to get input the same way without needing to redefine it all for each interface. Does something like this exist already and I just haven't seen it?

https://github.com/shellcomp/shellcomp.github.io/blob/master...

https://github.com/oilshell/oil/wiki/Shellac-Protocol-Propos...

> Does something like this exist already and I just haven't seen it?

OpenAPI (Swagger) kind of, although AFAIK it doesn't lend itself to specifying how to map things onto a CLI. A good next step for the project?

Sounds a bit like SOAP, COM, or XML-RPC

I have been thinking of building something like this for some time. It would be great to have something that automatically generated a:

* CUI

* TUI

* Web UI

* REST API

* Config files

* Environment Variables

* GUI

There are pieces of this in various places. There are a number of command line libraries that integrate config files, command line options, and environment variables, including Traitlets (used in Jupyter). The original Enthought Traits did this for GUIs (declare a set of of classes with datatypes, call .configure_traits() on them, and voila, an autogenerated GUI for them. The Django admin interface is very similar for the web. What's needed is to bring this into a single package.

I also very much like this idea, as I find writing CUI/GUI code kind of boilerplate, annoying, and often feels like this is sth which could be automated.

In fact, I also implemented a similar idea to automatic GUI generation for my music player: https://github.com/albertz/music-player

I tried to summarize the idea here: https://github.com/albertz/wiki/blob/master/automatic-user-i...

Maybe you have some further good references?

Sounds nice in theory, but with the state of modern development, this kind of thing will be implemented as a CLI frontend for an electron app.

But seriously though, the reason why this kind of standardised interface doesn't exist is because there's no one-size-fits-all solution. Everyone will "need" an extension to the spec, or start adding metasyntax to fit their specific use case.

Have to plug Typer[1] here - it's built on top of click, but uses the info already there type annotations, optional keyword arguments, default arguments, etc., to create the most "common sense" CLI for a function. For all my internal tools, has saved me a ton of time and gruntwork.

[1] https://typer.tiangolo.com/

I've been using Typer for my projects recently and I have to agree, it's great! My only gripe with it is that its autocompletion-installation arguments can't be turned off, so it "pollutes" your help text in a way. It's not a big deal, but especially for smaller tools with small CLIs this annoys me a bit.

Better solution would be to clean up the argparse API. For example why is the explanatory text argument “description” when you call the parser constructor (argparse.ArgumentParser) but “help” when you call the argument constructor (parser.add_argument)? Also why are the docs for nargs (whose options I always forget) so buried in the argparse docs?

"description" and "help" are different things.

The docs are open source, you can fix it and send a request a pull!

No, I don't see any functional difference between the explanatory text for the argument and the explanatory text for the program. They could both accurately be called "help" or "description," but for some reason the original author did not want to choose. (I agree the dictionary definition of the words are different, but in this context there is no functional difference.)

Also, your first sentence explains why I don't do what you suggest in your second sentence :-) Some people may not agree with my reasoning (which is obviously their right), not worth the trouble to figure out how to submit a patch and then risk having it rejected.

The "help" is just one component of each argument/option's description, which also includes the short/long syntax guide.

I use click[1] for things that don't need the complexity/features of argparse, personally, which ends up saving a lot of time.

[1] https://click.palletsprojects.com/en/7.x/

For things that don't require the complexity/features of argparse, I use sys.argv. Doesn't require any extra packages and other programmers have no trouble understanding the code.

Also it avoids a pallets package, which is good because I prefer to avoid any unnecessary exposure to its maintainer.

What issues do the pallets maintainers have?

This is HN not Reddit, so I will just say I personally don't like him and talk about sys.argv.

Nearly every script I write only requires something like the following:

    for arg in sys.argv[1:]:
        if arg == '-v':
            ...
    if ...:
        print(f"Usage: {sys.argv[0]} -v ...")

In the very rare occasion that I have an actually large command line project, argparse is built in and works fine.

What I like about argparse is that it will handle stuff like --infile -v (-v means filename in this context since it's value of the infile parameter)

or -- -v (-v is forced to be interpreted as positional in this case since it comes after --).

Yeah, I would use argparse if I had to support something like that.

To be honest, the extent of my command line parsing is almost always just this:

    try:
        infile = sys.argv[1]
        outfile = sys.argv[2]
    except IndexError:
        print(f"Usage: {sys.argv[0]} <infile> <outfile>")

Moreover, click provides a lot of additional CLI-related stuff (printing with colors, asking for input, progress bars) with IMHO pretty reasonable default behaviour. And it's really well-documented!

I normally use Click but sometimes argparse is great for CLI tools that have zero dependencies and aren't massive in scope.

Since most systems have Python, distributing your argparse driven file means curling it down and making it executable instead of having to pip install things.

Click lacks flexibility, requires almost the same amount of boilerplate than argparse, and add dependencies.

You might want to look at Google’s python fire which makes this sort of thing obsolete.

https://github.com/google/python-fire

Just comment your code and get pretty arguments.

Plus one on this, it's a cool project/idea, that really takes advantage of the language's dynamic nature.

It's a great example of a project that only works (without codegen / build toolchain-support) for a dynamic language.

I use docopt for CLI programs.

https://github.com/docopt/docopt

It is a simple DSL with implementations available in multiple languages. I've used it in python, GoLang and in Java too. Lots of examples too.

Tools I use in Ruby: Thor.

https://github.com/erikhuda/thor

Because it's great that you have a library to parse args... but that's only about half of what you actually want when you're building a snazzy CLI tool.

What if this existed for bash? I wonder if anyone would be interested in hacking the same for getopt(3) to write bash scripts that accept options.

Great idea.

Not exactly the same, but this was posted a bit ago: https://github.com/matejak/argbash

Neat. The way I solved this about a decade ago is to create a base template script for my editor. When I hit Ctrl-N a new main script opens with heading, main(), status code return, CLI parsing, and basic logging set up already.

New argparse params are either uncommented and/or cut and pasted from existing ones. It works pretty well.

That's way cool. What editor do you pull this magic out of?

Geany, but I don't think it is unique functionality. I suppose you could just copy from a read-only template file with any OS or tools.

Now that I look closer, I actually have to use the File menu, it doesn't have a default shortcut assigned.

Haha I had a similar problem with argparse so I built a declarative cli and I documented the hell out of it, I added all the recipes in the README. For me it's just ctrl+f in the README and I get what I need. So far it's worked well and people contributing to another project using it haven't complained.

https://github.com/Woile/decli

But nowadays I would recommend something like typer: https://typer.tiangolo.com/ it's quite nice and has autocomplete built in.

I've been using my own interface to easily define CLI applications: https://github.com/breuleux/coleo (kind of a work in progress).

Admittedly, the way it works is kind of black magic, so it's probably not for everyone, but it's so simple: just annotate an existing variable with Option and it becomes a command-line argument. You can define options where they are used in the program instead of defining them in one spot and then using them somewhere else.

The argparse builder, this looks like sth you get for free by any advanced IDE, like PyCharm, right? Any IDE will always tell you what options / parameters there are, and their type, and documentation, so it doesn't matter if you forget them.

I use argparse daily. One CLI script I have uses over 1000 LOC to define all the commands/arguments/flags. Am I doing something wrong? Is there a way to unclutter my CLI script but still have the commands? Import them from another file, perhaps?

As another commenter here mentioned, you might want to consider fire [1], depending on the complexity of your argparse-usage it might check 90% of your boxes for you.

[1]: https://github.com/google/python-fire

Thanks, that looks pretty slick!

As someone who's used argparse recently, this would have been helpful. argparse is great, but it's (personally) rather cryptic especially if you're not intimately familiar with it.

The "+" button to add another argument doesn't work in Safari. It works in Chrome.

I'd recommend docopt/docpie which create your parser from the documentation you write

I want one like this for OCaml.

Bleh. When you need a in-browser tool to use a library, you really know it's messed up.

Isn't there anything like structopt https://docs.rs/structopt/0.3.19/structopt/ for Python?

Looking at that link, argparse is very similar to structopt. It just uses the syntax and idioms of a different language. The structopt attributes appear to be approx. the same complexity as argparse functions.

Indeed. My guess would be that structopt's design was based on Python's argparse, which itself derived from lessons learned from the earlier Python module optparse, which was created around 20 years ago. I'm not sure where declarative CLI argument parsing originated, but I first encountered it in optparse.

There are multiple alternatives for argparse if you look for third-party libraries.

The thing that makes argparse particularly useful is because it is in stdlib, so if you use you can create a Python script without external dependencies.

Reply on news.ycombinator.com