Safe Shell String Interpolation | Hacker News Reader

Safe Shell String Interpolation

samwing.dev

21 pointsWingy7 comments

No love for Python's shlex.quote[1] or their equivalents in other languages to quote arguments for shell? In his case, which is interpolating a variable inside a shell double-quoted string, I would probably use environment variables, or else use shell `printf` with `shlex.quote`. But generally I interpolate directly into the command line using `shlex.quote`, when I can't avoid executing using the shell.

I was happy to learn about Python's template string proposal (i.e., t-strings) from https://peps.python.org/pep-0750/, although it probably won't become common for a while.

[1] https://docs.python.org/3/library/shlex.html#shlex.quote

Yes, you can quote the values and embed them directly, but what if you want to do something like this?

    shell('echo "Hello, {user}"')

You would have to do something like:

    arg_for_echo = f"Hello, {user}"
    shell(f"echo {shlex.quote(arg_for_echo)}")

The main drawback of quoting values like that is that you can't embed them inside an argument in the shell. The environment variable "trick" I used avoids this problem.

Using quotes to sanitize arguments like that is how google/zx (library for writing shell/JS hybrid scripts) handles arguments: https://google.github.io/zx/quotes

My proposal:

    shell('echo '+shlex.quote(f'Hello, {user}'))

i.e. never quote any (non-constant?) shell args by hand.

Pipeline in Python w/o invoking the shell:

    username = input("Hello, what's your name? ")
    p1 = Popen(["figlet", f"Welcome, {username}"], stdout=PIPE)
    p2 = Popen(["lolcat", "-f"], stdin=p1.stdout, stdout=PIPE)
    p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
    banner = p2.communicate()[0]
    print(banner)

https://docs.python.org/3/library/subprocess.html#replacing-...

Don't use the shell unless you absolutely have to, and when you do have to, use shlex.quote for quoting arguments:

https://docs.python.org/3/library/shlex.html#shlex.quote

    username = input("Hello, what's your name? ")
    banner = check_output(f"figlet "Welcome, {quote(username)}" | lolcat -f", shell=True)
    print(banner)

For something this simple, you could also just use `subprocess.check_output` twice:

    username = input("Hello, what's your name? ")
    banner = check_output(["figlet", f"Welcome, {username}"])
    banner = check_output(["lolcat", "-f"], input=banner)
    print(banner)

This is one reason why, really, nobody should use a shell that was optimized for commandline use, for scripting.

On the commandline, all your inputs are usually known and trusted, so the shell language will take a few shortcuts to make everything more convenient for the user. But of course, when scripting, these assumptions don't hold and you need a language with more notational rigor.

Bash et al. are dangerous scripting languages, and they become even more dangerous when users (who typically don't want to understand the subtleties) start using LLMs to generate code for them.

That example is pretty sad, the solution has always been the same (even in sh), use an argv based wrapper instead of something like system(3):

  exec(`figlet "Welcome, ${username}" | lolcat -f`)
  // Should be 
  spawn(["sh", "-c", "figlet \"Welcome, $1\" | lolcat -f", "argv0", username])

> This is one reason why, really, nobody should use a shell that was optimized for commandline use, for scripting.

Tcl showed me you can have your cake and eat it, you just need sane quoting rules (like don't make IFS splitting the default and use matched braces for your quoting syntax to solve nesting) and arrays to escape the infamous quoting hell.

Though I agree that sh is dangerous for newbies, I always advise "1. ALWAYS set -eu, 2. shellcheck" to them.

> Though I agree that sh is dangerous for newbies

The problem is that expert users of sh will write scripts that newbies may at some point modify. This would be much safer if the scripts were originally in a better language.

Bash is a footgun. Even expert users have shot themselves in the foot with it at some point in their career.

Reply on news.ycombinator.com