How to parse config files with Bash | Hacker News Reader

How to parse config files with Bash

opensource.com

20 pointsdevonnull14 comments

> There is one more really cool thing to know about sourcing. You can use a single dot (.) as a shortcut for the source command.

That's backwards. Dot is the POSIX-standard command; source isn't. So it's more like, in some shells you can use the readable source command instead of the cryptic dot, at the cost of portability.

source does not go without a mention in POSIX. The specification says that if the command name in a command matches one of a certain set of identifiers, the results are unspecified. source is one of those identifiers.

This can be dangerous on untrusted files, because it's easy to forget that it will run arbitrary shell commands, rather than just setting variables to static values.

Yeah, this essentially the old "parse json with eval" (or more generally configuration file is code, but json vs eval was a ceaseless problem that was fairly regularly actually exploited rather than just being theoretically exploitable).

Fun fact: the reason json has as many constraints as it does is because rather than simply parsing json (a trivial thing) the json[2].js implementation used a regex to "validate" it before .. throwing it at eval. That necessarily required huge restrictions on what could be there - you can't use "undefined" in json because it's not actually a keyword, no comments (can't handle those with regex), etc all come from that :-/

I read that comments were forbidden in json because some json generators/parsers started putting data about how to interpret them in comments, fracturing the standard.

Loading json as javascript also led to seemingly odd patterns, like putting an infinite loop before the start of a json payload to prevent injected javascript stealing data.

If someone can fool you into using an untrusted file which is a pure configuration file with no code, that's not safe, either.

A little safer and not much harder: use `git config -f`. Something like this, though you may need to test and tweak a little:

  touch "${APP_CONFIG:=${HOME}/.config/myapp.conf}"

  conf_has() { git config -f "${APP_CONFIG}" --get key."$1".val >/dev/null; }

  conf_get() { git config -f "${APP_CONFIG}" --get key."$1".val; }

  conf_set() { git config -f "${APP_CONFIG}" --replace-all key."$1".val "$2"; }

  conf_del() { git config -f "${APP_CONFIG}" --unset-all key."$1".val; }

  conf_list() { git config -f "${APP_CONFIG}" --get-regex '^key' | sed -e 's/^key.//' -e 's/\.val /\t/'; }

The article started out really well but ended too soon.

Often you want a source of truth that is true for more than just your bash script.

It should have included a discussion about how sourcing config lets the config author execute code.

I also think you want to export these values. You source your config, maybe setup an environment and then execute a Python script that reads values from the environment.

Other things I was hoping for: a lot of modern tools come with config in json. Is there a reasonable way to read this into bash script?

Yeah the article is pretty terrible IMO. It seems to be pitched at folks who don't know any better, so just showing that you can use 'source' like a kind of config parser without pointing out that it totally just executes code is pretty reckless.

I thought it was going to be something more like "here's a way to read KEY=VALUE lines into an associative array" (which is pretty easy, and can lead to all sorts of fun discussions/learnings about arrays in bash, subshells, whitespace/$IFS, etc)

Shameless plug, you can use bash to parse a .INI configuration file.

It leverages a one-liner awk:

    ini_buffer="$(print "%s" "$raw_buffer" | awk '/^\[.*\]$/{obj=$0}/=/{print obj $0}')"

Supporting bash APIs using that one-liner awk: https://github.com/egberts/bash-ini-file

The suggested alias for a safer rm command doesn't work in macOS so I came up with an equivalent function:

    rm() { mv -f $1 $HOME/.Trash; } # move source to ~/.Trash for later removal

And if you have two files with the same name...

This made me curious about what macOS does. Using the finder, whether via menu command or drag and drop will rename the file automatically to ${filename} hh.mm.ss.${ext}

But mv does not. Interestingly, it doesn’t appear Apple has added their own cli tool for moving something to the trash in the same way that they provide `open` or `pbcopy` or `pbpaste`

You could execute inline AppleScript to tell the Finder to move the file to the trash, but that’s pretty roundabout

I haven't tested it, but this tool probably handles name collisions in .Trash gracefully:

https://github.com/ali-rantakari/trash

How about writing relatively complex things not in Bash? If your shell script requires configuration, it shouldn't be a shell script.

Reply on news.ycombinator.com