Hacker News Reader

Comment by seodisparate | Hacker News Reader

One could use `printf 'first\0second'` in the Bash shell as an example of making a "string" with null embedded into it, but you can't store that in a Bash variable.

    $ TEST_VAR="$(printf 'first\0second')"
    bash: warning: command substitution: ignored null byte in input

I'm not familiar with working with null characters in Bash in this way, but I think there might be a way to do it.

View on HN

Shells and their aversion to null characters was my first introduction to Perl way back when. Tcl, at the time, couldn't handle null characters either. Various Awk implementations had different issues with them. Python didn't yet exist, C was too tedious for many things.

One option with shells is some set/get functions that encode/decode to base64, hex, etc. Feels pretty clunky though.

I am sure you know this, so I'm just being pedantic here, but it's not that really the shells that have an aversion to the null character so much as that the exec() system call and the main() convention require C strings for program names, program arguments and environment variables, and since shells are thin layers above exec() and the environment, shells kinda have to also use C strings. Sure, nothing stops a shell from using counted byte strings and then allowing nulls in non-exported variable values, but because they evolved in a system that was so deeply based on C and C strings... they don't.

That's the path TCL initially took, but they added counted byte strings later when it became a barrier. And zsh appears to support them relatively well.

Once you get past merely putting together pipelines of command executions and grow all the language functionality you need to much beyond then yeah, you end up needing a language that allows you to have nulls embedded in strings.

You don't actually need to store it in a variable. Do it the bash way and write it to a temporary file. You can then get the sha1 of that file and then zlib compress and copy it to the appropriate dir and filename.

Reply on news.ycombinator.com