Hacker News Reader

bash has handled spaces in filenames easily for a long time:

    #!/bin/bash
    # run this scri0t with 0 or more filename args
    for file in "$@" ; do
        do_something_with "${filename}"
    done

> Nobody has a good solution! Bash is just permanently fucked when it comes to file name

From the man page bash(1):

    Special Parameters
       @  Expands to the positional parameters, starting from one.
          When the expansion occurs within double quotes,
          each parameter expands to a separate word.
          That is, "$@" is equivalent to "$1" "$2" ...

It's possible to write bash that handles spaces in filenames properly, but it's really hard to do it consistently. Unless you test it then you've probably got it wrong.

Most people don't test their bash on spaces. Look at the dismissive commenter above who just things "nah why would people use spaces in filenames?". Do you think he quotes everything properly? No chance.

CMake is also really bad about this. Although it is arguable slightly better because it uses semicolons as list delimiters instead of spaces, and semicolons in filenames are going to be very uncommon. I would expect there isn't a single CMake project out there that works if you build it in a path containing a semicolon though!

To compound the problem, bash tends to attract the kind of care-free security-unconscious people who don't want to be bothered with going out of their way to handle unexpected "trivialities" like spaces in file names or carefully sanitizing user input or meticulously testing their code.

The same kind of lazy care-free people who love the convenience of PHP's "register_globals", because it "saves typing".

If they're too lazy to use something like Python in the first place, they're probably also too lazy to be conscientious and anal retentive enough to write and test bash scripts that handle spaces in file names properly, or do any kind of user input sanitization.

And they don't think twice about running their scripts as root, either. Or writing blog postings telling other people to copy and paste their code snippets into a root shell, or worse yet download and run them without auditing first, by piping curl into a root shell.

I recently had an Xcode project fail to build (blender, which uses cmake), because I was using a version of Xcode that had a space in its directory name (because I had foolishly added a space followed by the version number to the name "Xcode 9.4.1.app"). Of course that was because of some of the tools and custom build steps were calling out to bash. The problem wasn't in any of the source code or configuration files, it was in the file name of the tool itself! It took hours of frustration poring over machine generated project and makefiles to diagnose the problem from the obscure unhelpful error message, which I will never get back.

In the long term, isn't it more efficient of a lazy person's time to simply use a scripting language that just doesn't have these problems in the first place, and doesn't require you to preemptively bend over backwards and dodge silent invisible bullets all the time?

It's pretty ridiculous that a scripting language intended to be used to operate on files falls flat on its face and is so difficult to use when you have something as common and innocuous as files with spaces in their names (or worse yet, file names that begin with dashes -- think of the security implications of that: not something a lazy careless bash scripter would ever pause to reflect about)!

It really is an extremely common and unsolved problem. Notice how each "better answer" is actually worse than the last and comes with its own host of problems and edge cases and work-arounds:

https://unix.stackexchange.com/questions/9496/looping-throug...

https://stackoverflow.com/questions/3967707/file-names-with-...

https://www.cyberciti.biz/tips/handling-filenames-with-space...

https://www.tecmint.com/manage-linux-filenames-with-special-...

https://www.hecticgeek.com/2014/02/spaces-file-names-command...

And how do you safely iterate over the output of "find" (including properly handling file names beginning with dashes, while you're at it), without forking another sub-bash for each file name?

As I said, that hasn't been a problem for a long time:

    $ find . -type f
    ./foo bar
    ./--stupid-filename
    ./-annoying dir name!/--very annoying path...

    $ echo "pid=$$"
    pid=31281

    $ while IFS= read -r -d '' file ; do
    >     echo "[pid=$$] do something with '${file}'"
    >     list[i++]="$file"
    > done < <(find . -type f -print0)
    [pid=31281] do something with './foo bar'
    [pid=31281] do something with './--stupid-filename'
    [pid=31281] do something with './-annoying dir name!/--very annoying path...'

    $ declare -p list
    declare -a list='([0]="./foo bar" [1]="./--stupid-filename" \
       [2]="./-annoying dir name!/--very annoying path..." [3]="./iter.sh")'

Using the character literal $'\0' also works as the delimiter, but Bash doesn't support sending null bytes in args. In bash, $'\0' is the empty string:

    $ x=$'\0'
    $ declare -p x
    declare -- x=""

> What if one of your file names has a space in it? Most bash scripts will crash when that happens.

And good for them! What kind of savage puts spaces on their filenames?

All my Whitespace scripts have file names with nothing but spaces in them!

https://en.wikipedia.org/wiki/Whitespace_(programming_langua...

I also use spaces in the file names of all my C++2000 source code that takes advantage of the wonderful new whitespace overloading feature -- it's conventional to use the ".cpp " and ".h " suffixes with trailing spaces.

http://www.stroustrup.com/whitespace98.pdf

That kind of savagery is deeply respected, of course! And it should go even further.

Now that the modern compilers and interpreters (clang, python3, etc) allow unicode in variable names, the next step is certainly to accept spaces in variable names. With a slight modification, the python language can be adapted to support spaces inside variable names.

After all, people have the right to name their variables however they want!

Not to mention that Unicode has several delicious varieties of invisible non-breaking, zero width spaces and shy hyphens to choose from! (Oooops, I mentioned it.) Tastes great, less filling, and perfectly justified!

https://en.wikipedia.org/wiki/Non-breaking_space#Width_varia...

https://en.wikipedia.org/wiki/Zero-width_space

https://en.wikipedia.org/wiki/Soft_hyphen

I seriously would have never thought that anybody would really consider using spaces in filenames. For me it's a sign to not work in such an environment, which might be the reason I rarely even see it. But sure if you want a script that is widely used you might want to handle that case as well.

In any case I don't say put in a Bash script and let it grow to ten million lines over 20 years.

My approach to bash scripts is this:

1. start with a bash script to understand a problem and see how a solution might be structured. PoC level is enough here.

2. When you know you solve the right problem and the customer is willing to make something bigger, then put ~80% of the code in a good programming language (Python is my personal favorite, but Ruby, Java, PHP, C#, C++, Lisp might also work). That happens about 3 months into a project.

3. If you start to hit bottlenecks and performance ceilings put the 30-50% that are relatively stable and most performance hungry into C/C++ code and really optimize the heck out of it. This should be done only with an aged project, like 2 years in, and very pessimistically, since highly optimal code usually is badly readable and badly maintainable. You certainly don't want to go into that code 6 years later and drastically change its API to your python code.

So a ten year project would have a thin layer of bash/docker at the top, a strong core of highly flexible code (of Python for me as well) in the middle, and a leg of high performance C code here and there.

I honestly have never seen a project that really needed the C-layer though. In modern IT world it seems most problems are old at least from a political/management point of view between 2 weeks and 3 months. So in fact, although I count myself as a Python coder, the last few years I've spent writing way more bash and docker related stuff, which then is either never touched again or directly thrown away after a few weeks, no matter if successful or not.

PS: Please don't use spaces in file names. Not just for scripts. It is also not clear for users where something ends, and they are even less used to putting string indicators around file names than you are about handling spaces with bash there.

And last but not least stuff like spaces in filenames, starting scripts with upper case letter etc, they might even be just a small annoyance. But you can be sure that skilled developers will put you 2 classes below their average when they see this, because it simply shows a lack of the right spirit.

Not everyone who uses files is a developer. People should be able to name their files what they want. When the time comes that somebody runs a script on it, it should not do something wonky because they decided that "Spring Pictures" was a much better name than "spring_pictures".

People also shouldn't use the razor for cutting grass. It's sadly true that you need to learn how to use your tools and every tool needs some learning to use efficiently. That's the nature of things.

I do agree though that filesystems are not the most user friendly abstraction of data. Maybe we can do better than that.

But also filesystems have the huge advantage that they are somewhat user friendly while also being somewhat technology friendly. Therefore I'm also not too mad about their existence. There are a lot of end user problems that can be solved quickly exactly because they just need a professional one time query and FS+bash+clis are such a good combo to craft that query.

Razors can cut more than grass, and some file systems are more user hostile than others, while making it difficult to clean up forensic evidence.

https://en.wikipedia.org/w/index.php?title=Comparison_of_fil...

(Spoiler: Note the last column. Also: https://en.wikipedia.org/wiki/Hans_Reiser ...)

> People should be able to name their files what they want.

This is not true, and I do not see why it should be like that. You cannot, for example, have slashes in filenames; and for good reasons. Disallowing spaces in filenames solves a lot of problems.

Slash is the ONLY character you're not allowed to have under Unix. There are no good reasons to disallow spaces. Disallowing characters in file names that you're allowed to have solves absolutely no problems, it only causes them.

There used to be a bug in the Gatorbox Mac Localtalk-to-Ethernet NFS bridge that could somehow trick Unix into putting slashes into file names via NFS, and Unix would totally shit itself when that happened. That was because Macs at the time (1991 or so) allowed you to use slashes (and spaces of course, but not colons), and of course those silly Mac people, being touchy feely humans instead of hard core nerds, would dare to name files with dates like "My Spreadsheet 01/02/1991".

https://en.wikipedia.org/wiki/GatorBox

I just tried to create a file name on the Mac in Finder with a slash in it, and it actually let me! But Emacs dired says it actually ended up with a ":" in it. So then I tried to create a file name with a colon in it, and Finder said: "Try using a name with fewer characters or with no punctuation marks." Must be backwards compatibility for all those old Mac files with slashes in their name. Go figure!

If you think nobody would ever want to use a space or a slash in a file name, then you should get out more often and talk to real people in the real world. There are more of them than you seem to believe!

> Slash is the ONLY character you're not allowed to have under Unix. There are no good reasons to disallow spaces. Disallowing characters in file names that you're allowed to have solves absolutely no problems, it only causes them.

I think that there are more good reasons to disallow spaces than slashes. For one, it is very useful to have a natural character to separate filenames in a string. Then your bash scripts can be simpler! :)

> trick Unix into putting slashes into file names via NFS, and Unix would totally shit itself when that happened

LOL. I love this stuff. There is a similar trick that you can play today to wreak havoc on osx systems. You commit into a git repository a new file whose name only differs in case with an existing one, and ask a mac user to update the repo. Hilarity ensues.

> If you think nobody would ever want to use a space or a slash in a file name, then you should get out more often and talk to real people in the real world. There are more of them than you seem to believe!

But these users should not see the filename anyway, but the "title" of the document, which can be stored inside the file and can have whatever character they want. At the minimum, the user-facing interface can convert user-typed spaces into another character (e.g., a unicode non-breaking space) when writing the file into the filesystem. Raw spaces in filenames are an abomination that make shell scripting unnecessarily difficult. I would love a mount option that disallow the appearance of such things in my disk.

That ship has sailed a LONG time ago. We're here. We're spaced. Get used to it.

So you think all files should have a human readable title inside of them, huh? Then you're OK with MICROS~1 8.3 file names, and not ok with any file format that doesn't allow a human readable title inside? Should empty files be prohibited too?

https://en.wikipedia.org/wiki/8.3_filename

That would be a great argument if only you were Hitler and could tell everyone how to name their files, but you're not. I can think of a lot of great changes I would make if only they'd make me Hitler. Or Darth Vader! Or even Scorpius.

https://farscape.fandom.com/wiki/Scorpius