Why not parse `ls` and what to do instead

For decades, on most Windows computers I run web browsers, there's always Internet Explorer. So do you still always use IE because installing Chrome is "wasteful"? It's a hell of a lot bigger and more wasteful than Python. As I already said, that is a weak and misleading and pernicious argument.

So what exactly is bash better than Python at, besides just starting up, which only matters if you write millions of little bash and awk and sed and find and tr and jq and curl scripts that all call each other, because none of them are powerful or integrated enough to solve the problem on their own.

Bash forces you to represent everything as strings, parsing and serializing and re-parsing them again and again. Even something as simple as manipulating json requires forking off a ridiculous number of processes, and parsing and serializing the JSON again and again, instead of simply keeping and manipulating it as efficient native data structures.

It makes absolutely no sense to choose a tool that you know is going to hit the wall soon, so you have to throw out everything you've done and rewrite it in another language. And you don't seem to realize that when you're duct-taping together all these other half-assed languages with their quirky non-standard incompatible byzantine flourishes of command line parameters and weak antique domain specific languages, like find, awk, sed, jq, curl, etc, you're ping-ponging between many different inadequate half-assed languages, and paying the price for starting up and shutting down each of their interpreters many times over, and serializing and deserializing and escaping and unescaping their command line parameters, stdin, and stdout, which totally blows away bash's quick start-up advantage.

You're arguing for learning and cobbling together a dozen or so different half-assed languages and flimsy tools, none of which you can also use to do general purpose programming, user interfaces, machine learning, web servers and clients, etc.

Why learn the quirks and limitations of all those shitty complex tools, and pay the cognitive price and resource overhead of stringing them all together, when you can simply learn one tool that can do all of that much more efficiently in one process, without any quirks and limitations and duct tape, and is much easier to debug and maintain?

Love it.

  for name in subdir/subsubdir/*; do
    echo "${name#subdir/}"  # subsubdir/foo
  done

  find . -name '*.hs' -exec basename {} \;

  cd some/dir/here
  find . -name '*.gz'
  cd - # changes back to previous directory

Of course you can pass `sh -c '...'` (or Bash or $SHELL) to `find -exec` or xargs but then you easily get into quoting hell for anything non-trivial, especially if you need to share state from the parent process to the (grand) child process.

You can actually get `find -exec` and xargs to execute a function defined in the parent shell script (the one that's running the `find -exec` or xargs child process) using `export -f` but to me this feels like a somewhat obscure use case versus just using an inline while loop.

The "-r" flag allows backslash escaping record terminators. The "find" command doesn't do such escaping itself, so that flag will cause files with backslashes at the end to concatenate themselves with the next file.

Furthermore, if IFS='' is not placed before each instance of read, or set somewhere earlier in the program, than each run of white-space in a filename will be converted into a single space.

EDIT: I proved your point even more. The "-r" flag does the opposite of what I thought it did, and disables record continuation. So the correct way to use read would be with IFS='' and the -r flag.

there are other tools than `ls` with their soul purpose to list files; some have "improved" features than ls, ect

similarlly from above (and well said btw, even what is not quoted): > I do know that when I'm scripting, I'm generally wrapping what I do by hand, in a file. I'm codifying my decisions with ifs and such, but I'm using the same tools that I use by hand

a lot of us do similar and we know/expect the ins and outs; and all it takes to break our scripts is some edge case we never thought of. we are fortunate that our keyboard layouts are basically ascii; other languages are less fortunate. now introduce open source community driven software where an ls escaped bash code deletes somebodys home directory (as ls was parsed and an edge case of some users files cause some obscure "fun" times). an edge case is still painful..

and finally, sometimes its better elevating said bash script to python (or awk), ect. just depends on the situation and level of complexity of logic

  for i in *.jpg; resize 200x200 "$i" "thumbnails/$i"; end

I switched the command to a graphics magick based resize since that's the tool these days, default quality is 75% (for JPEG), but is included as a commonly desired customization. ,, is from a different comment in this thread; it seems better self-documenting than the single , I'd traditionally use.

  find . -maxdepth 1 -name "*.jpg" -print0 |\
  xargs -0P $(nproc --all) -I,, gm convert resize '200x200^>' -quality 75 ,, "thumbnails/,,"

  for f in subdir/*; do
    ...
  done

  ( 
  cd subdir || exit 1
  for f in *; do
    ...
  done
  )

If for some reason you do need the "find | xargs" combo (maybe for concurrency), you can get it to work with "find -print0" and "xargs -0". Nulls can't be in filenames so a null-delimited list should work.

Yes: bash is probably fine.

No: real programming language time.

How do you check if they are still valid files?

I do. I replaced all of the automation scripts on my rpi with pwsh scripts, and I'm not regretting it. Not having to deal with decades of cruft in argument parsing and string handling, learning little DSLs for every command, etc. is so worth it.

Hinder myself? An ephemeral cloud machine would not keep my custom shell anyway. By having to install it _every single time I connect_ I just loose precious time.

I want to be familiar with tools that are _already_ installed everywhere.

The shell is supposed to be a bottom feeder, lowest common denominator, barely usable tool. That way, it can build soon and get stable real fast. That (unintentional) strategy placed it as a core infrastructural piece... everywhere.

Of course, there's scripting and using it on the terminal. But we're talking about scripting, right? Parsing ls and stuff. I want the fast, lean, simple `dash` to parse my fast, lean simple scripts. pwsh is fine for the terminal leather seats.

    #define _GNU_SOURCE
    #include <dirent.h>
    #include <fcntl.h>
    #include <malloc.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <unistd.h>

    #define BUF_SIZE 32768

    struct linux_dirent64 {
      ino64_t d_ino;           /* 64-bit inode number */
      off64_t d_off;           /* Not an offset; see getdents() */
      unsigned short d_reclen; /* Size of this dirent */
      unsigned char d_type;    /* File type */
      char d_name[];           /* Filename (null-terminated) */
    };

    int writeall(char *buf, size_t len) {
      ssize_t wres = 0;
      wres = write(1, buf, len);
      if (wres == -1) {
        perror("write");
        return -1;
      }
      if (((size_t)wres) < len) {
        return writeall(buf + wres, len - wres);
      }
      return 0;
    }

    int main(int argc, char **argv) {
      if (argc != 2) {
        return EXIT_FAILURE;
      }
      int fd = open(argv[1], O_DIRECTORY | O_RDONLY);
      if (fd == -1) {
        perror("open");
        return EXIT_FAILURE;
      }
      void *buf = malloc(BUF_SIZE);
      ssize_t res = 0;
      do {
        res = getdents64(fd, buf, BUF_SIZE);
        if (res == -1) {
          perror("getdents64");
          return EXIT_FAILURE;
        }
        void *it = buf;
        while (it < (buf + res)) {
          struct linux_dirent64 *elem = it;
          it += elem->d_reclen;
          size_t len = strlen(elem->d_name);
          if (writeall(elem->d_name, len + 1) == -1) {
            return EXIT_FAILURE;
          }
        }
      } while (res > 0);
      return EXIT_SUCCESS;
    }

If bash was ever actually going to get json parsing in reality, it should have done that two decades ago like all the other scripting languages, since JSON is 23 years old. So don't hold your breath.