Show HN: Bish – Shell scripting with a modern feel | Hacker News Reader

Show HN: Bish – Shell scripting with a modern feel

github.com

84 pointsbishc42 comments

For a long time I couldn't figure out why moving from bash to Python felt like such a big leap, and it was quite frustrating (path manipulation is particularly difficult).

Recently I discovered a post [1] which touches on this:

> The whole point of short code is saving human bandwidth, which is the single thing in a computing environment that doesn't obey Moore's law and doesn't double once in 18 months. Now, which kind of bandwidth is the most valuable? I'll tell you which. It's the interactive command bandwidth.

It makes the argument that syntax is actually really important in building an interactive language - '[]' is better than '()', ' ' is better than ','. And it's true, bash (and, as it happens, tcl) ruthlessly pursue this - strings don't require quotes, function calls just require spaces (rather than brackets and commas) and there's generally very little punctuation for the most common operations.

I bring this up because it's interesting to see bish has gone in the other direction (curly braces, semicolons, brackets, quoted strings) in the name of a 'modern feel'. It makes me wonder if there's a way to bring the power of python (etc) into the shell in a syntax-friendly way.

[1] http://yosefk.com/blog/i-cant-believe-im-praising-tcl.html

Interesting notion! I personally am not a fan of whitespace sensitivity, but I understand why many people are. For me, after spending years writing code with curly braces and semicolons, they are effectively invisible (and thus do not seem to waste my "human bandwidth").

That being said, the choice to include those syntax features in bish was made entirely to make writing a parser easier, not for any deep philosophical reason.

>was made entirely to make writing a parser easier

That's called optimizing bish developer bandwidth, rather than optimizing bish user bandwidth. Neither requires philosophy :)

I wouldn't exactly call it "modern". It seems like csh++ while avoiding its mistakes by having Bash as the backend.

I tend to write shell scripts in Ruby even more often than bash. I'm proficient in both, but I find that the larger the script, the more I'm affected by the lack of language support in bash, and the many oddities such as the many problems with quoting and string splitting and the hacks/special mechanisms needed to work around them.

Even though bash has some "modern" features, they're typically a bit weird and hard to remember: bash has arrays, for example, but the syntax is bizarre; you can write functions, but you can't declare the parameters, only access them as $1, $2, etc.; and so on. Occasionally I have to look up some expression syntax (like which bracket syntax to use for arithmetics, or whether "if" should have one or two brackets, and whether it's [[ a && b ]] or [[ a ]] && [[ b ]]) just because it's so damn different. It's painfully obvious that bash wasn't planned; so much of its evolution can be felt in the layers of hacks.

And with Ruby it's much easier to refactor to do things like logging (eg., when spawning a sub-command, only show its output if it fails), or provide cleanup (express your script with the command pattern, with commands that know how to commit and undo themselves), or use existing libraries.

Ruby's syntax — the ability to omit parantheses, the support for running shell commands with backticks — makes it particularly suited. For example:

    if [ -e ./somefile ]; then
      ./somecommand ./somefile
      (cd ./somedir && make)
      installdir=`getsomeconfig`
      cp build/mybinary $installdir/bin
    fi

In Ruby:

    if File.exist? "somefile"
      `./somecommand ./somefile`
      Dir.chdir 'somedir' do
        `make`
      end
      installdir = `getsomeconfig`
      cp "build/mybinary", "#{installdir}/bin"
    end

The last line requires that you initially do:

    require 'fileutils'; include FileUtils

Now you have a bunch of utilities (cd, cp, mv, mkdir, mkdir_p, etc.) as global Ruby methods.

Bash sucks when it comes to arrays. Last week I had to change the 'input field separator' because bash arrays are kinda-sorta-but-not-really space delimited, which causes problems where elements have whitespace. I then had to revert the IFS immediately after the required operation, as that screwed something else up later in the script.

I like bash scripting, but the moment to use another language for a script for me is "will this require an array?"

The semantic value of that punctuation is primarily for readibility and comprehensibility, then, where human bandwidth is way more limited. Achieving this goal is much more critical than reducing keystrokes, IMO. "Code is for humans to read, execution is an afterthought" or whatever the famous quote is. This isn't simply included to achieve a "modern feel".

IIRC Brian Kernighan said that. And the Kernighan & Pike book regards the shell, not C, as the primary language of Unix.

Check out Batsh (which "compiles" to Bash *.bat):

Editor: http://batsh.org/

Source: https://github.com/BYVoid/Batsh/

I think that this may be unnecessary given the existence of the sh[1] Python module. It's incredibly simple and concise, e.g.:

    from sh import date, ls, egrep, git

    print date(), ls('~/project')
    print egrep(ls('~/project'), '-o', '\.py$')
    print git.add(egrep(ls('~/project'), '-o', '\.py$'))

[1] http://amoffat.github.io/sh/

Perl started with roughly the same goal. awk, sed, grep, etc. weren't quite powerful enough, C was too low-level, Perl mixed them all up with real programming language features, while still being very usable in one-liners.

I don't think this improves on Perl. Being compiled to Bash might be interesting, if your deployment systems don't have Perl. It used to be unthinkable that any UNIX/Linux system wouldn't have Perl...but, several distros no longer install Perl by default (which is annoying as hell, to me, as Python isn't at all an acceptable substitute for one-liners and shell+ tasks). But, in that case, it would need to compile down to POSIX shell, because Ubuntu doesn't install bash by default, it uses ash (a small POSIX shell). So, bash has the exact same flaw as Perl for that use case.

Anyway, I don't generally think this adds things that I wish for when working with shell scripting, and having to compile it (even a quick compilation like this is likely to be) takes away one of the biggest benefits of scripting languages.

This reminds me of another Larry Wall quote about the bad (old?) days: "It's easier to port a shell than a shell script." I'm honestly surprised and a bit disappointed that Bash isn't universal these days.

This seems like a really bad idea, since it apparently throws out good things like pipes, input/output redirection, wildcard expansion, and interactivity, but doesn't offer anything over the standard scripting languages.

> because Ubuntu doesn't install bash by default, it uses ash (a small POSIX shell). So, bash has the exact same flaw as Perl for that use case.

Do you have a citation for this? I just checked my boxes and servers and they all have bash and are using it directly as the default shell. I never had to install bash myself on these machines.

Ubuntu uses dash as the system shell. The default login shell is still bash. While /bin/sh is symlinked to /bin/dash - you are probably using bash from your terminal, unless you changed it.

Actually I just checked by /bin/sh and it's pointed to /bin/bash on my laptop (14.10) but on my desktop it is /bin/dash (also 14.10). Now I have some investigating to do. I suspect I did something that changed that but I don't remember doing it.

Oops, you're right. dash and not ash. Definitely not bash, by default, however. I had to rewrite our install script to be POSIX friendly when the switch was made a few years ago. Not entirely awful, I guess, as the Ubuntu folks provided a really nice guide for converting.

https://wiki.ubuntu.com/DashAsBinSh

It's actually dash. Sorry. Misremembered.

https://wiki.ubuntu.com/DashAsBinSh

Hell and Shelly are based on a similar concept, but also try to use the power of the Haskell type system to prevent common errors and bugs.

[0]: https://github.com/chrisdone/hell

[1]: https://github.com/yesodweb/Shelly.hs

Recently I've returned to writing scripts using Byron Rakitzis' reimplementation of the Plan 9 rc shell, and been very happy with the results. It doesn't have the idiosyncrasies that bash has, and which seem to be a motivation for bish. Variables are not rescanned when expanded so you don't have to remember hacks like '"$*"', for example. At the same time it adds some features which really help when writing scripts (a list type and a pattern matching operator for example). And yet it still feels like a shell rather than a 'proper' scripting language: pipes and redirection are still core parts of the language rather than things you have to program using the standard library.

It's included in the standard Ubuntu repositories (http://packages.ubuntu.com/trusty/shells/rc) and it looks like it's in homebrew as well. I'd recommend giving it a try. The original Plan 9 paper describing rc is a good place to start and a good read, although you should be aware that Rakitzis' version made some minor changes to the syntax (a more usual if...else instead of Duff's 'if not') and dropped some Plan 9 specific pieces.

Why not fish (friendly interactive shell)?

Not the OP, but I was under the impression that fish de-emphasizes shell scripting/programming and puts more emphasis on the interactive experience.

It feels like bish is trying to make using bash as a programming language tolerable.

That's what I was going for. Not a replacement for bash the shell, but a language that provides a bit more comfort for programmers while retaining the ability to run on any system your scripts did previously.

We've done this.

Back in the Day, it was BSD Unix, which used tcsh as the interactive shell and Bourne shell as the scripting shell, as tcsh had tab completion but has a scripting syntax which is harmful to sysadmins and other living things, and Bourne shell had a sane scripting language but no interactive functionality beyond cooked mode.

We developed Korn shell, Bourne-Again shell (bash), and, eventually, zsh to move beyond that idiotic experience. These days, our non-interactive scripting languages are Python, Perl, Ruby, and Ruby with a different mix of libraries, all of which have better FFI other support libraries than any shell does.

My point is, unless bish has Python-class libraries, I don't see the point of taking on the annoyance of having a scripting shell different from my interactive shell.

fish does make scripting so much saner. - $array just does the right thing, which bash can only approximate as "${array[@]}". This alone is worth everything. - (command) splits the output on newlines. - Block syntax is simpler, with exception of boolean composite conditions that need awkward begin ..; end wrapping - "everything is a command" syntax makes writing sometimes more verbose but reading easier. - prototyping at the command line then turning into a script is easier thanks to multi-line editing with indentation and syntax highlight.

There is however the portability factor. I've used fish as my only interactive shell for at least 5 years yet I rarely write a script complex enough to justify requiring users to have fish installed. Even when I write only for myself this sits at the back of my head and biases me to #!/bin/bash... From this perspective, "bish compiles to bash" made me immediately take note as a wise move.

Was "bish" intentional? http://www.urbandictionary.com/define.php?term=bish

As someone, who just recently started writing some small bash scripts, I wonder, why they did not create a universal comparator syntax and letting null, not-existing etc. evaluate to false, on top of the existing syntax. I get that it evolved, but since I did not evolve with it, I get frustrated by constantly tripping up on if-syntax, something you normally do not have to worry about when moving to other languages.

Bash hurts my ego.

def printall(files) {

    for (f in files) {

        print(f);

    }

}

# cwd, ls, and cd are all builtin functions.

dir = cwd();

files = ls();

print("Files in current directory $dir:");

printall(files);

cd("/");

files = ls();

print("Files in root directory:");

printall(files);

Whats wrong with this, much shorter Bash sequence:

echo Files in current directory `pwd`:

cd /

echo Files in root directory:

(OP here). You're right: for simple tasks like listing files or changing directories, you probably don't need bish. I should come up with some better examples where bish wins in readability.

Seems like this could be really useful for writing larger packagers, installers or `curl | sh` type scripts...

I believe the problem comes when trying to do stuff like check to see if a file is in a directory. I'm a ZSH user myself, and my default scripting language is Python for similar reasons to the OP's for inventing Bish. I can't for the life of me figure out the syntax for an if statement, especially for things like checking if files exist. Yes, Bash or Zsh have a shorter syntax for the example, but to do something like this[1] in bash/zsh would be, for me at least, needlessly difficult and painful.

[1] https://github.com/yramagicman/dotfiles/blob/master/bin/upda...

I'm with you. I have written a mountain of Bash scripts to solve various one-off problems, and I'm fucked if I can ever remember the nuances from one time to the next -- when to use [], when [[]], when () or (()), when the dollar sign precedes (), etc. Every single time I have to do anything in Bash I have to re-learn Bash. I suppose I keep using it because, even with having to re-learn every single element of Bash coding every time, it's still faster for doing a bunch of filesystem stuff for me. But god, I hate it. I can't think of a computer technology I hate more. I hate it more than R, and I really hate R too, for some of the same reasons.

I keep intending to learn Perl really well, which I think could absorb much of Bash, and also Awk and Sed, and then there would be only one thing I needed to know for this kind of work, and I might use it enough to remember the various weird syntactic bits from one month to the next. But I never quite get over the hump.

Always use double-brackets [[ ]] if you're writing Bash (not sh) scripts. They are better.

Double-parentheses (( )) are only for arithmetic, e.g. "((count++))" or "echo $((1+1))"

The dollar sign precedes parentheses--i.e. $()--when you're substituting the output of a command, e.g. "echo Today is $(date)". It's easy to remember, because it's just like using the dollar sign for substituting the value of a variable, i.e. "date=$(date); echo $date".

You might want to look into the fish shell. Its scripting is much less idiosyncratic. You don't have to quote any variables, arrays, or command substitutions. e.g.:

    set foods apple banana 'cucumber casserole'
    for f in $foods
        echo $f
    end

prints:

    apple
    banana
    cucumber casserole

...not:

    apple
    banana
    cucumber
    casserole

...as would be the case if you forgot to quote "$foods" and "$f" in Bash.

My dotfiles[1,2] are full of various functions to do stuff. Half of them I jacked from other people[3] when I was using Bash and ported to ZSH when I discovered Oh-my-zsh. However, a few are my own. The ones that I did write are often one-liners that take an argument and pass it to a series of commands that I'd rather not type. For example history | grep -i command, ls -lah | grep dir, or my favorite, cd dir; ls. If I have to do any logic, aside from determining what OS I'm on, I look to python.

[1] https://github.com/yramagicman/dotfiles [2] https://github.com/yramagicman/zsh-aliases [3] https://github.com/mathiasbynens/dotfiles/blob/master/.funct...

You seem like the kind of person who'd appreciate Percol[1]. It's fantastic for shell history. I have it bound to Ctrl+R:

    history | percol --prompt-bottom --result-bottom-up

I'm using fish, so the whole thing looks like this:

    function _percol_key_bindings

            function __percol_ctrl_r
                    set tempfile (mktemp)
                    history | percol --prompt-bottom --result-bottom-up >$tempfile
                    and commandline (cat $tempfile)
                    commandline -f repaint
                    rm -f $tempfile
            end

            # Bind Ctrl+R to percol history function for now
            bind \cr '__percol_ctrl_r'
    end

[1]https://github.com/mooz/percol/

Ammonite http://lihaoyi.github.io/Ammonite/#Ammonite-Ops provides a similar thing for Scala

A definite case of http://xkcd.com/927/

If you want to replace bash (in an environment where Perl, Python, and Ruby all exist in a space for more advanced scripting) you have to do a hell of a lot to improve on the existing standards.

Then again, a fun weekend project playing with compilers is a valid use of one's time, even if the product doesn't change the world forever :)

Available as a docker image:

  $ docker run -ti imiell/sd_bish /bin/bash
  bash-4.3# bish
  USAGE: bish <INPUT>
    Compiles Bish file <INPUT> to bash.

Magic here:

https://github.com/ianmiell/shutit-distro/blob/master/bish/b...

Note I needed to hack the code a little.

Great idea and looks like a good implementation. Are there native hashmaps and arrays? Those are things I like from Powershell.

How does this compare to candidates like Batch named here:

http://stackoverflow.com/q/10239235/309483 ?

Starting a new project in C++ in 2015? Why not Go, Rust, or any compiled, memory-safe language for systems programming?

I feel like Bash is a very limiting language to compile to, because you cannot store the null byte in a (byte)string.

Why?

There is a section of the readme titled "Why" :-).

Reply on news.ycombinator.com