Beyond Bash: Shell scripting in a typed, OO language | Hacker News Reader

Beyond Bash: Shell scripting in a typed, OO language

docs.google.com

49 pointslihaoyi35 comments

Shell languages are defined by their terseness.

The author misunderstands how important this is: https://docs.google.com/presentation/d/11vZzXCfAA0aOFAuHA0nA...

The author has created a radically "better" shell language which is twice as verbose as bash. In the space of human-computer interaction where shell languages lie, this is tantamount to total failure.

The worst parts of shell are the flow control structures (if/else...). If only we could improve this without throwing out the entire system. Else we will lose the benefits of the ultra-terse and completely decentralized "language" that is unix shell.

This is mostly true, and it doesn't look like this is the solution. My belief is any solution needs to be built from the ground up as a language for shell use, not adapting Python/Scala/Haskell to shell use.

However, there is some non-zero amount of verbosity that would be worth trading for better control flow, composability, data structures, etc. As long as you can do simple operations and pipes with the same syntax

  | grep foo
  cp * path/
  rm -rf

it's ok if some operations are more verbose in exchange for that.

Programmers don't live in bash, and deal with the pain points of switching between bash and an actual programming language on a regular basis. I know I'm there. A change that sometimes is more verbose but lets you do more things in shell would still have benefits.

You might not win over the people who are bash experts, but you could still produce a better tool for many people.

I didn't check it out yet, but I think Tulip (http://www.jneen.net/posts/2015-07-29-tulip-language-updated) has a great potential in this space.

The author explicitly states terseness is a goal, and compares alternatives' overhead to bash. However, he recognises there is a limit whilst remaining safe/close to Scala.

I don't agree control structures are a major problem. There are subtle differences between correct and incorrect expressions, and they are not checked at "compile" time.

How about an interpreter that executes from an AST, which can also re-generate formatted source in two forms, "revealing" and "terse"?

One should be able to also write a debugger that can switch between these modes in real-time.

Beautiful response to a recurrent misunderstanding.

"Shell languages are defined by their terseness."

It is common to see people put a layer of verbosity on top of the Bourne shell (or system(3)) to make "a new shell".

It is also very common to see people put a layer of abstraction on top of a large, verbose scripting language and claim the result to be a new, "terse" language (with all the power of the shell).

But what I like about the Bourne shell is that it is built from only C. And it does not add too much verbosity. It is, as you say, defined by its terseness.

There is also terseness in the roff-like typesetting languages, assembly languages, FORTH, k/q, etc.

Compared to k/q, sh is verbose.

For me, verbosity means loss of power and loss of time.

I am glad there are terse languages.

They may never again be popular but I think they will always exist.

"The worst parts of the shell are the flow control structures..."

For some reason I dislike if/then/else. Instead I make heavy use return values, ||, test(1) and case/esac.

I always wished Bourne's shell (cf. Joy's C shell) made use of more C operators. There's a file called arith_lex.l in the Almquist sh source but these operators are not used in the sh language.

I would switch to the Plan 9 shell but Bourne sh remains more useful due to its ubiquity.

If you don't mind using python, the sh package [0] can really get you a bash-style repl (dinamically) typed.

.. [0]: http://amoffat.github.io/sh/

Yes, in my mind this is the way to go (the other alternative, probably unviable, being scsh https://github.com/scheme/scsh). I recently made a decision not to write any bash scripts that include a conditional or loop; for all those use cases python+sh is the way to go. The reason: http://mywiki.wooledge.org/BashPitfalls

I can't think of any environment where python would not be available. You can even ship sh.py as a single-file dependency along with your script, so `import cp from sh` will work as expected.

Haha! Just reading this thread I got the idea to replace bash with something scheme-like. You have a shell which works like it usually soes, but that easily let's you write any utilities in scheme.

I use scheme to write most of my system stuff (most revently a custom automounter for my raspberry Pi) and thought it would have been a nice project. You could probably go macro galore and make it quite comfortable to do piping and integrate the regular system utilities in the language.

There's also a bash-like Python REPL like Xonsh ⌘ http://xonsh.org/

Still in early development, but very useful.

Some people do their shell scripting in a pure, functional, strictly typed language [0] — and I don't judge them!

[0] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...

That's cool! We should probably join forces. After all, Martin Odersky has stated that moving people onto Haskell was the primary motivation for inventing Scala, which will soon be renamed Hascalator

https://twitter.com/myatlyuka/status/577625111575969792

At the same time, Typesafe is apparently being rename Reactive Banana, which is more evidence of the increased Haskell focus within the community

https://www.reddit.com/r/scala/comments/36ho4u/breaking_type...

EDIT: found a better slide for Hascalator

My friends answered immediately: «Saying that Scala is a gateway drug to Haskell is like saying that poor-quality heroin cut with speed is а gateway drug to LSD».

If anything is going to displace bash at this point it's going to be something that brings substantial benefits to the user. Stricter syntax is good but not of the immediate concern to the user typing his 2x-longer command.

The next evolution of shell is going to be something different. Something that doesn't depend on an antiquated TTY protocol, something that can display images and other constructs more complex that text, and more importantly something that allows the user to accumulate commands instead of typing the same thing over and over again.

Maybe something like ipython/jupyter

Using languages other than shell already brings lots of benefits to the user. That just doesn't matter to the people who are writing large programs in shell. It isn't like those people are basing their decision on rigorous user studies.

PowerShell

There's the shell language and the shell interface. The design of those things influence each other, but they're fundamentally separate even though bash/zsh/fish often make changes to both.

The original article is really about languages, and I think that even with the best interface in the world, we want a better language.

Just changing the language is in the diminishing return area and is why it hasn't happened yet. Something else need to change.

Any sort of shell programming is going to be full of side-effects. No matter how strict the language, building the script will still involve trying on the target system, maybe accumulating shell commands in a script, and then have these long full cycles where the script is tested on a fresh system to make sure it's still working. This is happening every time I do any sort of sysadmin-type of stuff. Even in ansible, chef, docker, ...

To escape that something deeper needs to be changed.

That's Powershell in the Windows world.

Powershell is exactly what bash should be (and this person wants). Their time would have better spent using and improving the pash project (open source Powershell on Mono): https://github.com/Pash-Project/Pash

Powershell is primarily a repl to a homogeneous .NET-based set of tools, which doesn't describe Unix very well. If you want a type-safe repl that interacts with all the handy tools written for the text-stream-only Unix world you're going to have to do some interpretation.

Powershell has some pitfalls of its own (I still much prefer it to Bash despite these):

  1. Dynamic scoping
  2. Squashing lists of size 0 to nulls and 
     size 1 to single elements when they come out of pipes.
  3. Incomprehensible API for XML handling.

On the other hand, when a Powershell program starts getting too big, it's pretty trivial to port parts of it to one of the other .NET languages.

> 3. Incomprehensible API for XML handling.

If you think Scala has a better API for XML handling then you're in for a surprise...

Very true pitfalls, and there are other ones like any non-captured value anywhere in a function becomes the output of the function.

Cool stuff. I really like the idea, but why so many symbols? On slide "2.5 Piping" I find the description on the right to be far more readable than the examples on the left. Also the descriptions on the right are familiar, idiomatic Scala which is also a plus. The stuff on the left is impossible to decipher without some documentation and I don't see how it adds value except from saving a few chars. OTOH it introduces significant load on the developer that has to look them up (again and again) in the manual. If saving these chars are important then one might defend introducing a single new operator for some well thought-out abstraction, but introducing 7 new operators is just to much for anyone to remember. Especially the two last operators seems entirely redundant as the language already has quite compact syntax for function invocation (as does any language).

Congratulations on making something cool and useful.

But I think it's not Bash vs Ammonite, it's more accurately (Alternatives to Bash) vs (The Ammonite alternative to Bash).

I've never worked in an environment that uses sh for large systems. My use is just my daily conversation with my OS and file system. I say things on the command line. Sometimes I say certain things in sequence often enough that I collect them into personal "jargon" scripts of two to twenty lines of Bash. For anything larger I generally use Python.

My dividing line between Bash and Python (or any similar language) is conversation vs contract. A conversation is relatively informal. Anything larger requires negotiation and agreement by parties (me and the OS) using technically understood formal language (to strain the metaphor).

I think Ammonite is more like the contract than the conversation, but maybe not as much as Python.

Li Haoyi has been doing a lot of cool open source Scala work. Ammonite is an example of that. You can find it here https://github.com/lihaoyi/Ammonite

I for one would not to pay the cost of starting the JVM up every time I want to invoke a shell script.

When I see this unifying of language logic and shell ~logic, as a Lisp head, I can't help but to think of lisp machines:

https://www.youtube.com/watch?v=o4-YnLpLgtk

Path normalization does not work like that. If `/bin` is a symlink referring to `/usr/bin`, then `/bin/..` is the same as `/usr`, not `/` as the author suggests.

https://github.com/valvesoftware/steam-for-linux/issues/3671 is a nice motivating example of why I think this project is useful

I have a large multi-concern shell script for building a web-app. The "complexity" seems to impress my workmates, but I dislike it and avoid extending it if at all possible. Something like this might vastly improve the expressiveness, and scalability.

Wasn't this one of the points of Perl? New languages seem small, but tend to grow.

Pretty cool... more documentation and examples here: http://lihaoyi.github.io/Ammonite/

Using a lot of shell can really be dangerous. When you are on the ops side you know how to avoid most of this pitfalls by using tools like Ansible, Puppet or Chef.

Reply on news.ycombinator.com