Es: A shell with higher-order functions (1993) | Hacker News Reader

Es: A shell with higher-order functions (1993)

wryun.github.io

66 pointscatern21 comments

Interesting! I keep getting frustrated with the arcane Bourne syntax and its limitations. I've toyed with a lot of shells. But when it comes down to it, I don't want to switch because the alternative is not ubiquitous.

> I don't want to switch because the alternative is not ubiquitous.

This is the sad truth about us, developers. "Disruptive" "Progress" "Tech advance"

except, still suffering using C, C++, Shell, etc. Developers at large are so change averse that is not matched by any other group I think.

I use Byron's rc every day, and it's installable on all machines I use.

Though I also use dvorak and other things where I control the computer I input on 99%+ of the time. Do you not have access to a compiler or package manager on machines you use?

Thanks, I'll check it out.

But the thing is, if only a fraction of the time I spend in shells is a different shell, I can't build productive habits.

I may have access to them but lack the permissions or resources to install things.

You could try zsh which is close enough to bash, but has a fair amount of modern features.

I feel the same way. Would love to get behind a new shell that has a chance at ubiquity.

The basic thesis behind my Oil project is that a new shell won't become ubiquitous unless it's compatible with bash. In other words, the goal of the project is to replace bash.

Just like any new compiled language needs the ability to reuse C code, I think any new shell has to somehow "reuse" existing shell code, rather than starting from scratch.

I'm even more convinced of that now than I was 3 years ago when I started the project. Even I didn't realize how deep the "shell ecosystem" is, i.e. how much inertia there is to overcome.

Why Create a New Unix Shell? http://www.oilshell.org/blog/2018/01/28.html

I'm looking for people to test it now! I made a release a couple days ago and have gotten several good bug reports, but I need more.

It runs many real bash scripts:

(1) Thousands of lines of distro build scripts (unmodified)

(2) Thousands of lines of bash completion scripts (with slight patches)

(3) scripts generated by virtualenv, git-prompt.sh, etc.

https://github.com/oilshell/oil/wiki/How-To-Test-OSH

Testing it out is straightforward but maybe not super friendly. That's what I would like feedback on and help with :) Feel free to join https://oilshell.zulipchat.com/ (log in with Github) and give feedback.

----

EDIT: An important goal is to fill out this page of shell programs that Oil can run:

https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run...

Basically anything you use "in real life" is important!

Am I right to say that OSH is a subset of bash? If so, when does it differ from bash?

I couldn't find this info on the blog. There are a lot of links :)

Roughly speaking, it's a "sane" subset. But it's also a large subset -- it aims to run all bash programs either unchanged or with trivial patches. So far that's been working out better than I expected.

Occasionally a shell script will rely on something really odd, but that hasn't happened too often. For example, arithmetic in ksh/bash has some really horrible corner cases, but nobody relies on those. They mostly just increment counters and so forth.

OSH also has some features that bash doesn't have, so it's not really a subset.

Eventually there is supposed to be a legacy-free shell language called "Oil", but I've had my hands full with OSH, so I don't have much to say about that. But I think replacing bash is more important than creating an incompatible shell language that probably won't get used!

Do you know if there is a comprehensive list of known differences somewhere?

There are some difference listed here:

https://github.com/oilshell/oil/blob/master/doc/osh-manual.m...

Those are the ones that I thought people might hit. There are surely some other differences, which may or may not matter in practice.

This is why I'm looking for people to test OSH now. It's easier to change it sooner than later. If you want to use your favorite shell script unmodified with OSH, please test it :) Or let me know if you have difficulties testing it.

Do you know if there is a comprdhensive list of known differences somewhere?

I've switched to using python where possible.. but I guess there are reasons not to do that?

I've switched to Perl (and Perl 6) which seem to be better suited.

Python feels way too cumbersome for day to day use, unless you do a lot of data crunching.

There are bash/python hybrids like [0], but good luck getting them on some legacy production server that still runs distro from 8 years ago, or more modern but isolated server, or... There are (were?) also some language quirks that I can't remember because it's been a while since I last tried xonsh.

[0] https://xon.sh

(EDIT: changed URL from an outdated one.)

When it comes to scripting, I do prefer Python over Bash.

The problem with first class functions in a shell is that the main duty of a shell is to fork processes. Once the shell is forked, the only communication available are inter process. All of the environment is no longer shared.

For example, in es:

  ; let (i = 0) { fn hello {echo $i; i = <={%sum $i 1}}}
  ; hello
  0
  ; hello
  1

The closure variable i is incremented after every invocation of hello, but if hello in a subprocess, then i doesn't change:

  ; hello | wc -c
  2
  ; hello
  2

This is because hello is a subprocess of the main shell when it is in a pipeline. When it is used without the pipe, then it is executed within the shell environment, not a child.

I have been using a fork of es since Dec, mine has the %sum builtin above, along with some other things: https://github.com/injinj/desh

> We applied concepts from modern functional programming languages, such as Scheme and ML, to shells

Note that ML here refers to a functional programming language[0], not machine learning.

[0] https://en.wikipedia.org/wiki/ML_(programming_language)

First order functions and the ability to override builtins. Definitely giving this a try.

I've tried out several shells as primary drivers---bash, mksh, zsh, fish. These got me thinking more generally in terms of "userspace tools" instead of just interfaces with coreutils etc, which made me wonder, can we use our favorite REPLs in lieu of a standard shell?

I've tried using Haskell's ghci as a "shell" and, fixable ergonomics aside, it seems doable. We already have interfaces to kernel syscalls, so in principle it's "just" a libraries problem. File I/O is probably the easiest, and with /proc and /sys, that's enough to replace several info-gathering system tools, e.g. df, lscpu, mount (just listing).

Really wish I could immerse myself in Haskell for a few years and see if I could learn enough to take this REPLshell idea somewhere.

This is really cool & sweet.

Reply on news.ycombinator.com