Shell, Awk, and Make Should Be Combined (2016) | Hacker News Reader

Shell, Awk, and Make Should Be Combined (2016)

oilshell.org

35 pointstosh9 comments

Open articleView on HN

Already done decades ago. It is called perl

I was just about to write that. But Perl is not in fashion anymore, will be reinvented in a decade.

Perl isn't a shell!

https://www.oilshell.org/blog/2021/01/why-a-new-shell.html#a...

It also doesn't have any incremental build tool builtin. (not that it should, and not that Oils does)

It does have awk though!

Though these days I see both awk and make much more than I see Perl. Something interesting happened there

(author here, wow this popped up)

I still think this is true, after so many years :) Shell, awk, and make all started out as limited DSLs, and then they grew bad languages on top.

Also, shell, awk, and make are still weirdly popular, and even increasing in popularity, in 2024 vs. 2016. I honestly think this is due to a "design deficit" -- in some ways we are designing worse systems software in the 2010's and 2020's, compared to 1980 or 1990.

Shouldn't something better already exist?

This is probably due to incentives -- there is a negative incentive for interoperability. (The Internet itself of course being the ultimate in interoperability; we're missing that in software - https://www.oilshell.org/blog/2022/02/diagrams.html )

---

So combining them makes sense, but in retrospect, it isn't trivial! I made comments on this last year:

Review of Awk and Make after 6 years - https://www.oilshell.org/blog/2023/06/ysh-sketches.html#revi... (some crisp opinions)

Other thoughts:

There are contributors interested in the Awk paradigm, so it may still happen. Though I sorta narrowed the slogan to: "Shell, Python, JSON, YAML" in the last post - https://www.oilshell.org/blog/2024/02/release-0.20.0.html#sl...

Ideally we would have "everything that is known to work in Unix" but a little better, with problems fixed, and more integrated with structured data.

And awk is part of that set. But there's no promise -- it depends on what people want to work toward.

I believe the main limit is contributors, not the codebase or design itself. I mentioned contributions recently:

Ideas for Contribution and Feedback - https://www.oilshell.org/blog/2024/01/release-0.19.0.html#id...

Also, good news as of this weekend -- Oils is completely translated to fast native code via C++. The next release will show ZERO test failures in C++ vs. Python, out of thousands.

So you can download a tiny tarball with pure C++ and get working shell, with rich data structures, and an upgrade path from legacy, in about 30 seconds. Screencasts here: https://www.oilshell.org/blog/2023/12/screencasts.html#appen...

That's an interesting perspective. I would be surprised though how long any change would take to propagate given the big intertia behind all of these tools. The fact that you can find awk and (ba)sh on nearly all Linux machines makes it the default choice when writing a portable script or oneliner.

Hm, I guess my response would be that this sorta begs the question. WHY are awk and sh/bash the "best" thing on nearly all Linux machines, the default choice?

Why didn't something better come along in the 90's or 2000's or 2010's that made them less appealing or obsolete? That something could be installed "everywhere" by now.

Not to say there hasn't been progress of course -- I think say fish, ninja, jq, etc. are great projects. They also are installed in most places, and ninja and jq aren't that old.

IMO, it's just a lot less than you would expect given the huge growth in computing in the last 20-30 years.

> WHY are awk and sh/bash the "best" thing on nearly all Linux machines, the default choice?

From my experience with of them: They are good enough. Leapfrogging software with incremental advances only works sometimes and the "standard env" is already very advanced.

I do appreciate the features provided by, e.g., ripgrep over GNU grep but I can likely get the job done without it.

There are ofc some exceptions, like jq for JSON handling, but I have only really seen that used by developers. However I've rarely seen it on legacy/ production machines because it often makes more sense to use Python/ Perl at this stage.

Situation: Wow. Why three languages like this?

Solution: We should come up with a combo language that does it all.

New Situation: Wow, now we have four languages to deal with.

https://xkcd.com/927/

A defining feature of Oils is that it's compatible, not incompatible. It runs your old code because it's an upgrade of POSIX shell and bash.

It also runs external executables like awk and make, just like any other shell.

See FAQ - http://www.oilshell.org/blog/2021/01/why-a-new-shell.html

This is a lazy and tired comment BTW.

Reply on news.ycombinator.com