Hacker News Reader

Comment by chubot | Hacker News Reader

That's what Oil is, except it also runs bash scripts and lets you upgrade seamlessly.

That is, bin/oil is the same as bin/osh with a bunch of shell options on, including 'shopt -s simple_word_eval' which eliminates a lot of the quoting hassles.

You can try it now, but some things will be cut out after optimizing the prototype interpreter (in the name of time):

You Can Now Try the Oil Language http://www.oilshell.org/blog/2019/10/04.html

I'm looking for help too: http://www.oilshell.org/blog/2019/12/09.html#help-wanted

----

Also, I think there is more room than ever, because both Python 3 and Perl 6/Raku are worse for shell-like tasks than their predecessors! Mainly because of startup time and the string abstaction.

We discussed that a couple weeks ago here, and I think it's on the blog somewhere too:

https://news.ycombinator.com/item?id=22156151

View on HN

> Python 3 and Perl 6/Raku are worse for shell-like tasks than their predecessors! Mainly because of startup time and the string abstaction

Startup time can always be better, agree. But full support for Unicode is needed in this day and age, and that brings overhead, whichever way you do that, and especially so if you want to do it 100% correctly. If you're still living in an ASCII world, then by all means, go for it.

Additionally, in this world of source version control, and virtually unlimited diskspace for source-code, I think using scripts, rather than one-liners, will help better in maintainability rather than using one-liners.

Oil has better Unicode support than Python 2 or 3 for shell-like tasks, because of the way file systems, libc, and the kernel work.

Explained here:

http://www.oilshell.org/blog/2018/03/04.html#faq

which links:

http://lucumr.pocoo.org/2014/1/9/ucs-vs-utf8/ (by Armin Ronacher)

The summary is that there are two kinds of Unicode support:

- UTF-8 based: Go, Rust, and Oil (and Perl 5 it seems, not sure about Raku)

- array-of-codepoint based: bash/zsh/libc, Python, Java, JavaScript, Windows (JavaScript notably requires surrogate pairs, Python used to have build time configuration, now has complex storage heuristics, etc.)

This isn't a theoretical problem -- the Unicode problems in the other comment thread I linked are real and show up in practice.

Unicode support of Raku is grapheme based (NFG or Normalization Form Grapheme). The unicode theory: https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundarie... , about the implementation: https://6guts.wordpress.com/2015/04/12/this-week-unicode-nor...

Perl (5) is array-of-codepoint based, at the logical level. Those codepoints might be internally stored as their encoded-to-UTF-8-bytes, or they might not, but this does not affect the usage of the string.

Many don't really understand the string model (because many don't really understand character encoding) but it comes down to: all input and output is going to be bytes, which by default is stored as the codepoints sharing the ordinals of those bytes, and there are several mechanisms by which you can manually or automatically decode/encode those byte ordinals to the represented characters; for most text processing, you do this on STDIN and STDOUT.

"Full support" for Unicode is almost never what you want unless your program literally interacts with font rendering, and then it's not even close to enough. Most of the time the important requirement is that you don't mangle data as it passes through your program, which e.g. early py3 spectacularly failed and is still not better at than py2 in the very common case where your input is not actually necessarily well-formed Unicode, but just an unrestricted byte array that by convention usually contains utf-8.

Most of what people hope for Unicode support to achieve beyond that (case-folding, homoglyph elimination, &c.) is not only not achieved, but not even possible.

Once Oil has better documentation I'll probably give it a try. The documents outlined in "Docs I Want to Write" seem promising and are still only visible as drafts in Zulip which requires logging in.

That's fair, in the meantime this post gives some more examples:

http://www.oilshell.org/blog/2020/01/simplest-explanation.ht...

as well as other posts tagged #oil-language:

http://www.oilshell.org/blog/tags.html?tag=oil-language#oil-...

Reply on news.ycombinator.com