Oil 0.9.4 – User Feedback
oilshell.org
oilshell.org
If you know Python and shell, or C/C++ and shell, you can almost certainly contribute something!
I should add that if you've gone through Crafting Interpreters (https://craftinginterpreters.com/) like I've seen so many people do, Oil is nearly identical architecturally to the first interpreter in Java, even though it's written in (statically typed) Python. The ideas apply directly, but this is a bit more "real world", with more detail, and with the addition of about 20 syscalls to implement a shell!
> It's written in Python, so the code is short and easy to change. But we automatically translate it to C++ with custom tools, to make it fast and small. The deployed executable doesn't depend on Python.
Why Python developers will do absolutely anything to work around the limitations of python but not use a different language.
I personally have moved on to a Nim + Lua combo (depending on use case), because I can see your point. But I will always love Python.
I too am looking into nim more and more recently because I don't want to write code without static typing anymore and it has very nice syntax and native compilation.
Each of the tools you mention have their good part wrt/ one of those four categories, and the rest is an afterthought.
They did not end up doing one thing well, but one thing well and the other three too. Not as well.
The way, or one way to change that would be to borrow the overall idea and semantics from each "good" implementation, but to limit it to just that; and build four separate projects under one community, so that there is pollination of knowledge and efforts, but no cross-contamination of goals and semantics.
I'm not sure I am the person to start something like that.
Isn’t it a perfectly understandable reason? We regularly pan C++ on here for its smorgasbord of implementation options for any given thing, and the complexity, customization, and fragility of a C++ code base always seems to be high especially for non experts in the domain.
So, accordingly, people who know how to maintain code in a simpler and more flexible setup and can still convert to C++ for performance are taking a probable performance hit relative to “native” C++ (whatever that is in 2021), in order to maintain their tool with more ease.
It’s very silly you move to slandering python developers and not contemplating the trade offs meaningfully
Furthermore, there is no shortage of statically typed compiled languages to choose from. Go, Rust, Java(with GraalVM) or my recent favourite, Nim comes to mind.
I'm greatly enjoying Fish, I really appreciate `history merge` and fish_add_path, and only hit the occasional muscle memory issue with things like $(expr) should be written as (expr), but Oil looks far more like Bash++, which I'm in favour of also.
I talked about that here: https://news.ycombinator.com/item?id=28547185 (also linked at the end of this blog post)
Oil has the architecture and possibility to be something like fish (a great interactive shell). It has a nascent headless shell mode, e.g. for being driven by GUIs, which no other shell I know of has. As far as I recall, the Warp terminal that appeared here has to parse the stdout of a shell session in a weird way to deliver some nice UI features. The headless shell provides a clean API for that (or at least it should; people need to test it).
And Oil uses a principled and accurate parser is for autocompletion. No other POSIX shell does this.
But personally I'm putting that as a low priority in favor of the "batch" use case. I think the "killer use case" will probably be to have a better language for continuous builds and "cloud automation", which on current platforms like Github, Gitlab, and sourcehut use a lot of YAML and shell.
I use a lot of this automation myself, and I expect it to dramatically increase in the software world in the next decade. And I expect new platforms to appear.
There a lots of people complaining about playing whack-a-mole and with YAML and shell errors, waiting for a "git push" every time, etc. Also, these systems are profoundly inefficient. A better language could be the foundation for much better systems. Over the last year, the blog has hinted at this kind of thing pretty frequently. Get in touch if you're interested :)
I feel the pain of this.
In fact, it often goes like this:
1. Start in Yaml since things are simple enough
2. Use a multi-line Yaml string when things get worse
3. Extract to a shell script when things get even worse than that
Except things are still painful after step 3: I’m now writing bash. Shellcheck helps but still hit gotchas. Other team members avoid it completely.
The problem with “just use Python” is that it’s not as straight forward as doing that. We have to set up venvs and make sure deps in requirements.txt are installed. And we’re not using it on the project we’re building (the languages we are using simply are not as good for scripts).
What is needed is a better shell, where we don’t feel as apprehensive about using shell scripts for project automation and those shell scripts are maintainable by everyone on the team.
I could internalise enough bash warts to write and maintain them, but I can’t expect to burden my team mates with crap that’s far more painful than it should be to maintain.
So yeah, I’m now at a point where:
1. I’ll try a new shell and try for pastures greener
2. I’m sick to death of debugging half-baked Lisp-as-Yaml for automation and if the guys who solve 1 also solve this, I’d owe a debt I could never repay.
Yup, this reminds me of how the Oil project started.
I was getting a lot of work done in shell -- invoking tools I wrote in other languages. And sometimes my coworkers were interested in that, although sometimes they avoided it.
But either way, I couldn't really recommend shell to them "with a straight face". It's just too weird and unfamiliar. These are largely Python, C++, and occasionally R users. I saw people write "shell scripts" in C++ simply because that's the language they knew.
So basically Oil is evolving shell into a language that will be more familiar to Python and JavaScript users (and yes even C++ or Rust users), which is the tagline on the home page.
By default, typos in variable names are worse than in Python and worse than shell with set -u: they result in nil rather than an immediate exception.
The 1-based indexing is something people complain about when coming from almost any other language (Python, JS, C, C++, Go, Rust, etc.). The key issue with shell is that most people write it without studying the language.
I'm also unsure of Lua's stability guarantees. It's an embedded language that can change on every version. I know it has broken compatibility at least 4 times, because they're on Lua 5.x. You're expected to change your programs when you upgrade version of Lua, because it's an embedded language by design. It's very different than a shell.
I have heard of people using Lua for "private" shell-like tasks, but I've never seen it in a real codebase, even though Lua is very popular and well documented.
LuaJIT/Lua 5.1 are not going anywhere. Development is done. NeoVIM has made LuaJIT it's default scripting language and has a pretty great FAQ[0] on the topic.
[0]https://github.com/neovim/neovim/wiki/FAQ#why-embed-lua-inst...
I am curious about oil but I found the "why use oil shell" explanation also slightly underwhelming. I'm also a bit concerned that its sales pitch seems to be almost entirely about "fixing" bash so that it is a better/stricter language. This is setting off alarm bells in my head coz it sounds like the exact same merry go round as last time.
Like the author I heartily agree bash is a shit language and I'll never use it to script anything over 25 SLOC, but the shell is so embedded at this point that I'm loath to switch unless there are some really big wins and/or the transition is seamless. I'm unconvinced of either in this case.
I do get the sense that there's some depth to this shell that isn't readily apparent from a surface glance and there is a lot that has been very carefully thought out, but whether that translates into some notion of "abstract language purity" or something that will actually benefit me on a day to day basis isn't clear.
I also don't quite get this:
>Why not use only Python? Because I'd rather write 100 lines of shell + 100 lines of Python than 500 lines of Python. This is basically the Unix Philosophy, and I didn't understand it until after I'd been programming for many years.
I don't think I've ever written a python program that could be decomposed into 100 lines of shell + 100 lines of python. I'm curious as to what kind of example the author had in their head when they wrote this.
In general I find python programs to be no longer than an equivalent shell, safer and often shorter, largely thanks to the library ecosystem.
The standard shell experience is decades old, and makes it cumbersome to keep up with the pace of today. The appeal of zsh/fish is the user experience and extensibility. I have a shell config using zinit [^1] that installs my dev environment for me (fzf, tmux, nvim, and my zsh-plugins) as long as zsh exists on the new machine. While one may argue that they prefer strictly standard bash so that everything is POSIX compliant, you just have to ensure that your scripts are run with bash or whatever shell the script was made for.
Because I forget. Because some scripts are run from others. Because I don't want to check the shebang of every script I run.
I would probably deal with the annoyance if I could see a really compelling reason to switch.
>The standard shell experience is decades old, and makes it cumbersome to keep up with the pace of today.
Supposedly but the features I'm missing out on seem to either be cosmetic or something I've been doing in bash for years.
>I have a shell config using zinit [^1] that installs my dev environment for me
I have something similar in bash.
http://www.oilshell.org/release/latest/doc/oil-language-tour...
The OSH part is mature, but the Oil part is less mature, which is why there is more written about OSH. (Yes it's a bit confusing that the whole project is called Oil, and it contains a part called the Oil language.)
----
I need to write a blog post about it, but basically ALL my Python programs are driven by shell scripts, and it saves a ton of code.
The oilshell.org website (which is bigger than it seems) is written this way: http://www.oilshell.org/site.html
All the tests and benchmarks are written this way, links here: https://www.oilshell.org/release/0.9.4/
There are a lot of examples in the benchmark/, test/, build/, etc. dirs: https://github.com/oilshell/oil/tree/master/benchmarks
Factoring into processes is an important design skill. It lets you write a lot less code and thus create more reliable and stable systems. It's basically policy vs. mechanism, or control plane vs. data plane. These are very important but seems to be out of fashion for some reason. So yeah I have been behind in writing about them, but there are some under #shell-the-good-parts
https://www.oilshell.org/blog/tags.html?tag=shell-the-good-p...
I kinda want to make a funny web page that insists that
Oil Shell != Shell Oil
just like
Breakfast (meal) != Fast Break (basketball play)
Blackjack (game) != Jack Black (actor)
Bookcase != Casebook (type of textbook for law students)
Backdrop (a kind of picture) != Drop back (football play apparently)
upturn (improvement) != turn up (to be found)
More examples appreciated :)
* love-in (peaceful public gathering focused on meditation, love, music, sex and/or use of psychedelic drugs) != in love
* takeout (North American word for food intended to be consumed away from a restaurant's premises) != outtake, a scene or song not included in the final film/album, although sometimes funny ones appear in the credits.
Great takeout: https://m.facebook.com/ninsbin
Great outtake: https://m.youtube.com/watch?v=bOpePo5Lx_I
Also thought of overturn (reverse a decision) / turnover (accounting or employees)
You got a typo there, compliment != complement
社会 → "shakai" → society, the world, public
... and probably thousands of others from Japanese.
call in != incall
insight != sight-in
in turn != turn in
instill != still in
upend != end up
You can't reach perfect optimization for searchability without picking a name nobody would pick before now or in the future, and anything less than perfect is arguably subjective, and/or circumstantial, in that it depends on how you formulate your questions, and what the rest of the world has been doing up until you pressed "search".
Search query disambiguation is a good skill to learn, e.g. "fish shell documentation", "go lang machine learning", "Airflow DAG best practices", "GCP Dataflow optimizations".