Hacker News Reader

You state the problem yourself

> [...] you just have to spend a little bit of time learning them

I can still remember the many times I tried to learn shell scripting and gave up too quickly. So 'little bit of time' might be an understatement. Actually, you need a good use-case, sufficient motivation and enough time at hand to bite through the first problems...

And show me a serious, modern programming language that is so susceptible to whitespace usage. Yes, there are some that care about whitespace (e.g. Python), but rarely it will cause a syntax error to not place a space before a bracket.

Another problem is that those text formats are a bit brittle. I am talking about having to use IFS="" and ls giving back something different to humans than to pipes. Most of the time you can solve it one way or the other but having to search frequently for solutions to such problems just sucks.

I am not saying that the old tools aren't worth learning. In fact, I love writing shell scripts (for the good parts) and can't remember the last day I haven't used a shell. But saying there are no problems to be solved is just wrong. So I think a little discussion about what could be done and some prototyping shouldn't hurt anybody.

I've now read a lot more of the comments, people are really excited about this, it has taken me a bit by surprise. There are a lot of comments in the vein of "bash sucks but it's everywhere". My dream shell is pdksh, which already exists, I would only favour bash because it has more advanced completion logic. Aside from interactive use I write everything in POSIX shell. I think popularity is being conflated with portability and that's a property that isn't valued enough. I have no problem at all using Bourne-compatible shells, I can't quite formulate it in my mind but I think they are probably misunderstood. To me, who hasn't been doing this too long, I don't think that bash sucks, I don't find bash/POSIX shell to be a bad language. In fact in terms of different things I write (not LOC) I think I write more things in POSIX shell than anything else, writing wrappers in shell is not to account for a deficiency, it's utilising that power to change your shell experience and workflow into what you want, into what fits your brain. This is my experience, I don't understand (it would be good to hear from others) why people have such a bad opinion of the current state of shells.

EDIT: wording.

When PowerShell first came out, I was very excited by the idea and then barely even gave it a chance. Instead I gained an appreciation for input and output being just text. Yeah, I might have to awk’wardly massage the data a lot of the time, but the fact that I can just run a command and see the output makes the flow of incrementally solving a problem really nice.

I’m gonna give nushell a chance, because tabular textual data seems like it still fits the bill while also making it less awk’ward to massage the data as needed for the pipeline.

I too don't understand the excitement these object-oriented shells elicit in some circles. It's exactly like you say, the job of a shell is to manage jobs and run programs. The only data processing that it performs is the processing of the command line arguments and nothing else, as far as I can tell.

yes, I have a feeling that Windows people are coming to Linux through "Windows Subsystem for Linux" to reinvent almost 50 years old concept which have been working well. Instead of using small bash which is everywhere (even in inird), more convenient zsh or Python, they are going to setup Rust environment, download gigabytes of crate dependencies and build a new shell in order to run their web development workflow script. Maybe in the 10 years, there will be Nodejs/Electron-based shell with fancy css animations and primarily mouse control. And tables will be sortable by a mouse click on a column header and searchable in real-time and there will be speech commands for common tasks...

The point of this project is standardized inputs and outputs, that's all. Working with commands is not standardized on traditional posix systems, in terms of either input or output.

The other comments about adding a "stddata" and handing this responsibility to userspace is a good one.

You shouldn't need to know a specialized DSL for each tiny core util.

You are missing something. Current shells are brittle and they're just bad tech in 2019. We've made them half decent by brute forcing tools and hacks for them.

Everyone has to reinvent the same basic tool output parsing because ls (or top or ps or...) can't spit out real structured data.

I disagree. I'm not saying that everyone should think the way I do, I guess realistically all I mean is that when I'm confronted with a parsing issue in this context I just write a shell script. Ok, this can be taken as reinventing, which I'm advocating against, but I see this as utilising the UNIX philosophy and by sticking to POSIX shell specifically (and using sed, awk, tr, et al) it is still a built-in and portable way to deal with this problem. Also I am not saying this because of some irrational submission to an old philosophy, I advocate this style of working because I find it easy, convenient and fast, and it works for me in the real world. But of course people have different ways of working and I respect that. This is just what works for me and how I see things. Definitely not meant as an attack against this project or different ways of working.

Why the hell was this downvoted?

One downvote and counting. This is just my opinion. Let's crank this up a notch. Those advocating for more advanced interfaces are missing the point. Text is all you need, no graphics or objects or widgets, just text. I have no problem with this, I enjoy it like this, we don't need an improvement. There I said it, any downvotes will be worth it, because I get to voice my opinion. In the end I will still be happily using my POSIX shell scripts, awk, sed and enjoy my life.

"Goto is all you need, no ifs or loops or dynamic dispatch, just jumps. I have no problem with this, I enjoy it like this, we don't need an improvement. There I said it, any downvotes will be worth it, because I get to voice my opinion. In the end I will still be happily using my assembler and enjoy my life."

I'm sorry, I couldn't help myself. Your comment reminds me of an anecdote I heard from the early times of structured programming. When structured programming was just gaining its feet, there was a certain class of programmers who just could not understand why people would want to write structured code. You can do everything in assembly they said, you have much more control over performance, etc. They looked down on structured programming as not "real programming".

There's a lot of benefits to adding some structure to text. I don't think that Nushell's approach is the best one, but to say that there are no problems and we shouldn't look to improve things is just backwards. We should always look to improve our tools and our craft, otherwise we would still be stuck writing assembly.

I get it, but to clarify, and maybe it's to do with the work I do, I have no problem with current shells, I think they are great. As I learn more about them and use different features I like them more. This is my experience.

Maybe you're just not very imaginative. I can see my workflow improve considerably by using a structured-io shell, and I'm already quite proficient in bash. So, I'm very happy some people are trying to make it happen.

Also, I'm not sure what your argument is, except "I don't get it"? It's fine, lots of people don't get stuff. Just move on, and maybe in a few years when it's mature, you'll see it again and go "Ah!"

Also goto being considered bad is already a very old idea. Yet it is still used, sparingly in the right places. Shell underpins a lot of stuff, it is tried and tested. Text is already structured, or can be, IFS, new lines, etc.

There's a lot of benefits to adding some structure to text.

There are benefits, true, but there are also potentially serious drawbacks. Chief among them, I would hazard, is the risk that we get locked into a format that didn't anticipate a (completely unknown) future need and we have to go back and rewrite everything again.

The beauty of text's lack of structure is that people are free to interpret it any way they please.

I keep seeing comments like this one, but just as you can parse textual data into something more structured, like the examples in the post, you can also print the structured data as text. So the tools that use and generate structured data are no less compatible because it's easy to go back and forth between the formats at will. You're not losing anything with this approach.

> You're not losing anything with this approach.

True, but you are adding something. And if you add things you don't really need, it becomes plain old cruft.

Like the others above, I understand the urge to make some things simpler by adding complexity, but for the long haul I'm convinced it's better to keep tools simple and add any needed complexity to the code you're writing (whether it's shell scripts or anything else). And then document the complexity so future you or some poor stranger can understand why the hell you did that in the first place.

But I don't look forward to reading comments like "X was deprecated so we're converting these tables back to text streams so they can be converted back to Y properly."

But isn't that precisely the problem? A lot of people are unsatisfied with the current solution which doesn't lend itself to a particular goal?

Aren't we now locked into a format (plain text) which is now becoming more and more of a pain (what type is this? is this text related to this other text?) and we're now having to go back to old utilities and add --json?

I find it basically impossible to compose programs where I have things like a list of filenames with spaces. It turns out some programs support some kind of option to separate with nulls, but they are inconsistent and not everything supports it.

So I end up having to put everything in one giant call to 'find', which defeats the compositionality of shells.

Null-terminated mode: turns out tons of tools have these: https://github.com/fish-shell/fish-shell/issues/3164#issueco... But yeah, 2nd "standard" hurts compositionality.

Default posix/bash splitting of output by spaces is very big-prone and having to type "$array[@]" all over the place is annoying. But splitting by lines is much saner. Now newlines in file names, that's just evil and should be outlawed at OS level. I think OSX did this (?)

In bash life can be easier by setting IFS (Google "bash strict mode" though I have reservations about the IFS part).

But really, its easy to fix in a non-posix shell. Fish splits by lines and interpolates variables as arrays by default.

Text is all you need

In essence, yes. Yes you can get pretty far with sed and the likes. But it's not because it's all we really need, that other things (objects in this case) don't offer an improvement and make things better/easier/more convenient. Just like we theoretically could get along fine with just the CLI, doesn't mean that it is the single best thing in the world for all possible occasions.

True, but also when using shell for scripting, when you want to use an associative array or some other "advanced" feature, from a shell point of view, well then you should move on to a different language. So in the same sense in such cases you should move on to a different language rather than trying to get the shell to do all things for all people. I am a very strong proponent of shells and shell scripting but realistically it's only designed to do certain things, those things it's perfect for, for some things it's bad for, use something else.

"Text" is a useless specification when building a program that produces data that other programs consume, or consumes data that other programs produce. There is nothing actionable in knowing that you have text to work on other that you can presumably discount the appearance of nulls in it.

Text is awful and annoying to deal with.