Hacker News Reader

Comment by nojvek | Hacker News Reader

I’m surprised JessFraz who is employed by Microsoft doesn’t talk about powershell pipes at all.

Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects.

It makes working with pipes so much fun. In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`. In poweshell, ls outputs stream of file objects and you can get the field you want by piping to `Get-Item` or sum the file sizes, or filter only directories. It’s very expressive once you’re manipulating streams of objects with properties.

View on HN

> In Unix you have to cut, awk and do all sorts of parsing to get some field out of `ls`.

I'm guessing you've mentioned using `ls` as a simple, first-thing-that-comes-to-mind example, which is cool. I just wanted to point out that if a person is piping ls's output, there are probably other, far better alternatives, such as (per the link below) `find` and globs:

https://unix.stackexchange.com/a/247973

That has been my experience, at least.

If you're using cut and awk to get a field out of ls, maybe what you want is actually stat(1)?

She is actually employed at GitHub now.

https://twitter.com/jessfraz

Which is owned by...

Still MSFT, in a way!

The downside of objects is performance due to the e.g. increased overhead a file object carries. Plus the exact same can be argued with everything being bytes/text on Unix, in that it make everything simple but versatile.

> get some field out of `ls`

If you're parsing the output of ls, you're doing it wrong. Filenames can contain characters such as newlines, etc, and so you generally want to use globbing/other shell builtins, or utils like find with null as the separator instead of newline.

I've been following her on twitter for awhile. She went to MSFT only a couple years ago. From what I understand her expertise is in Linux.

> Powershell pipes are an extension over Unix pipes. Rather than just being able to pipe a stream of bytes, powershell can pipe a stream of objects.

Unix pipes can pipe objects, binary data, text, json encoded data, etc.

The problem is it adds a lot of complication and simply doesn't offer much in practice, so text is still widely used and binary data in a few specific cases.

What happens when there are upstream changes to the objects? Does everything downstream just need to change, and hence, the upstream objects returned by programs need to be changed with care? Or is it using something like protobuf, where fields are only additive, but never deleted, for backwards compatibility?

Or are the resulting chain of pipes so short lived, it doesn't matter?

What happens when you rely on non standard behaviour of unix tools, or just non-posix tools that change from beneath you?

I’m not saying that this makes powershell pipes better/worse, just that this problem isn’t unique. Microsoft tends to be reasonably committed to backwards compatibility but I don’t know the answer to the question

I think they were talking about objects at execution time, not version compatibility.

I don’t really understand what this means. The parent specifically writes “backwards compatibility.” The only other thing I think you might be referring to is if the reading process mutates the objects does that somehow affect the writing end of the pipe but I think that doesn’t make sense from a “sane way of doing inter-process communication” standpoint. Is there something else you are referring to? Could you elaborate please?

Powershell is a shell language, everything is dynamic, and does not error when you access a non-existent property on an object.

Don’t you have to rewrite every single program to make it able to read those “objects”?

You don't have to rewrite them; just write them. Every new OS/language needs its tooling/libraries to be written.

Never parse ls.

> Never parse ls.

I have heard this several times, but either I do not understand it or I disagree. Do you mean parsing the output of the ls program? Parsing ls output is not wrong, the program produces a text stream that is easy and useful to parse. There's nothing to be ashamed when doing it, even when you can do it in a different, even shorter way. I do grep over ls output daily, and I find it much more convenient than writing wildcards.

One can certainly do fine by grepping ls output in one-off instances, but I'd be really hesitant to put that in a script.

For given paths, stat command essentially lets us directly access their inode structs, and invocations are nicely concise. The find util then lets us select files based on inode fields.

Both tools do take a bit of learning, but considerably less than grep and regexs. Anyway, I've personally found find and stat to be really nice and ergonomic after the initial learning period.

Reply on news.ycombinator.com