Hacker News Reader

I find PowerShell a delight to script in, but terribly cumbersome to use interactively. The exact opposite is true for me when it comes to Unixy shells. I think part of the staying power (whether intentional or not) of composition via character streams is that in a text-based environment, humans also operate on character streams. We can structure things in machine memory as objects or lists or whatever, but when the information hits our meat cameras it's all just characters. When I run a command, the output I see is what I'll get when I pipe it somewhere.

I feel like PowerShell is a step forward in the sense that it's a new thing with a more consistent design than the cobbled, creaky Unix environments we've become accustomed to. But at the same time I feel like it throws away one of the most intuitive aspects of the Unix interactive shells (character streams).

Couldn't there be a middle ground where command and object member names have nice short aliases, but they work on and produce objects instead of character streams? I find it questionable that the productivity of the unix shell is due to it being stream-only - it could easily fall back to a stream (using each classes representation function) if you output it to a stream receiver, like the terminal itself or a text file.

That's what I'd like to see, is to add a "--xml" or "--json" flags to any common command (such as ps, ls, grep, find, ...). And have appropriate handling in the shell for xml or json output.

Many FreeBSD utilities support a --libxo flag that produces structured output, although there isn't special handling in shells yet.

    % ls --libxo json
    {"__version": "1", "file-information": {"directory":[{"entry": [{"name":"bar"}, {"name":"foo"}]}]}
    }
    % ls --libxo xml  
    <file-information __version="1"><directory><entry><name>bar</name></entry><entry><name>foo</name></entry></directory></file-information>

Well I haven't really used PowerShell, but if it emits objects, can those objects not be printed in some form?

Is it not reasonable to think of objects as having a description of some kind. Think of URLs, for example.

    find -iname '*.c'

could emit file:// URLs rather than just filenames. Then anything can unambiguously understand the format of the output.

Yes, all Powershell objects can be cast to string ($strvar = [string]$var) or piped to Out-String.

But 'casting to string' isn't very specific. Does casting to string mean that you always get a unique identifier for the object? Does it mean you always get a dump of the object's contents?

You get a printout of the object - effectively calling its toString() method. Note that this is not a reversible nor standard process.

That's one of the main downsides of object-pipeline systems; in UNIX you can stop and serialise a pipeline to a file, then unserialise later. But you can't do this with object pipelines in Powershell because the objects have to be "live".

PowerShell itself ships with a large XML file that governs how certain types are displayed by default if none of the Format-* cmdlets are used, e.g. FileInfo objects are usually rendered in a table with just the most commonly used properties displayed, which makes the interactive use of Get-ChildItem (Aliases: dir, ls, gci) very similar to ls -l. Explicit piping to Format-Wide, or Format-List will override those defaults (they only apply if no formatting is specified).

Sure, everything can be made into strings at some point, but I've very rarely felt the need to do so in PowerShell (and regularly see on Stack Overflow that it only leads into problems down the (pipe)line if people try).

> I find PowerShell a delight to script in, but terribly cumbersome to use interactively.

That's because you can't be both! The shell wants to be both an API and a user interface, and these are two goals that are fundamentally at odds. An API should be structured, strongly-typed, unforgiving; a UI should be discoverable, intuitive and unconstrained by the needs for automation or programmability. Programmers are stuck in the idea that a command-line shell running in a terminal emulator (emulating hardware from the 70's) is the pinnacle of interaction design, which is extremely sad.

"ls" is oriented for human viewing, not composition with other tools. It's admittedly bad at that.

But there's "find", and there's the convention of separating filepaths with a null character. Because null cannot occur in a filename, this is an unambiguous list.

Programs do "take arguments" (another unambiguous list of null-terminated strings, if you quote them). Like when you specify a command to find with "-exec", for it to run for each file. That's another fine way to do composition.

Another example of composition by arguments is "sudo". It switches to root, then runs the arguments as a new process. Another example is ssh, it runs the arguments as on the remote server, and stdin and stdout still work. Check it out:

    tar -c testdir | ssh devbox sudo tar -C /opt -xv

That created an archive from a local directory and wrote it to stdout, into ssh which ran sudo on the remote system, which ran tar, which changed to /opt and extracted an archive from stdin. None of those tools have special handling for each other, but they all work together. "Interesting" filenames in the archive are no problem here either.

> "ls" is oriented for human viewing, not composition with other tools.

Is there any basis for this claim? My understanding is that many of the GNU ls options are just shorthand for things that you'd traditionally do compositionally. It even reverts to a plain, more pipe-friendly style of output if its stdout is not a TTY.

It still sucks at the job if you assume a system where the convention of not using newlines in filenames is ignored. Unix-likes are typically filled with these conventions that will qualify or disqualify any particular approach to a problem, which imposes a certain load on the users, administrators and developers that I assume isn't as severe in Powershell. An inexperienced user will constantly have to second-guess their solutions and may not experience any issues until much later, e.g. when it turns out that their script will have to accept spaces in filenames, or when their `rm -rf "/$INSTALLDIR"` needs to handle unset variables.

I'm all cool with this since it offers brevity and a profound flexibility. If you want, you can implement typed, object-passing IPC like Powershell on top of Unix. If you don't, you may be content stream-editing and grepping in file listings to get what you want in the format you need. Besides that, the most compelling case against Powershell that I can thing of is that Windows programs typically aren't very composable. It's conventional for even the most mundane and simple tools to be operated using full-fledged GUIs, if only to offer a file selector and a run button.

>> "ls" is oriented for human viewing, not composition with other tools.

> Is there any basis for this claim?

Of course. For starters, there's the fact that the date modified column format depends on how long in the past it was modified! This makes it impossible for `sort` to work with.

So suggesting that the `-t` option of `ls` could be removed in favour of using `sort` is entirely wrong.

I could go on and on. There are excellent criticisms of UNIX/Linux. This is not one.

>there's the fact that the date modified column format depends on how long in the past it was modified!

Not if you do the sane thing and use the -T option. Of course that requires the user to read another huge man page to find out that such an option exists.

I think you have failed to understand the kind of alternative people are proposing. Nobody is saying that the current ls sorting flags should be removed and replaced by passing the output of the current ls command to sort. That would be broken and difficult to use for all the usual reasons that passing around unstructured text is broken and difficult. The idea is passing more structured data between processes so that the problem you describe could never even happen. What you describe is actually the type of issue people want to solve (but in the general case, not just for ls).

Formatting data for humans to read and formatting data for machines to read are different problems. Current unix cli tools have to attempt to do both, which often means doing neither well. If you are lucky there are at least some flags you can use to make the output more structured. If you are unlucky the command magically reformats the output based on the type of device it's printing to.

I think that you are confusing my argument with another one. I'm not in the "ls should only list files" camp. I'm merely questioning the idea that it isn't oriented towards composability on the basis that even the more featureful implementations revert to a simple filename-newline format that's perfect for stream editing or grepping when their output isn't a TTY. If history is anything to go by this is not an unintended side effect, which would suggest that ls is oriented towards composition, even if poorly and unexclusively so.

Nah, Im not confusing your argument, I'm saying that the changing date format inhibits composability.

The truth is that shell is partly composable but with many concessions for friendly interactive use.

You could remove the compromises but it would not be much fun to use.

As a Linux user, who's worked for both Red Hat and IBM's Linux group: damn straight. Here's my posh profile: https://github.com/mikemaccana/powershell-profile/blob/maste...

What the PowerShell is trying to be is more of a Lisp descendant, something to which Unix has been mostly deaf.

"Unixy" invokes very specific concepts; it is not a synonym for "computer sciency", "intellectually agreeable" or "non-sucky in some ways". (Relative to some things, it sometimes is these things, though!)

Dumb character processing is "unixy", and Bash is in fact "dead unixy".

The "Unixy" approach is called "worse is better"

https://dreamsongs.com/RiseOfWorseIsBetter.html

Worse-is-better is rather a class of which unixy-approach is just one instance, or perhaps a derived class.

There is all kinds of worse is better. A $20 Shimano-branded rear derailleur made in Indonesia on your bicycle is worse-is-better; nothing to do with Unix.

Isn't that what I said?

A funny cat picture posted all over the place is called a meme, but not all memes are cat pictures.

I agree that being able to more easily give structure to data is probably an improvement, but I don't see why it's particularly "more Unix-y" - could you elaborate on what particular aspects of "Unix" or "Unix philosophy" you see it as better exemplifying?

Programs are passed arguments. As in, execve gives you a list of arguments that end up in argv and have absolutely no need to have anything to do with STDIN.

Yeah, the input is actually a list of strings (arguments), a map of strings to strings (environment variables), and a string.

(Assuming that we're not doing anything interesting with the file descriptors; I can see an argument for excluding that possibility from this discussion, but certainly it's a simplification.)

That's completely orthogonal. GNU bash has nothing to do with what arguments you can pass to GNU ls. You could use bash to run notepad.exe just as you could use Powershell to run GNU ls.

If that's not the case, if Powershell actually does integrate most of what are on Unix-like systems separate commands, then that's terrible.

>Meanwhile, Unix commands are shorter, yes, but it's a lot more difficult for me to remember, e.g. that if I want to run a command across a number of files, I have to pass the command to find.

You don't have to do anything of the sort. If you want to run things across a number of files, you need a list of those files, and you need to know what you want to run across them. That list of files can come from anywhere. Does it come from a text file? A script? Or find? Doesn't matter.

If you do get them from find, you run things on them in precisely the same way you run things on any other list of files: a for loop.

    for file in $(find ...); do
        rm $file
    done

Yeah you can also do

    find ... -exec rm {} \;

or

    find ... -print0 | xargs -0 rm

or any other number of other ways of doing it. But you don't have to. I'd probably use xargs to delete those files, honestly.

> That list of files can come from anywhere. Does it come from a text file? A script? Or find? Doesn't matter.

This is an important concept. A powershell script might need to change depending on the source, sometimes it'll be a file object and other times a string, in unix it will always be a string. This is important for composability.

In practice, I seem to end up with a `sed` or `perl` script to mangle one string format into another between processes anyway. And then it all breaks between systems because half the most common commands have different defaults between systems.

You can't even encode lists of strings in a format that's reliably understood by all UNIX programs.

Of course one any of those filenames contains a space, your first two examples break down, and in the worst case result in the wrong files being deleted. Of course one can work around that, but in reality, this quickly gets difficult, and often is forgotten. The print0 works for some cases, but not in general (e.g. when one needs to do more complex processing)

The second example is fine, to be fair. {} is replaced by a single argument, no shell magic here.

The second one doesn't fail, and the first one only fails because I wrote `rm $file` instead of `rm "$file"`, which I have done every single other time I've written the same thing this century.

Writing in first one won't save you since "in" splits by whitespace, not newline. If you really have to do that, use "find ... | while read file; do" instead.

Your first example breaks for filenames containing whitespace.

Filenames containing whitespace are, and always have been, a bloody stupid idea.

No, assuming filenames dont have whitespace is an idea that has fucked over people for a long time.

Sure, it's annoying in a shell when whitespace is used as a separator but that seems more of an issue with shells than with filenames.

Filenames should describe the file. In language, descriptions quite often require whitespace.

Whitespace should have been banned from filenames decades ago. If you need 'whitespace', use a hyphen or an underscore.

I agree, if you ever want whitespace to have any special meaning in your shell (and you do), allowing spaces within filenames just causes no end of headaches.

For interactive use, tab completion already takes care of the escaping. For anything else the actual headache is that things are passed around as strings and command argument parsing is done for the expanded string where spaces can cause different argument parsing unexpectedly. This is certainly not limited to filenames but any argument that can contain spaces and is passed to a different command without proper quoting. The very fact that it's so easy to get wrong is perhaps a sign that things are a bit broken. But not necessarily at the filesystem level ...

No, file names should describe the file. hyphens and underscores are hacks.

Systems should be better at handling files with unicode characters and spaces. Its the fault of the shell that whitespace has been overloaded, not filenames.

But whitespace was banned from file names decades ago, and look what happened: people wanted it added in. So it was added in. That's progress!

If the shell can't handle this well, the problem is that the shell is not very well designed, and we need something better.

Meanwhile, I never have problems with spaces in file names when I'm doing stuff in C++, C#, Python, etc.

Well they should be banned then, but they are not. Also it might be a stupid idea for regular use but not so stupid idea as an attack vector. Along with filenames starting with a dash.

But we must live with they.

Better:

    find ... -delete

Nice. Also:

>If you do get them from find, you run things on them in precisely the same way you run things on any other list of files: a for loop. for file in $(find ...); do rm $file done

If you know the number of files (and hence total command line length) will not be too high (i.e. not exceed the maximum length of a Unix command-line [1]), and that the filenames will not contain spaces or other problem-causing characters, you can shorten that to:

    rm $(find ...)  # or rm -i $(find ...) for interactive deletion.

This will only run rm once, instead of once for each file.

If either or both of the above conditions are not met, your 2nd or 3rd snippets are preferable.

[1] IIRC, there is a constant defined for that maximum, somewhere in standard C library's header files (maybe something like BUFMAX). I've read that early versions of Unix commands used to crash in some way (core dump or seg fault) if the length was exceeded, and seen that a few times myself. Also may have read that it was quite short initially, like 512 or 1024 bytes. People might have done something to fix that, apart from xargs, later.

Update: I googled for this phrase:

maximum length of unix command line

and looked at the first few hits; found some info:

xargs --show-limits

can give info about the maximum length, which POSIX calls ARGMAX, not BUFMAX as I said above.

and a sample output of xargs --show-limits is:

Your environment variables take up 2446 bytes

POSIX upper limit on argument length (this system): 2092658

POSIX smallest allowable upper limit on argument length (all systems): 4096

Maximum length of command we could actually use: 2090212

Size of command buffer we are actually using: 131072

There is no limit per argument, but a total for the whole command line length. In my system (Fedora 15/zsh) its closer to 2Mb. (line 4).

(from http://stackoverflow.com/questions/6846263/maximum-length-of... )

One of the hits also says that

getconf ARGMAX

shows the value, on POSIX-conformant systems.

I remember VMS fans praising their proprietary ecosystem of commands. How's that working out for them?

Powershell is Open Source. https://github.com/PowerShell/PowerShell/blob/master/LICENSE...

You only say that because SCO lost a lawsuit. UNIX was a closed source proprietary OS.

SCO lost a lawsuit because all claims brought in the suit were entirely baseless.

That literally has nothing to do with my point. UNIX was a closed source proprietary system, and so was VMS.

If you look at the commit history, the BSDs are very much derived from AT&T Unix.

Sorry, I don't know what your point is.

My initial comment was a response to the snarky "proprietary systems suck" comment, saying that UNIX itself was a proprietary design.

The original tiny Version 1 through Version 6 system may have been more or less proprietary (although provided with source for comparatively little money) but the Unix ecosystem hasn't been proprietary for a very long time. Many of the commands people think of today came not from AT&T at all, but from Berkeley.

Even as the commercial Unix platforms split, flourished, and fell apart keeping their own source more or less private Ultrix, Solaris, Irix, AIX, SCO, SunOS, OSF/1, HP/UX, Digital Unix, and others were more or less "Unix" with a few caveats about compatibility. POSIX and the SUS were formed. Systems got certified.

Then the GNU tools were written and Linux became their kernel. Then, the Berkeley vs. AT&T lawsuit ended and we got the explosion of open source BSDs - NetBSD, OpenBSD, FreeBSD, Midnight BSD, DragonflyBSD...

So yes, technically the original AT&T Unix was proprietary. That's not all there is to Unix nor has it been for decades. There is no Single VMS Specification. There are no independent systems certified as VMS compliant. You run what DEC / Compaq / HP / VMS Systems supplies and you like it or you don't. There is no Single Windows Specification. You take what Microsoft gives you or not. Unix is an open standard.

They're still using them, on a platform that's still supported && under development?

Pity the poor VMS ppl. never having the pleasure of trying to figure out what "distro" with which kernel to run on their hardware.