“It's easier to port a shell than a shell script” (1998)
gopher.quux.org
gopher.quux.org
I have gone to a number of conferences with MS employees over the years, especially MS lawyers, and with respect to software, they often seem like they have been indoctrinated into a cult. As if the existence of software outside of MS Windows does not exist and as if MS holds the key to solving all problems. But MS does not offer solutions. They create problems (e.g., for anyone who has to deal with their licensing) and then offer "solutions" to the problems they created. It is like someone who proposes to "simplify" something by introducing more complexity.
The idea of MS embracing a proper shell seemed laughable to me. Windows users expect pointing and clicking (and today they want to be able to touch the screen).
Now fast forward to 2015 and it seems like PowerShell has a significant number of users. I am sure it is helping Windows users greatly. Good for them.
But I still think it is laughable. The name still makes me laugh.
I'd rather use something like Back Orifice (which has a simple UNIX client) to control Windows from a proper shell on a computer running UNIX. Or SSH via Cygwin. Or even SUA which has ksh and tcsh. Alas no simple Bourne shell (better for writing portable scripts).
In order to write a script, you first have to change your system security settings.
More subtly, because it's comparatively new it's not very dogfooded and there isn't a rich source of examples. UNIX systems usually have lots of shell scripts in /etc for you to learn from.
If the UNIX designers had chosen some other lowest common denominator, all of these diverse utilities would be communicating over that protocol instead of ASCII. The choice to use "text" came first and the ecosystem came afterward.
"Text" makes a good lowest common denominator because it's low. Everything, including humans looking at it, can deal with text. Essentially all programming languages on all platforms provide basic primitives analogous to getline(), find_first_of(), substr(), etc. You can take text from Unix and easily do something useful with it on any arbitrary platform.
MSIL assumes an entire infrastructure that isn't universally available. You can't take MSIL from Windows and easily do something useful with it on any arbitrary platform. It requires you to have a solid .NET virtual machine for your operating system, an API for dealing with MSIL, bindings for that API in every language you want to use, etc.
There is nothing analogous to convert MSIL to. EBCDIC and ASCII both have a representation for 'A'. Only .NET has "Microsoft.Win32.Registry.CurrentUser.CreateSubKey()".
In Linux/Unix I can also pipe audio to /dev/audio for example. And image processing through a sequence of steps by pipes is quite often done. Are these objects coming with default metaparameters to make piping easier? Or are there other things implemented that might be cool?
In UNIX, you have bytes, and the underlying system doesn't know what they represent. Most tools assume they're ASCII or UTF-8 (or whatever) encoded text, but that's all they agree on. In practice, you're working almost always working with structured data - tables or lists or mappings - but you have to remember that for the output of this program, you want to cut on '=' and take -f4, and first you need to pipe it through head and tail to clean up some junk output[1], and this other thing for that program, and whatever.
[1] This is a way to get the `time` field from the output of `ping`.
That's because the object pipeline is functionally analogous to function chaining within a single runtime like in Python or Ruby, not streaming data between arbitrary executables like in Unix pipelines.
This would be a lot clearer if PowerShell advocates compared it to things that it's actually functionally analogous to, but they are trying to position it as a Unix shell competitor.
No it is not. See my response below.
Pipelines in Unix shells use OS-level functionality to connect standard streams of data, often encoded as text, between arbitrary executables written in any language.
The reason this is confusing is because one of the main tactics used in PowerShell promotion is to disingenuously compare PowerShell's "object pipeline" to Unix pipelines in order to try to make it appear to be more novel than it is, when they should be comparing it to what it's functionally equivalent to, which is function chaining within a single runtime.
If PowerShells pipelines are merely equivalent to function chaining, then you should have no problem replicating the following very simple pipeline with function chaining in Ruby, Python or JavaScript:
cat log.txt -wait | sls "error"
In case you need an explanation for what it is doing: It continously monitors the log.txt file for new lines, and selects those that contains the substring "error".But I am curious: Why is it important whether the pipelines could be implemented using function chaining or not? PowerShell is a shell where I interact with the system through commands. Yes - those commands are not the typical Unix executables - but they act as commands within the shell
Is your problem that it is not a Unix shell? You could equally well say that Unix shell pipelines are just processes connected by file descriptors. That is factually correct but does not represent the true utility of pipelines.
PowerShell pipelines are not file descriptors. PowerShell commands do not execute in separate processes. But PowerShell commands are versatile and can be combined in a way analogous to Unix shell pipelines where the output of one command is consumed and acted upon by the next command.
The object of a operating system command line shell is to expose the operating system features to the user of the shell. Why does it matter whether it uses Unix file descriptors. Even if PowerShell pipelines were equivalent to "just" chained functions, why does it matter?
I get the point that you need the .NET runtime to run PowerShell, because it is implemented using .NET. At what point do we consider a runtime part of the operating system. When it is intrinsically distributed with the operating system and cannot be uninstalled?
Of course, this kind of piping is present in nice languages that precede the PowerShell.
A one page Lisp macro will give you a left-to-right syntactic sugar for filtering. Clojure has a threading operator, Ruby has cascades of dots: object.{ blah }.foo().bar() ... and so on.
Non-ASCII has been supported for a long time.
I used "ASCII" instead of "text" to stress that "text" is just a protocol like anything else. The UNIX I'm running is set to use UTF8.
PowerShell's object pipeline is functionally analogous to function chaining in languages like Ruby, Python, and JavaScript, not Unix pipelines. PowerShell's object pipeline does not exist outside of the .NET runtime and you can not stream objects between arbitrary executables outside of the .NET runtime.
You can't compare an apple to an orange and say one is "objectively better," no matter how hard PowerShell advertisers try to pretend their apple is an orange.
No, in those languages function results are bound upon returning from the function. PowerShell's pipeline would be more like function chaining where each function is a co-routine that can yield partial results, which would make it considerable more complicated.
So your attempt at trivializing PowerShell fails from the start.
For your information, here is roughly how a PowerShell pipeline is set up, processed and torn down:
1. BeginProcess is invoked for each command in the pipeline, starting with the last command of the pipeline, working towards the first. When each command is initialized this way, it can start receiving objects from preceding command on the pipeline. Each command may start produce objects already during this phase, as all subsequent commands are ready to process them. If it produces objects, ProcessRecord of the subsequent command is invoked immediately. 2. ProcessRecord is invoked on the first command of the pipeline. If it yields objects, ProcessRecord of the subsequent command is invoked for each object, thus "pushing" objects through 3. EndProcess is invoked for the first command of the pipeline. It it yields objects, ProcessRecord of the subsequent command is invoked for each object. When EndProcess completes for the first command of the pipeline, EndProcess is invoked for the subsequent command and so forth until EndProcess has been invoked for all commands of the pipeline.
As you can see, each cmdlet has 3 phases: initialization, processing and completion. All cmdlets (functions?) become active at the same time, with the control flow passing to each command (function?) multiple times during each phase. This can be implemented in any turing complete language, but function chaining is a lame attempt at trivializing it. Throw in reactive sequences and it may get closer.
Consider how Sort-Object (alias sort) is implemented: It does nothing during BeginProcess, during ProcessRecord it adds to an internal collection but does not yield any objects (yet), during EndProcess it sorts and yields all of the collected records. This is clearly analogous to how sort in byte/text stream is implemented: When the process starts it does nothing but initialize and starts listening on stdin, for each "line" on stdin it adds to an internal collection, and when stdin is closed it sorts the collection and writes the "lines" to stdout. 3 phases.
Even so, it doesn't really matter how it is implemented, a shell is inherently about how you interact with the shell.
> PowerShell's object pipeline does not exist outside of the .NET runtime and you can not stream objects between arbitrary executables outside of the .NET runtime.
You are locked in the Unix view of the world and is unable to consider that long standing assumptions about "the way to do it" may need a refresh. If you want to leverage objects for system management then yes, you need a system-wide object model. Unix was not built with an object model. Windows was sort-of build with an object model (handles) but it is not versatile enough as a system-wide management model.
However, the good thing about objects is that they are abstractions which can easily wrap existing operating system concepts, such as files, processes, access control lists, drivers, users, groups, network connections etc. .NET is a mature object model and with a lot of tooling. Building PowerShell so that the abstractions over network adapters, users etc become represented by .NET classes enable a lot of tooling and infrastructure right from the beginning.
No matter how much you want to try to talk around it, PowerShell is a .NET CLI, the object pipeline exists within its .NET runtime, and the commands are instances of .NET classes. Unix shells work primarily with arbitrary executables outside of the shell runtime using OS system calls to create pipelines with standard streams.
Here's the cold hard fact stated yet again: PowerShell's object pipeline is functionally analogous to function chaining in languages like Ruby, Python, and JavaScript, not Unix pipelines.
Ok, what then is the functionally analogous function chaining of Ruby, Python or JavaScript to the following:
cat log.txt -wait | sls "error"
A simple function chain will do.They are honestly too long.
I haven't used inforno-shell, es or rc, so can't comment on them directly. That said, if they are easily comparable to bash, ksh, etc, then you probably owe it to yourself to see what powershell is about and why it's fundamentally different.
I specifically contrasted them against Bourne or C-like shells, so I thought that much was clear that they are not.
I have tried PowerShell. I think this comment summarizes it well: https://lobste.rs/s/vzjalp/why_i_dislike_systemd/comments/sh...
If you want to run a script simultaneously on a number of hosts, you use the Invoke-Command cmdlet (or it's alias icm).
If you want to stop a process, you use Stop-Process (or it's alias kill).
To copy files or directories you use the Copy-Item cmdlet (or one of it's aliases cp and copy).
Hence, to stop WINWORD on a host1, host2 and host3 and then copy a directory from host0 to each of them, you would write:
icm host1,host2,host3 {kill -name WINWORD; cp \\host0\C$\source C:\dest }
As simple as it should be!The commenter is correct that you cannot start an application in another session from remote. However, this is a security restriction that prevents cross-session contamination and shatter attacks. It is a (security) limitation of Windows, not of PowerShell.
Under Windows, all services run in session 0, the interactive user run in session 1, and if more users log on (e.g. remotely through PowerShell) they are each created in their own session. Windows put severe restrictions on what programs can do cross-session - even if the processes run under the same user account: They cannot send messages, interact with the desktop windows, inject code, start threads etc.
The commenter also alludes to other security settings that you'll encounter when you first start using PowerShell: - Execution of script files are disabled by default. You need to use the Set-ExecutionPolicy cmdlet to allow script execution, and for desktop editions you need to enable remote management. Both has secure-by-default values.
To enable script execution:
Set-ExecutionPolicy RemoteSigned LocalMachine
I.e. scripts that originates from the Internet (downloaded through a browser or received through mail/IM/etc) or other untrusted zones require digital signatures to run; locally authored scripts are allowed to run with no digital signatures.To enable remote management run this command:
Enable-PSRemoting
On Windows Server versions it is now assumed that the administrator will want to use remote management, so it is enabled by default for servers. For desktops you'll need to set it through a group policy or with the command above.By asking if inferno-shell and friends were easily comparable with C-like shells I was attempting to determine if they were additionally hard to compare with powershell, and infer from that your unfamiliarity with powershell. That, obviously, was a stretch.
It worked better than one might expect. You'd get emails about failed backups (backup flag files removed after a successful backup), backups were versioned, and restore was quick & easy (execute a simple script from the shared drive, then sys c: from a boot disk).
Ah, but no; I would end up saying some things that would have been too ironic to come from a Microsoft person, such as:
- why are you Unix people making a POSIX standard which covers the shell command language, if you're going to insist on one reference implementation being used on everyone? Just dump the source code listing of that into a document, add a few comments here and there, and call it the standard.
- isn't it healthier to have multiple implementations rather than a "monoculture"? Do we all use the same C compiler?
- the MKS shell could be made to diagnose and reject the use of Korn shell extensions, helping the programmer write a more portable script.
- wait, did you say your name was Korn, or Bourne? I didn't quite hear you from the back. If you're Mr. Korn, I would appreciate it if Mr. Bourne would now stand up, if present, and ask you why you don't just use the shell with his name on it.
a) Red herring. No one is talking about a reference implementation of Unix shells, but about Korn shell compatibility in particular.
b) Red herring. Don't imply it's a Korn or Korn-compatible shell - that's the issue here.
c) Red herring. Though the conversation was never recorded, it's implied that nothing was said about "portability", per se, but in particular compatibility with the Korn shell language specification.
d) Red herring and an insult. Stephen Bourne didn't write the first Unix shell. Ken Thompson did. Is your implication that no one should iterate or create new designs? Microsoft creating a Unix shell is fine, but the issue at hand is that it wasn't a Korn shell like they claimed it to be.
https://en.wikipedia.org/wiki/MKS_Inc
MKS claimed to have a Korn-shell workalike, AFAIR.
> The MPM asserted again that the shell was pretty compatible and should work quite well
which sounds reasonable, and isn't an assertion that MKS has the Korn shell in it such that it is 100% compatible. (And could anything even be 100% compatible, if not running on Unix? The Korn shell source ported to DOS woudln't be 100% compatible with the Korn shell, just "pretty compatible" so that many things work "quite well".)
They probably had some good reasons to with MKS, because in fact it isn't easy to port a shell: not to the DOS environment. Or Windows. The minimal example of doing this reasonably well might today be the MSYS environment used by MinGW, which has to re-create significant infrastructure (a whole C library from the ground up and other things).
Maybe Microsoft also had favorable licensing with MKS, avoiding the AT&T license.
Any shell that isn't compatible with the _reference_ implementation of that shell is flawed.
A reference implementation accompanying a document can be at odds with the specification. If a document says that a reference implementation shall be held correct if there is any discrepancy, then specification is greatly diminished in value. It's obviously just a best effort to describe the reference implementation, and not a forceful standard.
What if a buffer overflow is discovered in the reference implementation? Everyone must have the same one to be compatible?
In the discussed scenario, it appears Microsoft were working with the MKS vendor, relying on them to provide compatibility.
Microsoft eventually bought something called OpenNT in the late 1990's and renamed it Interix.
Microsoft also had their own licensed and branded AT&T Unix called Xenix in the 1980's:
https://en.wikipedia.org/wiki/Xenix
They helped make Unix popular: Xenix had the biggest installed base among Unixes.
In the mid 1990's, I visited a computer animation school which had a classroom for learning Unix. It was equipped with some vanilla PC box which ran Microsoft Xenix, and about a dozen terminals attached to it. Students would sit at these terminals and follow the instructor along, running shell commands, editing with vi and so on.
Edit: here's the gopher link for it gopher://gopher.quux.org/0/Humor%20and%20Fun/Microsoft_KSH.txt|/MBOX-MESSAGE/1