Hacker News Reader

Comment by Analemma_ | Hacker News Reader

PowerShell is my guilty pleasure of the computing world. Once you've piped strongly-typed objects around between your shell commands, text scraping seems barbaric by comparison. The problem, of course, is that you can't use all that power for much, since the pool of PS-compatible tools and utilities is much shallower than it is for Unix shells. I'm really hoping this will help spur a new wave of PowerShell-compatible tools.

Come to the dark side, we have (strongly-typed) cookies.

View on HN

After a number of years using PowerShell, my conclusion is the opposite: text "scraping" is just better for most cases.

Normal shell usage means doing a lot of one-shot pipelines incrementally. This is just easier and faster to do when passing text around because you can look at it and not have to inspect some object for it's type and what attributes/methods it provides. Parsing the text is not the problem here (although many people think it is), reasoning about what we're trying to do is.

And the over-verbose syntax doesn't help.

I compare this to human languages. It would be tempting to create a language with minimum ambiguity and clear cut concepts, but it wouldn't be practical. I guess PowerShell as an interactive shell is somewhat like this.

For automation, PowerShell is nice. The language sits between a shell script and going the Perl/Python route, but I still prefer shell scripts for simpler things and Perl/Python for more complex tools.

Having said this, PowerShell is the only sane choice on Windows and has made my life easier by no small amount. I never enjoyed managing Windows servers with their focus on GUI tools and their terrible CLI. PowerShell changed that and even "invaded" products like SQL Server and Exchange making them also nice to manage.

I'm not entirely convinced that "plain text" is "simple." For one, utf-8 is a variable-length encoding, which can cause all sorts of subtle bugs and sometimes leads to security issues because somebody failed to parse a character correctly somewhere. On top of this, using plain text means that every program has to choose its own control characters, essentially an encoding within an encoding. This is great for readability, but it's not always easy to know how a certain program will treat certain corner cases, or even really just at all. I'm not sure what you mean by "you can just look at the text," because unlike something that encapsulates functionality alongside the data, text input into a program might do anything.

For one, utf-8 is a variable-length encoding, which can cause all sorts of subtle bugs and sometimes leads to security issues because somebody failed to parse a character correctly somewhere.

UTF-8 is nice in that extended characters, despite consisting of multiple bytes, will never contain a low-ASCII character amongst them. Unless you're dealing with byte offsets, splitting and scanning UTF-8 strings by delimiters works just like ASCII.

because unlike something that encapsulates functionality alongside the data, text input into a program might do anything.

On the other hand, that "encapsulated functionality" is even more hidden and non-obvious. Unlike a stream of bytes that you can easily inspect simply by dumping them into a file, PowerShell objects are far more opaque entities.

Is there no trivial way to print a PowerShell object? As JSON or XML or an ascii table formatted into columns? That seems like a point of friction, then, I'd agree.

By default, an object at the end of a pipeline will be printed to the screen in tabular format.

    (australis) ~\Desktop % Get-ItemProperty e
        Directory: C:\Users\Dustin\Desktop
    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    d-----        8/12/2016   9:06 AM                e

I believe types can specify which columns to display by default; if you want more info, there's always `Format-List`:

    (australis) ~\Desktop % Get-ItemProperty e | format-list
        Directory: C:\Users\Dustin\Desktop
    
    Name           : e
    CreationTime   : 8/11/2016 10:58:56 PM
    LastWriteTime  : 8/12/2016 9:06:13 AM
    LastAccessTime : 8/12/2016 9:06:13 AM
    Mode           : d-----
    LinkType       :
    Target         : {}

It's also possible to format any object via `ConvertTo-JSON`:

    (australis) ~\Desktop % Get-ItemProperty e | ConvertTo-JSON
    {
        "Name":  "e",
        "Parent":  {
                       "Name":  "Desktop",
                       "Parent":  {
                                      "Name":  "Dustin",
                                      "Parent":  "Users",
                                      "Exists":  true,
                                      "Root":  "C:\\",
                                      "FullName":  "C:\\Users\\Dustin",
                                      "Extension":  "",
  ...

> I believe types can specify which columns to display by default;

Indeed, the default view (table/list/custom) can be specified, and for the view the default properties can be specified.

Consider how 'ls' produces a sequence of FileSystemInfo objects (when run against a disk file system). The view specifies that the list be grouped by "parent".

On the terminal this gives a nice list a la:

        Directory: C:\Dell\Drivers\T7MFF\PhysX\files\Engine\v2.8.0


    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    -a----       12-12-2014     07:42         362232 NxCooking.dll
    -a----       12-12-2014     07:42        5520120 PhysXCore.dll


        Directory: C:\Dell\Drivers\T7MFF\PhysX\files\Engine\v2.8.1


    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    -a----       12-12-2014     07:42         363256 NxCooking.dll
    -a----       12-12-2014     07:42        5823736 PhysXCore.dll


        Directory: C:\Dell\Drivers\T7MFF\PhysX\files\Engine\v2.8.3


    Mode                LastWriteTime         Length Name
    ----                -------------         ------ ----
    -a----       12-12-2014     07:42         339192 PhysXCooking.dll
    -a----       12-12-2014     07:42         412408 PhysXCooking64.dll
    -a----       12-12-2014     07:42        5952248 PhysXCore64.dll

However, the underlying stream is still just a continous stream of FileSystemInfo's. It is the terminal display format definition that causes it formatting to break and write a directory heading whenever the next item has a different parent from the previous one.

You can certainly print them, but there's no guarantee you'll get all its contents by default, and there's still the problem of how to create those objects from some other output, e.g. in a file.

http://windowsitpro.com/powershell/powershell-objects-and-ou...

It could be said that, in some ways, unstructured streams are far more WYSIWYG, which can conceptually mean easier understanding and use.

Or it can mean a lot harder to understand and use. With types you get all of the metadata that you might not even have if it was plain text. In addition, every application chooses it's own serialization format. In the end, the only programs that compose well are those that work on text, not structured data (i.e. grep, awk, sed, etc...)

> You can certainly print them, but there's no guarantee you'll get all its contents by default, and there's still the problem of how to create those objects from some other output, e.g. in a file.

That's what CliXML is for. Export-CliXml writes objects to a file. Import-CliXml reads them back.

I think they're assuming the default view output is being used for Export-CliXML, when really it's serializing the object passed to it.

You can print them, but you can't serialise them in a way that can be later unserialised.

> You can print them, but you can't serialise them in a way that can be later unserialised.

Nope, that's wrong:

    Export-CliXml
    Import-CliXml

(https://technet.microsoft.com/en-us/library/hh849916.aspx)

For instance:

    PS> ps|Export-Clixml myprocesses.xml
    PS> Import-Clixml .\myprocesses.xml

    Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
    -------  ------    -----      -----     ------     --  -- -----------
        297      15     3856      13304              4024   0 aeagent
         88       7     1420       4808              3804   
    ...

CliXml (Command Line XML, IIRC) is a serialization format for PowerShell objects. It is the format used to transfer objects to/from remote machines when PS remoting is used.

Granted, the re-hydrated objects are not the original objects. Most method will not work after re-hydration as the object has been ripped from it's context. Think ProcessInfo (the objects returned from 'ps' (Get-Process)) - in it's original form it can be used to interact with the process, e.g. terminating it. In the rehydrated form it can be used only to inspect properties.

Speaking from experience, this is rarely a problem. CliXML works remarkably well.

> And the over-verbose syntax doesn't help

This is basically the reason I have not been willing to put any time even trying to learn ps. I mean, trying to translate

unzip file.zip folder

To ps was a nigthmare. Like:

[System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)

I don't _want_ to learn to type stuff like that for simple shell scripts.

Use this.

Expand-Archive file.zip

But that's picking an example where

a. You can still use the native binary just as well. Especially in this case.

b. There was no PowerShell cmdlet for it at the time. There also isn't one for executing Python code, you still have to call python. Or, if you're so inclined, load up IronPython's assembly, and use an overly verbose .NET method call ...

c. There now is a cmdlet for doing so: Expand-ZipFile.

I'm actually curious as to those complaints: Do you never write a function in bash to abstract away something? A function in other languages? Is everything just a series of crystal-clear one-liners?

Shell scripts in Unix-likes tend to glue together a bunch of other programs and (usually) not do much programming in the shell's language. You can write PowerShell just the same, actually. It is a shell, after all.

That was picking an example from my previous try to work with ps. I was not the admin of the server so no installation of extra programs. Or upgrading ps to the newest version.

The shell scripting I have needed has been typically very simple. Like schedule to copy this file from that server and run this command.(what was precisely all that I was trying to achieve the last time, only the file was compressed. Would have done with .bat, but that was even more difficult) As you say, using it as a glue and build the complexity to the programs behind the shell.

If you're used to cmd or bash, the jump in verbosity is very noticeable and definitely gets in the way.

It might be a cultural thing, because I'm no fan of C# and its typical style either; no surprise then, that I find PS syntax exceedingly verbose too. It just feels excessively bureaucratic and awkward to have to write so much. On the other hand, bash, awk, sed, and all the typical Unix commands and its associated ecosystem seem like they "get out of your way" far more effectively.

c. There now is a cmdlet for doing so: Expand-ZipFile.

That example already shows the verbosity increase clearly - why is it "Expand-ZipFile", and not "ZipFile-Expand", "Expand-Zip-Format-Archive", or something else? In contrast, "unzip" is short and easy to remember. The fact that a native binary might exist for a given task is irrelevant to the observation that the shell's language is itself more verbose.

I'm actually curious as to those complaints: Do you never write a function in bash to abstract away something? A function in other languages? Is everything just a series of crystal-clear one-liners?

Abstraction helps reduce code duplication but is not useful when each line of the script is quite different, and in that case PS remains more verbose. Ultimately, the overhead is still higher.

> Why is it "Expand-ZipFile", and not "ZipFile-Expand", "Expand-Zip-Format-Archive", or something else? In contrast, "unzip" is short and easy to remember.

Iff you know it already. PowerShell is built around a few conventions. One is that all commands share the Verb-Noun pattern. The verb always comes first. Then there are a bunch of common verbs that all share the same basic meaning, regardless of context. Expand is such a verb that is the opposite of the Compress verb. You may not like then choice of verb but there are always going to be names you didn't choose, so that's probably a petty point. In the end, PowerShell makes it easy to discover commands you may vaguely know exist. I'd argue that also helps remembering them once you know them. Just from knowing unzip you wouldn't be able to guess the command to extract a GZIP file, for example.

___________

P.S.: I have to retract part of my original post here. Expand-ZipFile does not exist natively in PowerShell. I stumbled across http://ss64.com/ps/zip.html and while that site does have documentation for all built-in cmdlets, I didn't read too closely and this was actually documentation for a wrapper around the .NET framework's own functionality. This does not change the discussion about the name and discoverability, though, except that New-ZipFile should probably use a different verb. Actually, the PowerShell cmdlets for handling ZIP files are Compress-Archive and Expand-Archive, exhibiting the properly mirrored verbs.

> Normal shell usage means doing a lot of one-shot pipelines incrementally.

Sure because no-one ever takes those "one-shot" pipelines full of `cut`s and `grep`s and distributes them in real products. /s

As far as I can tell, Powershell is designed for scripts that are meant to be distributed, so they must be robust. Normal 'text scraping' Bash is shit at that.

If we are not talking about interactive use then to distribute "robust" scripts, you could use nicer and richer Perl, Python eco-systems.

For interactive use, the conciseness, the human-readability of input/output and the fault-tolerance to unrelated changes in the input/output formats provided by Unix shells are also preferable.

The motivation for PowerShell was to have something good at both. Why switch contexts when moving from interactive use to coded automation if you don't have to?

Because the requirements change depending on which one you're doing: quick shell automation isn't the same as the kind of automation you'd use perl/python for.

And yet most of the time I see bash being used for the same kind of automation that you think is good for perl/python.

And not because bash is good, but because it is what people know (since they use it interactively) and because it is installed everywhere.

So if someone had a tool that was installed everywhere, and used interactively, that could also be used to create more robust automation tasks, that seems like a win to me.

It depends on how complex the automation is. I'd no doubt use a pipeline in places where you'd use perl/python.

As for interactive use and robust automation, Bash isn't as bad as you'd think. The reason I'd go to python is because of script complexity, not lack of robustness.

> I compare this to human languages. It would be tempting to create a language with minimum ambiguity and clear cut concepts, but it wouldn't be practical.

Lojban[1] is usable in practice (though that may be because it still allows you to be ambiguous when desired).

[1] http://lojban.org

Passing CSV around can be a useful middle ground, if you're using utilities that all do proper escaping and allow newlines in strings etc., such as csvkit.

What is proper escaping for CSV?

RFC4180.

The problem is that not all tools agree on this one, but: lines are separated by newlines, items are separated by commas. An item can be quoted (and this must be done if it contains a newline, comma, or double quote) by surrounding them with double quotes. Inside such quoted items, double quotes can be escaped by doubling them.

DSV is the default format for many unix utils, and AWK and cut parse it easily.

Yes, of course; that is one end of the "middle ground" I described. The problem is that in Unix, there is no common convention for how to escape the delimiter, so you cannot pass arbitrary strings. CSV can do this, with the proper tooling. Being able to pass around arbitrary strings is quite valuable, and although it is not quite the same as passing objects around, it has the advantage of still being human-readable text. That is why I called it a "useful middle ground" between Powershell and the delimiter-separated fields of Unix tools. However, you do need special tools such as csvkit, as Unix tools like awk, sort and cut cannot properly parse it.

Oh, you can pass arbitrary strings, you just can't pass arbitrary strings as part of compound datastructures. The closest you can get is using null or some other weird character to separate lines ($RS='\0' in awk), and something similar for field separation (like -0 in find, which is $FS='\0' in awk). But \0 is only one special character, so you need another one.

> text "scraping" is just better for most cases.

Unix pipes handle bytes, not just text. For instance, copy a hard disk partition to a remote hard disk via ssh:

  (dd if=/dev/hda1) | (ssh root@host dd of=/dev/sda1)

The KISS principle ("Keep it simple, Stupid") is the best way in many cases. In Unix you can quickly enter a pipe without special coding which does also non-trivial stuff. For instance,

  find -name "*.xls" -exec echo -n '"{}" ' \; | xargs xls2csv | grep "Miller" | sort

gets a sorted list of all entries in all Excel files which contain the name "Miller", no matter how deeply the files are located in the directories. Can you do this in Powershell quickly? I don't know, I am actually curious.

Objects in pipes are convenient and powerful. However, for most applications of pipes they are probably overkill. If things get tough you can simply use files rather than objects.

I would not be surprised if many Windows users will prefer bash pipes from the Ubuntu subsystem in Windows 10 rather than Powershell because it is much more handy for most pipe applications.

> Can you do this in Powershell quickly? I don't know, I am actually curious.

You can basically do the same thing with Powershell. I don't know of a 'built-in' module to handle XLS->CSV conversion, so you need to bring one in:

  Install-Module ImportExcel

Then:

  ls *.xlsx -r | %{ Import-excel $_ } | ? { $_.Name -eq "Miller" } | sort

That's my naive, Powershell noob approach anyway.

There's another option however, where you can leverage Excel itself (granted, this is likely to be a Windows only approach):

  $excel = New-Object -com excel.application

And you can now open XLS/XLSX files and operate on them as an actual excel document (including iterating through workbooks/ worksheets, etc.). It's all just objects.

Thanks for pointing this out. However, your example is not equivalent since "ls" searches only in the current directory. Nevertheless it is good to know that it's basically possible since it helps a lot to manage mixed Windows/Linux networks.

Not true. The -r flag I specified is for "recurse", will search current and all sub-directories.

A better way to do that find command is:

    find -name '*.xls' -print0 | xargs -0 xls2csv

This is all a big part of the reason why I personally advocate using a standardized text-based object serialization format (like YAML) for these sorts of things. In particular:

* Still text based and human-readable, which helps for debugging/troubleshooting * Still text-based and machine-readable, so it's inherently cross-platform (assuming that all said platforms agree on their text encoding) * Still less troublesome than piping arbitrary text through `grep` or `sed` or `awk` or what have you * Still provides the "everything is an object" benefit that's lost with arbitrary text streams * Still orientable around streams of data, at least for YAML (by using the document delimiter)

As a note, there have been strongly-typed shell languages for Unix for some time. Scheme Shell[0], Turtle[1] and TCSH[2] leap to mind; but there's nothing preventing a developer from using Perl, Python, Lisp or any other language that allows itself to accept stdin as an interpreted script.

0: https://scsh.net/

1: http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...

2: http://www.tcsh.org/Welcome

Edit: oh my, I posted the wrong link for tcsh. No one noticed, but my apologies nonetheless.

I want to throw my support behind scsh!

It's useful for those things that cross the boundaries between a program, and a shell script. I've found once you need functions for a shell program, it's time to step up to scsh.

I also wish scsh ports were more well maintained so people could use their favorite schemes with scsh syntax, but it's quite a large codebase to port, so remember that it's scheme48 scheme.

I thought I was the last person using scsh!

Now if I could just imbue my coworkers with your good taste....

Nice! :)

I write a lot of my stuff in guile when it's more formalized, but scsh's run syntax is just the bees knees. Can't get rid of it.

bits of SCSH have been ported to every scheme out there now. And some have been improved. Irregex, in particular, is a dramatically improved version of the SCSH regex engine.

Oh yes, ashinn's irregex is so good, I just include it in my home directory.

This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.

PowerShell will never accumulate a library as comprehensive as what any of those three have, each has had decades to accumulate packages of all kinds, and more are still being added.

It's odd, but not surprising given their history of "Not Invented Here", that Microsoft never even considered using Python or Ruby when they both have .Net implementations. Their enthusiasm for JavaScript wasn't sufficient for them to embrace that either.

Powershell can use any .NET library... "Nobody" meaning you. Which is perfectly fine. But there are quite a few users of Powershell out there.

Now that it exists there are users, but when people were hoping for a modern or more portable replacement for DOS batch files I'm not sure this is what they had in mind.

Think of it as a replacement for vbscript instead, with a commandline.

Powershell In Action covers a bit of the history and decision making around powershell; interesting stuff.

Ultimately, given that powershell was initially targeted at systems admins/engineers and that Windows is highly object-oriented(in contrast to Linux's text streams), I believe Powershell was a apt development. Having done plenty of systems engineering across linux and Windows it seems to fit the needs of Windows pretty well; a fantastically unique shell that wouldn't have existed if not for Windows.

Last IronPython release: 2014

Last IronRuby release: 2011

I can't speak to Ruby, but Microsoft has an active Python group.[1] Granted, it seems to focus on Visual Studio (which I'm told has excellent Python tooling) and Azure.

[1]: https://blogs.msdn.microsoft.com/pythonengineering/

Neat, but that doesn't seem to be integrated into .NET like IronPython was (well, is, just not maintained it seems). Which is what the post I was responding to was talking about.

Ah, touché. I'm not a Windows person, so I don't know the boundaries well.

Looks like things are changing in IronPython land:

https://www.infoq.com/news/2016/08/IronPython-Leadership

I used to love Python when I came to it years about after slogging along through QBasic, VB6, C++ and C, but in this day and age, it doesn't really offer much to me over doing things in modern C#, especially on the CLR. I appreciate some of the changes that were made to the runtime to support IronPython, like dynamic support and the DLR, but if I'm going to be doing something not-C#, then I'd rather go towards F#.

Both of these two projects attempted to solve a problem that very few people have - using .NET libraries from Python and Ruby. In exchange, you had to give up all of your existing native libraries. That is a trade-off that very few people wanted. Source: I started the IronRuby project.

I learned this lesson the hard way a few times in my career - always have respect for people's existing code investments.

I feel very similarly, but then I'm of the mind that if I can't apt-get it, I'm not interested, and I'll bet that "open source" doesn't mean to MS what it means to the open source world.

As for the NIH syndrome, one of my favorite Paul Graham essays is still "What Languages Fix" (http://www.paulgraham.com/fix.html) particularly for C#: the problem that C# was invented to solve is that Microsoft doesn't control Java.

C# incidentally solved many other shortcomings with Java. The languages are getting closer to parity, but for many years C# was pretty far ahead (if you only consider language features and not tooling/ecosystem). Between reified generics, better native interop, and many functional features, C# felt and feels more concise than idiomatic Java. And I've written plenty of both.

Disclaimer: MSFT employee expressing personal views

In what way was C#'s tooling not better?

Java seemed to have a lot more options for deployments and monitoring for a long time. More advanced 3rd party libraries in general. It had simply been around longer and thus had those tools available. Now that C# has been standard for a while almost all of those tools have a C# equivalent.

As a .NET dev I'd actually argue that the things available to C# now are actually better (especially Visual Studio), but that's certainly biased.

VS was probably always a better coding and debugging environment. I've used VS as far back as VS 2010 and it's always been great for C#, but I've never had more than a mediocre experience writing Java in an IDE. Did anything support VS's level of convenient debugging ability for Java?

I have not used VS as much as I have used Java IDEs, but I'd say they are on par. VS shines in how well the debugger is integrated with the rest of the environment, but both NetBeans and IDEA offer very close to the same level of convenience. Admittedly, I have never liked the Eclipse debugger.

Meh, to make Visual Studio do IDE-like things, people install things like ReSharper, built by the same people that are building IntelliJ IDEA. And I worked with both and IntelliJ is much smarter, and has support for way more technologies. And yes, it's built in Java and can be sluggish, but then again I'm using it on both Linux and OS X and runs just fine on Windows too, so it doesn't tie me to an OS I don't want.

Also, I keep mentioning this, but .NET lacks a build tool like Maven or something similar (SBT, Gradle, Leiningen). .NET is still at the level of Ant / Make and .NET devs don't realize what a huge difference that is.

What am I missing building projects with VS or xbuild?

For one, sane dependency management that doesn't require vendoring.

You can reference projects or other files that are in submodules. What about the .NET build process makes vendoring the only option?

Even so, it's always been more convenient in my experience when something just ships with fixed versions of dependencies already inside it. Those versions are known to work with the codebase and there's no chance of new bugs or other incompatible behavior being introduced from updates to the dependencies.

> What about the .NET build process makes vendoring the only option?

AFAIK, even NuGet doesn't allow you to just check in the dependency spec, rather than the contents of all of your dependencies.

> Even so, it's always been more convenient in my experience when something just ships with fixed versions of dependencies already inside it.

I agree. But that doesn't mean that they belong in Git, nor that they should even be copied to every single project.

> AFAIK, even NuGet doesn't allow you to just check in the dependency spec, rather than the contents of all of your dependencies.

I have a project where I only checked in the solution's repositories.config (packages/repositories.config) and the project's packages.config (ProjectName/packages.config), and it seems to work fine.

>Also, I keep mentioning this, but .NET lacks a build tool like Maven or something similar (SBT, Gradle, Leiningen).

You mean in terms of integration with Visual Studio? Because both Maven and Gradle can compile .NET code (msbuild or xbuild). Gradle doesn't natively support Nuget's format, but there are plugins for it.

> .NET is still at the level of Ant / Make and .NET devs don't realize what a huge difference that is.

Gradle just uses javac under the hood.

http://fsharp.github.io/FAKE/

No, FAKE is not a replacement for Maven. As I said, .NET is left behind in the Ant/Make era.

And since you mentioned an F# project, the absence of a Maven-like build, dependency management and deployment tool is why F#'s Javascript compilers are behind, because you have no shared infrastructure, no cross-compiled libraries, etc, because it's just too painful to do it.

Why isnt it a replacement.

NuGet/Packet for dependencies.

Octopus/Web Deploy etc for deployment.

All of which can be used from this tool.

>people install things like ReSharper

This used to be true but VS2015 is actually completely fine without ReSharper. Roslyn is amazing, I've even built my own custom code analyzers (ie. compiler extensions) that integrate in to the entire ecosystem seamlessly (nuget package registers them to VS project, IDE shows you code analysis on the fly, C# compiler uses them on every build, including CI/build server - and you can enforce stuff with it - eg. raise build errors).

>but .NET lacks a build tool like Maven or something similar

There are .NET build tools, FAKE, CAKE, etc. etc. but people don't use them much, tooling integration is notably missing. It would be nice to have something like Gradle in .NET but VS support for msbuild is good enough for most.

FAKE and CAKE are not replacements for Maven, but for Ant/Make. Maven and Maven-like tools on the other hand are so much more.

Here's a personal anecdote. I work on Monix.io, which is a cross-compiled Scala library for the JVM and Javascript/Scala.js.

I could easily do that because of SBT, which is Scala's flavor of Maven. Basically Scala.js comes with SBT and Maven plugins. And you can easily configure your project, indicating which sources should be shared between the JVM and Scala.js and which sources are specific. And afterwards SBT takes care of compilation for multiple targets, building JAR files for both the JVM and Scala.js. And then you can also sign those packages and deploy them on Sonatype / Maven Central. The tests are also cross-compiled too and running whenever I do "sbt test", no tricks required or effort needed to do it. And this project has a lot of cross-compiled tests.

And if I were to work on a mixed JVM/Scala.js project, I could also include a whole assets pipeline in it, with the help of sbt-web, another SBT plugin. SBT and Maven have a lot of plugins available.

The Scala.js ecosystem relies on Maven / SBT for build and dependency management. There's no "npm install" for us, no Gulp vs Grunt cluster fucks, no artificial separation between "real Scala" and Scala.js, only what's required. And trust me, it's a much, much saner environment.

Compare with Fable. Compare with FunScript. Don't get me wrong, I think these projects are also brilliant. But it's because of the ecosystem that they'll never be popular, because the .NET ecosystem inhibits reusable F# cross-compiled libraries.

Umm, .NET has nuget packages.

The JetBrains folks make incredible tooling.

I got into their stuff via pycharm and phpstorm and then ended up using intellij with the php and python plugins (plus node and a bunch of others).

I haven't enjoyed using a development tool so much since back when Borland was good, intellij is pretty much 99% of my development tool usage at this point and that 1% is mostly nano for quick 'I need to edit this one line' edits.

The thing they seem to get so right is that intellij with say php plugin feels like phpstorm and with python plugin like pycharm at the same time, it's not the features so much as how they are all seamless integrated and the tool as a whole feels designed.

One of the few bills I genuinely don't mind paying each month, everything thing else I've tried just doesn't compare*

* Your Milage May Vary.

> ...things available to C# now are actually better (especially Visual Studio), but that's certainly biased.

You call Visual Studio good? Have you actually used it?

VS2015 is painfully slow, and yes, I've installed all of the updates and I have no plugins or such. Heck, even Eclipse feels zippy compared to it. Something as simple as finding next text search match takes about a second. Changing to a different source code tab is also sluggish.

Context sensitive search (right click identifier -> Find All References) is useless for anything that is even a bit generic. Instead of actually analyzing context, it seems to simply grep all reachable files for a text string. Including SDK headers...

I use VS2015 only for C++, maybe that can have something to do with it. Maybe it works better for C#?

I also dislike how I have to login to use an IDE. Once it cost me 30 minutes of working time, I couldn't login to a god damn IDE that had an expired license. That's a great way to make developers hate your product and ecosystem.

There's a box you can tick that says no to all of that stuff, though I don't know if you can access it after the first run. (I never have to log in when I run Visual Studio.)

No, at least in the version I have, license expires quickly.

If I don't enter the credentials, it won't be able to renew the license, so it's not possible to use it at all. Regardless of any checkbox ticks.

It is a lot better for C#. All the features work perfectly for .NET languages, but because of the nature of C++ it's a lot harder to get these kinds of things working well with it. Some answers about why are here: https://www.quora.com/Why-is-C++-intellisense-so-bad-in-Visu...

Eclipse can run better on a potato but it's also very simplistic. You should also be able to turn off most of the features you don't want in VS.

Well, heck even Emacs with some LLVM wrappers can do better job at C++ Intellisense than Visual Studio. Eclipse's CDT runs circles around it as well (so much for being very simplistic). And don't get me started on trying to use Intellisense on large codebases - such as Unreal Engine's source code. It is just completely unusable (30+ seconds before the completion pops up, often blocking your IDE (a total usability no-no) with a modal "Intellisense operation in progress ...").

Also anything using complex templates (= pretty much any C++ code these days) throws it for a loop - making it useless where it would be needed the most.

And don't even get me started at the ridiculous state of refactoring support for C++ - even the stupid identifier rename doesn't work reliably, and that is the only C++ refactoring feature VS has (and we had to wait until 2015 for it!).

Sorry, but that something is hard doesn't mean it can't be done - especially when there are free tools which can do it and orders of magnitude better. Even Visual Studio can do it - if you buy the Visual Assist plugin.

Even the C# tooling isn't that great - yes, it does work better (I often do combined C++/C# projects) than the C++ one, but that isn't a particularly high bar to clear. Ever tried the Java tooling in Eclipse? Heck, that editor and the various assist features almost write your code for you ...

Where Visual Studio is great is the additional tooling - integrated debugger and the various profiling functions. Eclipse can't really compete with that, even though most of the tools (gdb, gprof, etc.) exist and have a lot of features that even VS's debugger doesn't have (e.g. reverse debugging, record/replay, etc.). Sadly, the Eclipse front-end doesn't support all of them.

However, the Visual Studio's editor and the tooling associated with actually writing and managing code (i.e. the part a programmer spends most time in) is stuck straight in the late 90's - just look at how code navigation (jumping to functions, navigating between files, projects, etc.) works in the ancient Visual Studio 6 and 2015 - it is almost identical.

There is also zero integration for alternative build systems like CMake. That is important for writing cross-platform code or even just basic sanity on large projects - the Microsoft's build configuration/project/solution system with millions of settings buried in cluttered dialog boxes is just horrid and error prone, especially with their myriads of mutually incompatible options - set one option incorrectly on one library and your project starts to mysteriously crash. Good luck finding that - and then spending hours recompiling your project :( However, nothing else is really supported properly. (Yeah, I know that CMake generates VS solutions/projects, but there is no integration within VS for it - so you have to be careful how you add files, how you change build options, etc.)

The saddest thing is that many of these issues are pure usability problems where changing/correcting a stupid design decision made a long time ago would reduce the programmer's frustration by maybe 80% already - like the modal blocking Intellisense operation windows, the ergonomic disaster that are the build configuration dialogs, the decision to show compilation error with templates on the deepest template in the expansion (i.e. somewhere in the STL header) instead of where it has occurred in your code, forcing you to mentally parse several pages of error messages to find the line number or just resorting the Intellisense matches to show you the project-relevant ones first instead of the unrelated stuff from the frameworks/header files so that you don't have to dig through it. I.e. common sense stuff.

C++ isn't going anywhere on Windows, no matter what MS and C# advocates say and it is really sad that the official tooling is in such poor shape.

I could continue ranting about many other things that are regularly driving me up the wall at work (like the incredibly slow and memory hungry compiler, horrid and unreadable error messages, especially for templates, etc.), but that would be pointless. IDEs are a matter of strong preferences and opinions, but I do believe that with Visual Studio, especially their C++ tooling, Microsoft has missed the boat many moons ago. And isn't even trying to catch up as it seems - they rather focus on adding stuff like Node.js support (are any Node programmers using VS or even Windows??).

.. if this is you not getting started, I'm kind of frightened for when it does happen ;)

Ah, sorry for the rant.

However it does piss me off when I see people uncritically praising VS as the best thing after sliced bread and flat out dismiss the alternatives without even looking at them - when VS is really terrible IDE.

Unfortunately, this is something that is difficult to comprehend for someone who has never seen how it actually could work properly - either because they have never been exposed to the alternatives or because they have the VS-centric "workflow" (more like kludges working around VS silliness) ingrained so deeply that anything else will perturb them.

"Something as simple as finding next text search match takes about a second."

I'm not sure how our configurations differ but on a fairly modern machine (say, surface pro 4 i5) I've not had visual studio 2015 slow down, and editing and searching is az zippy as with Vim for me.

I've had the experience that some third party tools have bogged down the experience, though.

I mostly run it on a Xeon workstation with 32 GB RAM, SSD, Nvidia K5000, etc. I think you can call it a modern machine. Other software runs fast.

I don't have any third party software installed on top of VS2015.

Only unusual aspect is that I also have several other Visual Studio versions installed, like VS2008, VS2010, VS2012 and VS2013.

I also have many Windows SDK and DDK (driver devkit) versions.

But that shouldn't affect, right?

I run it at work on Xeon with SSD and 16 GB. Project sizes might be different. I don't have experience how the performance would scale with a project with tens of millions of lines of code, for example. (seems to be fine for me below that).

It does exhibit strange behaviour (including becoming suddenly very tardy) that only goes away by deleting the local cache files (intellisense db and whatnot) so I'm not claiming it's technically perfect.

It is meaningless to make such comparisons unless you also compare codebase sizes. VS is perfectly zippy if you have a small project - but try to hit CTRL+SPACE to complete an Unreal Engine or OpenCV identifier! You will be sitting there for 30 seconds or longer while VS is busy parsing/searching files ...

The latest update to VS2015 fixes a lot of the performance issues for C#.

I'd have to agree, though I spend most of my time in the node space, today via VS Code (though I don't use the debugger at all). VS Proper with C# has felt better to work with than any Java tool/project I've ever used, though haven't done much with that ever. I jumped on the C# bandwagon in late 2001, early 2002 when it was still in development. Started moving over towards node around 5-6 years ago, and am currently looking for a reason to work with rust or go.

Most of my issues with C# have always, like Java, been about "Enterprise" frameworks, patters and practices implemented where they aren't needed, and complexity for complexity's sake. VS + C# is pretty damned nice, if you can work on a green project and not bring in everything under the sun.

except that C# as a language is by leaps and bounds better than Java (both in syntax and useful features departments), so there have been other problems to solve, too.

Its ecosystem isn’t. C# will never get accepted as an open source product. It only has open source code, it’s not actually ‘open-source.’ Compare it with Typescript, which—despite originating from Microsoft—is a truly open project, and getting love left and right.

C# might be better than Java/JVM, but it’s not better enough. The culture/ecosystem barrier is so high that C# would have to be miles ahead, technically superior in every way, to overcome it. I do hate the “worse is better” adage, but there’s no mistaking it, it applies here.

It’s just too little, too late.

But Typescript is awesome, keep it up.

Ironically, Google’s competitor (Closure Compiler) has actually been unsuccessful for similar reasons. To this date its main repo is just a clone of the internal master. For whatever reason they’ve never attempted to rebase on open-source release.

i have a hard time with this argument. on one hand what you say is true: C# is a strictly smaller community than java. OTOH that's true of pretty much any language, and yet python, ruby, elixir, swift, golang, etc. communities are healthy and vibrant.

if what you really mean is 'java people won't switch to C# anyway', then i agree, but C# isn't a really a language for them. it's a language for people who don't like and/or aren't forced to use java by their employers.

People who aren’t forced to use Java will choose Scala or other JVM langs, like Kotlin, Ceylon (a favorite of mine) or even Clojure.

C# the language is not exactly that exciting. I get it, it looks attractive next to Java, but it’s still a verbose, corporate-first, sort of thing. If anything, F# is much more competitive. Too bad it’s on CLR.

[disclaimer, also MS employee].

This has already devolved into opinion territory but I don't think you're giving C# enough credit.

I picked up F# relatively early in it's lifetime (2006ish?), back then there were many language features in F# that you just couldn't do in C#. The gap closed a lot when C# got LINQ, generics, and lambda/first-class functions (these are relatively old language features by now).

If I want to write in an quasi-functional-programming language style I can do it without having the language get in my way. I certainly wouldn't call it a "verbose, corporate-first" language, although the fact that it can be used for that is a bonus.

Don’t forget Rx. Not exactly language feature, but certainly a great contribution to come out of C#/.NET. And who knows if it would have happened without LINQ.

I like the language. Just not enough to use it over JVM. And I think most people feel the same way.

As if C# were any less driven by PHB-dictated internal enterprise mandates than Java is.

That is not quite true.

Many of us doing enterprise consulting do jump between Java and .NET projects all the time.

Sometimes even doing mixed projects, like the UI in .NET and the backend in Java.

i'll grant you that after a very brief consideration

> UI in .NET and the backend in Java.

makes a lot of sense.

The problem with native desktop Java applications that although Swing is quite powerful, it requires developers to go through books like "Filthy Rich Clients"[0] to really take advantage of it. Which very few do.

To this day I still meet developers that aren't aware how to change the default L&F to native widgets, for example.

Whereas Windows Forms and WPF already provide a much easier out of the box experience, and have tooling like Blend available.

I am curious what the JavaFX sessions at JavaONE will cover.

[0] http://filthyrichclients.org/

What do you mean by it not being open source? The core CLR is MIT licensed and the compiler is Apache licensed.

This sounds like fud to me.

I have no issue with the license.

So it's maintained in the open on GitHub, it's technically open source in terms of licensing. Yet you claim it's not really open source. Care to clarify?

It’s technically open-source, that’s the point. There’s more to open-source than license. Sorry but there’s no way for me to clarify without just repeating my original comment.

Your original comments mostly contained your personal opinions, not actual facts. The fact is that it is open source and you are FUDing.

> Your original comments mostly contained your personal opinions

I never tried to pass it as anything else.

There’s more to open-source than license.

Such as? You seem to have a mental model of things that make a project objectively open source, that don't include the license. I'd be curious what those things are.

I really don’t, it’s more of a feeling. With an open-source release like .NET it seems more like better documentation. In fact that was the case for early commercial Unixes—you needed the source code to actually use the system, but it wasn’t open-source.

Open-source as-documentation (for lack of better term) is still useful. It makes bug fixing a whole lot easier, for one thing. But it’s not quite the same as open-source ecosystem. For that you need to have a diverse set of actors, sharing the same goal. That’s what I think successful open-source project makes. You need to accept the fact that the project is not just yours. Something like that.

Of course Microsoft could do all those things. Who knows, it they’re determined enough they might turn it around. The problem here is like I said Java is just good enough. No one really cares, except people that could use some better documentation, that have been already invested in the ecosystem. That’s why open-sourcing is still valuable, but also why they’ll never gain any adoption of the kind they’d need.

Sorry if that sounds like rambling, it’s sort of late.

do you more mean like, the decisions, and planned changes, etc, aren't open? (along with being tied to the whims of the CEO and the company's money?)

They might be open, but there’s democracy and then there’s democracy. See for example recent MSBuild incident (but don’t try to argue about it it’s just an example).

As I said, it’s a feeling. The feeling is it’s Microsoft’s project, everyone else is along for a ride. And that’s fine, but it’s something different. Let’s just not pretend technical merits drive adoption, that’s rarely true.

> They might be open, but there’s democracy and then there’s democracy. See for example recent MSBuild incident (but don’t try to argue about it it’s just an example).

> As I said, it’s a feeling. The feeling is it’s Microsoft’s project, everyone else is along for a ride. And that’s fine, but it’s something different. Let’s just not pretend technical merits drive adoption, that’s rarely true.

Uhm. So many free software projects work like that. A company creates something, releases it as free software. Yes, people contribute (and that's awesome by the way) but in general all of the engineering talent works at the company because they wrote it in the first place (and they hire contributors). At SUSE this is how YaST, zypper, spacewalk, et al development works (with significant contributions from the openSUSE community, but we have teams of people working on those projects so our contributions are more of a concentrated effort). There's nothing new or different about this model of free software development (Sun did the same thing with OpenSolaris and Joyent does the same thing with SmartOS). Yes, GNU and Linux follow the hobbyist model but that's not how all projects work.

I was too harsh with the ‘open-source as-documentation’ term.

My point is this is just not enough to compete with JVM, which is already a vibrant open-source ecosystem.

by that definition, it'd be hard to call Python open source.

It is open source in every sense. The development is open, they accept patches/contributions etc.

.net is far more of a true open source framework than android is.

> It’s technically open-source, that’s the point. There’s more to open-source than license. Sorry but there’s no way for me to clarify without just repeating my original comment.

It's free software, and in that sense the license is the only thing that really matters. However, if you're discussing open collaboration styles then that's a whole different discussion. Either your project is free software or it isn't. Whether it has a diverse and open development community is a separate problem, and doesn't fall under "is this project [free software]".

That's not what Worse Is Better means. Did you read the Gabriel paper?

/nitpick

I got it from Unix’s haters handbook.

Well, you clearly didn't get the full definition. Go read Lisp: The Good News, The Bad News, And How To Win Big

Sure, it beats Java, but the VM is worse for running other languages, and that's where .NET loses. F# tries to be nice, but reified generics are more of a limitation than help in that world.

In JVM land, now Java just hands you some specific well tested libraries: You write business code with Scala or Clojure, which I'd pick over C# by about as much as I'd pick C# over Java 7.

That said, I have far more faith in Microsoft improving their tooling than I have about Oracle doing the same: It's just that Oracle has to carry a far smaller weight, because the good JVM languages aren't even theirs.

Uhm, the CLR (hence the name) was specifically designed to host different languages that can interoperate with each other and unlike the JVM was not built around the capabilities and limitations of a single language.

> Sure, it beats Java, but the VM is worse for running other languages,

based on what criteria?

>and I'll bet that "open source" doesn't mean to MS what it means to the open source world

The github repo use the MIT licence, so I don't see what you're talking about.

If you want it in your distribution repos, ask your distribution maintainers (and I bet Microsoft is going to do so anyway for the big ones)

Yes, please ask them....happy to help adding PowerShell to any distribution .NET Core supports.

> if I can't apt-get it, I'm not interested,

It was released just now, under MIT license. Give it a little time and it'll show up.

Or even more! It will need to be backported to Trusty Tahr if one wants to apt-get it on Windows (subsystem for Linux) ;-)

WSL will end up on Xenial at least.

>I'll bet that "open source" doesn't mean to MS what it means to the open source world.

Before Satya Nadella took over it didn't. Now it mostly seems to.

》I'll bet that "open source" doesn't mean to MS what it means to the open source world.

That doesn't make sense. ASP.Net, .Net Core, F# are all good examples of open source projects. This announcement promises the same for Powershell: Development with the community.

What are you missing? What's the criticism?

If you want to see a broken open source project, check out Android instead..

Ah, that's beautiful. Using a true-Scotsman argument to claim it's not "Real" open source, while denigrating Microsoft for NIH. Classic.

> As for the NIH syndrome, one of my favorite Paul Graham essays is still "What Languages Fix" (http://www.paulgraham.com/fix.html) particularly for C#: the problem that C# was invented to solve is that Microsoft doesn't control Java.

That's not what Graham said. He said, "C#: Java is controlled by Sun", which is quite a bit different from Microsoft not controlling Java.

Microsoft is fine with using and promoting languages they do not control. C++, for example.

The problem with Java from Microsoft's point of view was that Sun did not want people to use Java as a mere programming language. They saw Java as a platform, and wanted people to develop for that platform instead of developing for Windows, or Mac, or Linux, or whatever. Sun wanted all access from Java programs to the underlying facilities of the OS upon which a particular instance of the Java platform ran to go through the Java platform.

Microsoft wanted something with the good parts of the Java language without the baggage and restrictions of the Java platform, and so they made C#.

> ... if I can't apt-get it, I'm not interested ...

I feel bad for you, then. The Ubuntu repos (and Debian) have almost nothing that is even remotely new and will usually be behind on most things. Trying to keep configs for an ArchLinux box and Ubuntu box synchronized is a bitch if new versions have good features, because you can't use them in your general config unless you give up on apt and install from source.

For programming, I agree. However, Python doesn't work as a system shell; for that, you want something optimized for running commands, without having to wrap them in a function call with a list of arguments as strings.

did you try xonsh?

I did, and i found it a pretty big chore in practice. Its two modes (shell mode and python mode), never quite knowing which one you're in, I could never quite get used to that. PowerShell doesn't have this problem, because it was designed for the command line first and for scripts second (which is, incidentally, also its major downside if you ask me).

you can force captured subprocess mode if necessary

> Its two modes (shell mode and python mode), never quite knowing which one you're in, I could never quite get used to that.

Drive-by armchair comment: it seems like the obvious answer would be to change the prompt depending on what mode you're in?

I find this criticism of xonsh's "modes" baffling. Xonsh is brilliant precisely because it isn't modal the way, e.g., vi is.

Xonsh lets you freely mix python code and command invocations in a highly transparent way.

> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.

I'm not sure this is really right though. You could make the same argument against Ruby or Python in favor of Perl or Pike, could you not?

PowerShell syntax is quite minimal, has a novel methodology for strongly typed interactive use, and has the ability to directly invoke managed code. You can write shell components in any .NET language and invoke them (with care at writing time, this can be done very cleanly, but even without said care it's possible with a bit of a mess).

> PowerShell will never accumulate a library as comprehensive as what any of those three have, each has had decades to accumulate packages of all kinds, and more are still being added.

It has the entire .NET library at its disposal. I certainly never feel a lack of support using it.

[Disclaimer: I work for Microsoft.]

As far as I recall, the original impetus behind what became PowerShell (as handed down to us in a conference room by Ballmer himself, sometime around 2001) was to fill the gap between ops people who did enterprise administration manually with tools like MMC and engineers who automated enterprise administration with tools like C++/DCOM. The latter were necessary in a lot of cases, but they were expensive, and we needed to give the industry a way to do powerful automation without hiring a bunch of PhDs. So, yes, someone did ask for it - the IT industry.

Not "The IT Industry", surely only a subset of Windows systems managers.

That's actually a pretty substantial chunk of "the IT Industry" though. And if you're the people who make Windows, it's not surprising that it's a big enough chunk to do something about.

I personally find that the term "IT" has become a bit of a code word for "Enterprise Wintel", and that anyone working outside of the gravitational pull of Microsoft prefers to describe themselves as working in "tech"

I think that IT has become a code word for being a tech worker in a company, rather than a worker in a tech company.

> So, yes, someone did ask for it - the IT industry.

... who couldn't use the well-known mature shell tools since they were locked into a closed platform, and Windows' cmd.exe was comparatively very weak.

Well, lots of windows admins used things like cygwin and the like to get a unix-y working environment, but PowerShell offers a much more robust set of mechanisms to manipulate the underlying Windows operating system.

It wasn't cmd that was the problem (although, it is weak); it was how Windows exposed APIs to allow automated solutions.

You had two options: VBScript (which is a glorified COM+ agent) or a "real" programming language like VB6/C# which mixed COM+ with Win32 API calls (which are too hard to use for most non-programmers).

Powershell within itself does improve CMD. But that isn't primarily how PS pushed Windows automation forward, Microsoft said that every major server feature and server function should work in Powershell, so engineers at Microsoft had to look at it (be it their lacking COM+ interface, provide a new PS interface, or something else).

The net result isn't that CMD got replaced by something better. It is that we have a lot more APIs and ways of interacting with Windows features and services than we ever used to.

The ironic thing is that today you could write MS Dos-style console applications that work in CMD which provide all of this new automation functionality because of the Powershell push. Back before Powershell you'd have to hack it using unsupported APIs, registry entries and similar.

If all this API effort had gone into a real cross-platform language (Python, Ruby, or even Perl which was king in 2001), Windows would have flourished. They went for their traditional lock-in approach instead, so now they have to catch up.

Windows did flourish...

And are you suggesting that Perl would have been a better solution for a Windows interactive shell and automation tool?

Are you going for the more readable argument? Or the more strongly typed argument?

> Windows did flourish...

Sure. Just see all those buildings full of Windows servers running Facebook, Google, Twitter, Amazon...

It's moving the goal posts a bit to suddenly restrict this discussion to only servers, no?

There are plenty of Windows servers running plenty of businesses. They may not outnumber non-Windows servers, but I don't think "majority" has to be a prerequisite for "flourish".

There are also buildings full of Windows clients...

This discussion is about PowerShell, which is primarily intended to automate server management and has been ported to Linux, presumably, for the same purpose.

> This discussion is about PowerShell, which is primarily intended to automate server management

Many (me included) find PowerShell extremely useful for automating any sort of Windows management, client or server. So I disagree that the discussion should be implicitly assumed to be limited to Windows servers.

> Windows did flourish

Late reply, sorry. I don't think we're on the same page: Windows sold relatively ok, basically doing tying the battle for the Enterprise sector, but in the meantime it lost the cloud war and was literally wiped out of several other segments (mobile, web etc). But don't take it from me: if everything were hunky-dory, they wouldn't be busy steering like crazy towards opensource, something that was unthinkable 15 years ago.

> are you suggesting that Perl would have been a better solution for a Windows interactive shell and automation tool?

From a mindshare/commercial perspective, yes it would have been (and I say this as a complete Perl-hater). By starting from scratch, MS then had to spend a decade evangelising their new solution, with mixed results. It would have been tremendously easier to just offer first-rate support for established sysadmin tools (ssh, perl/python etc) from day 1; but again, don't take it from me: the big new Windows Server features are Ubuntu-bundling and native ssh support. This would not be necessary, were Powershell and Windows dominant from a commercial perspective.

> Powershell within itself does improve CMD.

Kind of depends on how you define improved. I find powershell slow enough that I usually still use cmd unless I'm dealing with a powershell script.

I was administering windows computers at a university at the time of windows 98and most of my automation came from ActiveX where I could register external libraries and script most OS interfaces with vb or js. Powershell is just that plus lots of objects and types

No. The well-known mature shell tools manage files and processes - neither of which are a focus of managing a Windows system. Windows revolves around services which you RPC to, which internally store their state - attempting to faff with their state from outside the service is almost always both undocumented and likely to cause failure.

Powershell allowed those services to standardise their RPCs in such a way that sysadmins could call them easily. Alternatively, you'd have COM or a variety of other RPC mechanisms, depending on the service, often undocumented themselves as they were only ever intended to be used by a GUI packaged with the service.

It's been years since I used Windows APIs -- back then there was OLE Automation, which allowed dynamically, late-bound (call by name, really) RPC calls into COM from scripting languages. Has this been superceded by something else? Using IDL-defined COM interfaces from scripts sounds like something nobody would want to do.

I'm having to do this on a project right now, and I wholeheartedly agree. Nobody would want to do this.

Indeed, nobody would want to do that, which is why Powershell exists. Scripting VB or Javascript against COM or whatever else the software vendor decided to use was generally an awful experience, if the interface was even documented at all. Powershell replaces all of that for system administration purposes - you can even run Powershell commands easily from within any .Net application and get structured, object-oriented output, which is what at least all the Windows Server GUI stuff does now.

You're still stuck with OLE/COM for e.g. Excel automation, though, I think.

> who couldn't use the well-known mature shell tools since they were locked into a closed platform

Yeah, I remember the bad old days, when I'd download that CygWin installer and Windows would just give me the old skull and crossbones and say "You're not allowed to install GNU tools, Sucka!" One time I jailbroke XP and got it installed, but then CMD was all "You can't run grep.exe and sed.exe under CMD, Sucka! Not allowed!" I was so sad that I couldn't use those tools on Windows, so sad.

I read this a while back. It's an interview with the dude who wrote PS https://www.petri.com/jeffrey-snover-powershell

His HN handle is JeffreySnover; He is commenting on this story at this moment.

> a way to do powerful automation without hiring a bunch of PhDs.

I remember writing Windows software with DCOM was difficult, but it was more on the painfulness side than a difficult intellectual endeavor. Sure you didn't need PhDs to automate things.

> So, yes, someone did ask for it - the IT industry.

The part of it stuck in Windows, that is. Vendor lock-in is a powerful thing.

Yeah, DCOM was a task of fillout the correct fields with the correct values, and try again until you get it right. Anyone with passing C++ knowledge could do it through shear brute force.

It's one of the things which made me never want to program on Windows again.

I ported a bunch of COM to Linux in the late 1990's. Not DCOM with the full RPC, but just in-process COM. I had .so shared libs with DllMain and DllCanUnload now in them, I had CoCreateClass, I had a significant part of the registry API implemented in terms of a text file store (HKEY_LOCAL_USER being in the home directory, HKEY_LOCAL_MACHINE somewhere in /etc, ...) and such. IUnknown, QueryInterface, AddRef, Release, ...

That's nice, but having CORBA implementations - why?

To port some code as-is.

Netscape did that with XPCOM [1] they once thought that this is a good idea - i guess for versioning of interfaces; nowadays firefox has moved on from this.

[1] https://en.wikipedia.org/wiki/XPCOM

Haha, funny enough our company did this as well - we had lots of legacy C++ code using COM and wanted to port to Linux. It was not that hard after all.

All those sysadmins would have been perfectly happy with just a properly working unix like shell.

Yes. Like all those early car buyers would be perfectly happy with faster horses.

false equivalency

No, actually it's quite apt.

PowerShell, or at least the concept behind PowerShel, is an improvement/superset over the Unix shells -- and can do whatever the shells can do, plus stuff they cannot do, because they don't support typed entities.

Its problems are not technical or conceptual -- they are ecosystem and historical: lack of compatible tools, windows-only, etc.

Or to the point of the original parent: the industry considers powershell a solution in search of a problem.

Only those that disregard the inventions from Xerox and ETHZ.

Powershell ideas go back to how REPLs work in those environments.

Both PS and bash are horses actually.

And everybody remembers how Ford was forced to reintroduce a carriage with a horse a couple of decades later because nobody was really using those horseless carriages. Right?

https://blogs.windows.com/buildingapps/2016/03/30/run-bash-o...

Unix like shell + Python/Perl/Ruby = car

That being said I really don't mind that MS went the PowerShell route. It improved their ecosystem a lot and I belive was also one of the engines behind the more open minded move towards open source...maybe someone at MS can chime in but I feel like PS was always one of the strong projects pushing that forward from talking to some MS engineers.

I was part of the Hotmail team which consisted mainly of Solaris admins trying to mass administer a giant number of Windows servers. We kept pleading with the Windows team and upper management (forgot who it was at the time, Raikes?) to give us a powerful shell and ssh on Windows. We ended up licensing FSecure's ssh daemon for windows. We also used cygwin, too.

>forgot who it was at the time, Raikes?

It was Allchin when I was there... about that era.

> This is why I find PowerShell to be an answer to a question nobody asked. If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.

How are Python/Ruby/Perl going to give you structured objects from "ps"?

That's the promise of object-based pipes. You can get useful records out of your command-line tools without having to write an ad-hoc regex.

> How are Python/Ruby/Perl going to give you structured objects from "ps"?

The usual answer would be "by parsing /proc instead and returning meaningful objects that expose the relevant parts of the OS" but why would you want to do that when you could just output the relevant data in easy to parse XML or JSON?

I find that to be the issue. You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools. It not being in the format you consider a structured object does not mean it's not a object or even parsable. If you are using ad hoc regex I suspect you are not using all the tools available to you.

I feel like Kernighan and Pike do a much better job of explaining than I could ever.

https://www.amazon.com/Unix-Programming-Environment-Prentice...

> You are considering it just RAW text when it is actually formatted text that has been parsable for years with common unix command line tools.

Parsing command output with sed/awk/etc (ie. "common unix command line tools") is absolutely an ad hoc parser.

Let me give you an example that I recently ran into.

I have a tool that parses the output of "readelf -sW", which dumps symbols and their sizes. The output normally looks like this:

     885: 000000000043f0a0   249 FUNC    WEAK   DEFAULT   13 _ZNSt6vectorIPN3re23DFA5StateESaIS3_EE19_M_emplace_back_auxIJRKS3_EEEvDpOT_
     886: 000000000041c380    64 FUNC    GLOBAL DEFAULT   13 _dwarf_get_return_address_reg
     887: 0000000000424e60   122 FUNC    GLOBAL DEFAULT   13 _dwarf_decode_s_leb128
     888: 000000000043dca0   157 FUNC    GLOBAL DEFAULT   13 _ZN3re23DFA10StateSaverC2EPS0_PNS0_5StateE

So I wrote a regex to parse this. Seems pretty straightforward, right?

But then I noticed a bug where some symbols were not showing up. And it turns out those symbols look like this:

     5898: 00000000001a4d80 0x801058 OBJECT  GLOBAL DEFAULT   33 _ZN8tcmalloc6Static9pageheap_E

Notice the difference? Because it's a large symbol, readelf decided to print it starting with "0x" and in hex instead of decimal. I had to update my regex to accommodate this.

That is what makes a parser "ad hoc". You write a parser based on the examples you have seen, but other examples might break your parser. Parsing text robustly is non-trivial.

Worse, it is an unnecessary cognitive burden. Readelf already had this data in a structured format, why does it have to go to text and back? Why do I have to spend mental cycles figuring out which "common unix command-line tools" (and what options) can parse it back into a data structure?

I happen to find your example of why to use a object a bit hilarious.

You are right, readelf has the object in a structure -- because that is what elf is...

           typedef struct {
               uint32_t      st_name;
               unsigned char st_info;
               unsigned char st_other;
               uint16_t      st_shndx;
               Elf64_Addr    st_value;
               uint64_t      st_size;
           } Elf64_Sym;

If you wanted the object why did you need readelf in the first place? Why not just read the elf format directly and bypass readelf all together? That seems to be what you are advocating by having readelf passing a object instead of what it does today.

You're asking me why I use a tool instead of parsing a binary format manually? Does that really need explanation?

If that is your attitude, why use any command-line tools ever? Why use "ls" when you can call readdir()? Why use "ps" when you can parse /proc?

You just pointed me to Kernighan and Pike a second ago. I didn't expect I would need to justify why piping standard tools together is better than programming everything manually.

I never said anything about not liking command line tools. In fact I love them and think they do a awesome job!

In any case you just proved my point. You think its insane to parse binary data while scripting and I do too. That is why I think the passing binary objects is insane on the shell.

Now if you were talking about text base objects (not binary ones) then that is an entirely different story and I feel that is what we do today. In your example you have rows which could be called objects, and members which would be separated out in columns. To argue a different text base format is better than another is not something I am interested in doing -- mostly because there are a million different ways one could format the output. If you were to do "objects" I think they would have to be in binary to get any of the benefits one could perceived.

To be honest I feel the output you posted is a bug in readlef. I would expect all data from that column to be in the same base.

I will level with you I can see some benefits of having binary passed between command line programs but I think the harm it would do would outweigh the benefit.

But if you you really wanted to do that you could. There is nothing stopping command line utility makers from outputting a binary or any other formats of text. You don't need shell to make that happen.

What I think everybody is asking for is for command line developers to standardize their output to something parsable -- which I feel that most command line utilities already do that. They give you many different ways to format the data as it is. Some do this better than others, and I think that would hold true even if somebody forced all programs to only produce binary, or json text format when pipped.

This isn't about binary vs text, it is about structured vs. unstructured.

The legacy of UNIX is flat text. Yes it may be expressing some underlying structure, but you can't access that structure unless you write a parser. Writing that parser is error-prone and unnecessary cognitive burden.

PowerShell makes it so the structure of objects is automatically propagated between processes. This is undeniably an improvement.

I'm not saying PowerShell is perfect. From what I understand the objects are some kind of COM or .NET thing, which seems unnecessary to me. JSON or some other structured format would suffice. What matters is that it has structure.

I still don't think you appreciate how fragile your ad hoc parsers are. When things go wrong, you say you "feel" readelf has a bug. What if they disagree with you and they "feel" it is correct? There is no document that says what readelf output promises to do. You're writing parsers based on your expectations, but no one ever promised to meet your expectations. But if the data was in JSON, then there would be a promise that the data follows the JSON spec.

If the data was in JSON it would promise a that it followed the JSON spec -- but its not, it follows its defined spec, which in the case of readelf is apparently undefined.

Other programs that expect to be machine parsable define in great detail the output. In your initial post I replied to you mentioned ps. In the case of ps it has many options to help you get the data you want without using standard parsing tools. That is because its output was expected to be consumed by both humans and possibility other programs.

Now take readelf on the other hand. It clearly talks about in its man page about being more readable. Its author cares about how it will look on a terminal and even goes through the effort to implement -W which makes it nice to view on larger terminals. It even shows in print_vma, where somebody wen tout of their way to print hex if the number was larger than 99999. If the author really cared about the ability to be parsed they would have added a OUTPUT FORMAT CONTROL section that would provide you the contract you are looking for. Just saying if the data was in JSON does not solve your problem. Why? Because the author of readelf did not spend time to define its output properly in the man page it is not likely he/she would have implemented a json output type when piped little alone take the time to provide the object structures in the man page.

You say it's not about binary vs text but I don't think that can be said. There are lots of things to consider.

* Speed of encoding and decoding. * Memory consumption issues with larger objects needing to be fully decoded before being able to be used or processed. * Binary data would need to be encoded and would likely result in much more overhead.

Its not clear to me that a binary option would not be better than a text one. Pipes today are not just used for simple scripts and system management.

There are lots of things that concern me, maybe it is just the implementation details.

* Not all command line programs are written with the expectation to be parsed. How do we handle that? Force the programmer to make all output parsable regardless if they ever intended on the program being used in some script? * Would a program put the same thing to stdout even if it was flowing to a terminal? Are terminals not for humans? * Would structure be enforced? One of the awesome things about stdin/stdout is that you can send ANY data you want.

That all said I would love it if programs who intended on their output to be parsed offered a JSON output. I am not against structured output. I am against forcing programmers to shoehorn their output into some format that may not be the best for their program. I think a well designed and documented command line tool that expects to be parsed by other programs will go out of its way to ensure the format is documented and adhered to when operating.

It does follow a standard. It's DSV. Unix tools are really good at handling that. Awk and cut specifically.

> From what I understand the objects are some kind of COM or .NET thing, which seems unnecessary to me. JSON or some other structured format would suffice.

They are .NET objects, which, in some cases wrap COM or WMI objects. The nice thing about them isn't just properties, though. You can also have methods. E.g. the service objects you get from Get-Service have a Start() and Stop() method; Process objects returned from Get-Process allow you to interact with that process. Basically wherever a .NET class already existed to encapsulate the information, that was used which gets you a lot more functionality than just the data contained in properties.

I would like to see how it will look in PowerShell. $(readelf).ShowMeTheSecondStringFromTheEnd or $(readelf).PrintTheLastColumnInHeX?

A comparable PowerShell cmdlet would give you one object per line with properties corresponding to the columns. And no, those properties usually have sensible names, instead of "LastColumn".

Of course, for wrapping the native command you'd still have to do text parsing if you want objects. This was more as a comparison of the different worlds here not so much as "if I ran readelf in PowerShell it would get magically different output".

I've actually become a pretty big fan of line terminated json (all string linefeeds, etc are escaped). Each line is a separate JSON object... In this way, it's very easy to parse, handle your use case, and even pipe-through as JSON some more.

In this case, you can have objects as text, and still get plain text streaming with the ability to use just about any language you like for processing.

First off, readelf shouldn't switch between hex and base10. Secondly, that's DSV, so you shouldn't have written a regex for it. You should have either cut, or awk, both tools SPECIFICALLY DESIGNED to do what you want.

What's DSV?

DSV: Delimiter Separated values. readelf uses a delimiter matching the regex /\w+/. In AWK, this is $FS by default, so AWK will parse this by default. Or you can pipe

  tr -s [:blank:]

To cut, which will give you the row you want.

The Unix answer is why are you using a regex on tab-separated values? Wrong tool for the job.

Of course the problem with Unix is that there are a thousand different semi-structured text formats, edge cases, and tools that must be mastered before you can make any sense of it all. Any time you point out the pain a Unix fan can just respond by pointing out your ignorance.

> The Unix answer is why are you using a regex on tab-separated values?

They aren't tab-separated. There are no tabs in readelf output.

Also if you assume they are space-separated, note that the first few lines look like this:

    Symbol table '.dynsym' contains 164 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND

> respond by pointing out your ignorance.

Indeed.

For the most common use cases there would be a standard function to get structured objects, in Python you have os.listdir() instead of ls, etc. If there is none then you can call ps and parse the result manually or call the system functions with c interop.

FYI IronRuby never made it into a usable state, never came close to other implementations like JRuby or Rubinius and IronPython hasn't received an update since the end of 2014, which for an open-source project like a language implementation means it's nearly dead.

That said adopting Ruby or Python as a shell language wouldn't make sense. Especially Python with its white space aware syntax, I mean can you imagine piping commands at the command line using Python, as I can't. These languages haven't made it as shell replacements in UNIX either, even though everyone complains about Bash.

lately i use nodejs and have replaced the cmd tools with node modules. It is more verbose then piping shell commands but easier to understand and the same script can be reused across platforms.

Really? I have to do this to execute docker commands and it is painful. A lot of child_process I/O and it gets hairy.

I find some of it's a lot easier with shelljs or mz, but ymmv. I also like that streams of line delimited JSON is really nice as well.

> If bash/zsh isn't cutting it for you, drop into something like Python, Ruby or Perl to do the heavy lifting.

or Lua?

Hehe, I was so happy when I came across a Lua module for Windows administration. And so sad when I found out it was meant for Windows NT 4.0 administration. :(

A current reincarnation of such a module would be really sweet, though.

> PowerShell to be an answer to a question nobody asked.

It's conceivable that people working in the MS World who have built up (or inherited) a base of PowerShell code are asking, "what would it take to run this on Linux, or at least some of it in some way?"

Of course programmers who don't currently work with the PowerShell at all are probably not asking "I want the PowerShell on Linux" in any significant numbers.

I've been loving using Node.js for scripts. You get all the NPM goodies plus STDIO, and they end up working just as well as shell scripts.

One of the big advantages IMO is that I only need to master one language for all of my needs, from browser to server to shell :)

How do you find dealing with async? Virtually everything I'd want to do with scripting would almost certainly be async in node libraries, and while I'm willing to pay the callback / promise tax in production code, for scripts it seems like the convenience of just dealing with everything synchronously would outweigh the advantage of keeping things in one language.

In scripts I'll keep things synchronous unless I have a specific need for async.

Personally for me I use es2015 and async/await in shell scripts since it's so close to writing sync code to me. I have an alias to my global babel so it's uber easy to transpile and run the code. I right a lot of node code for work so it's just native to me but I'd recommend others look into it. NPM is such an amazing ecosystem for it.

I tend to favor es2015 + async/await (via babel-node). Combined with bits from shelljs and mz packages, it all comes together pretty well.

Pure madness. I'm sure this is how elated the architects of the Tower of Babbel felt about themselves right before the crash.

There is also Ammonite Shell[0], XONSH[1]

0: http://www.lihaoyi.com/Ammonite/#Ammonite-Shell 1: http://xon.sh/

0: scsh is a nice scripting language, however it's not meant for interactive usage (PowerShell is) See: http://www.alphanet.ch/~schinz/scsh-faq/scsh-faq_4.html#SEC3... Scheme is also dynamically typed.

1: It's the first time I see it, but from a cursory glance and some docs (http://hackage.haskell.org/package/turtle-1.0.0/docs/Turtle-...) I think it doesn't include simply calling external commands (I couldn't find any info on how to wrap an external executable with type, too). Without this, it can't be a real "shell", only a scripting DSL for Haskell. It's not very good for interactive use as it relies on ghci.

2: tcsh is meant for interactive use and is a shell; I used it when working with FreeBSD but never wrote any scripts in it. However, I don't see any mention of types in it's manual (other than file types): http://linux.die.net/man/1/tcsh

PowerShell is statically typed like Turtle, has a nice scripting language like scsh and is meant for interactive use. It's not only typed, but also (partly) object oriented and with full and direct access to all .NET/Mono classes. That also implies that you can write commands in any of the CLR-targeting languages, by the way, like F# or Clojure-CLR.

PowerShell is unique (not literally so, as I'm sure there are other projects doing similar things, but I think it's safe to say that it's unique among widely used shells) in what it provides and in that it takes all that into interactive use. It's definitely worth checking out if you haven't already. Of course, keep in mind its problems, like the mentioned smaller number of already available utilities.

> I think it doesn't include simply calling external commands

proc and inshell. Did you check out the built-in tutorial?

http://hackage.haskell.org/package/turtle-1.0.0/docs/Turtle-...

You're linking to an old version. Here is the current version: http://hackage.haskell.org/package/turtle/docs/Turtle-Tutori...

Yes, I skimmed it. But please note that I wrote about "simply" calling external executables, where you can just type the name of a program and have it run. Turtle doesn't even try to allow this (while PowerShell has it), instead going in the opposite direction:

> Most of the commands in this library do not actually invoke an external shell or program. Instead, they indirectly wrap other Haskell libraries that bind to C code.

inshell "ls" would do what you are talking about, unless your bar for simple is just typing ls.

I was mistaken about tcsh' type system. It's not strongly typed:

    ~> @ i = 0 + "foo"
    @: Expression Syntax.
    ~> @ i = 0 + "1"
    ~> echo $i
    1

I've never seen tcsh referred to as strongly typed. Could you elaborate?

I was mistaken, it's not.

    ~> @ i = 0 + "foo"
    @: Expression Syntax.
    ~> @ i = 0 + "1"
    ~> echo $i
    1

There is also GNU Guile, with a fantastically easy to use FFI. Has a great manual too.

I agree... though my default reach-for these days tends to be node. I don't much care for ps, even if it's now available on Linux, I just don't see myself using it much... I like C# a lot, and even other .Net languages. But I don't like all the extra typing for one-off scripts, even if they do get re-used.

I can paste various commands for other terminals/environments in a tweet, the same can't always be said for ps. It's just a lot of typing.

I love the strongly-typed cookies but I absolutely hate the powershell syntax. It's like C# and Bash got together and had a really ugly baby.

As long as you bear in mind that it's main purpose is a usable replacement for the CMD.EXE batch language, then you should be willing to forgive Powershell it's warts.

Except that they added a bunch of new warts that don't have any relation to CMD.EXE. They took the opportunity to re-invent how a shell works from the strongly-typed object perspective but then they copied some of terrible syntax of 40 year old unix shells. I just don't understand it.

I'd rather write scripts in pure C#.

The point of PowerShell isn't just to write scripts but to work as an interactive shell.

I'm not at all convinced that the syntax is any better for typing in as commands in an interactive shell.

I think it works pretty well for that beyond the weird omission of anything like &&

It works fine as an interactive shell but that's not exactly high praise. It could be better for that and for scripting but instead tries to be more familiar to bash users. As a bash user, I'm terribly happy about it's syntax either but it's the price to pay for 40 years of backwards compatibility. Powershell has no such excuse.

True enough.

It sorely needs an && and a &

That is not quite right. At the end of the day it is all about automation. But we took the position that the way Admins automate is by scripting the things that they do in an interactive shell. Bruce Payette had a great way of saying it - he said that the lifecycle of 99% of scripts starts at the prompt and ends at the carriage return. Then, somethings you want to codify things so you put it into a script. Then as the function becomes more used and more important, you want to make it more and more formal without having to throw everything away and start with a new language.

So YES and interactive shell is important but it is important as a stepping stone to automation.

That is the PowerShell mindset. Jeffrey Snover [MSFT]

That is not quite right either. I enter about 200 commands on a normal workday and write maybe two scripts a month in a shell language. A shell language should be a shell language, it's not merely a stepping stone to write a script after using a command just once. If that's what I'm looking for I'll use Python. And from the other side, Python would be a terrible shell (by default), and that's fine because that's not what it's used for.

If Powershell is made for both, it has to make compromises both as a shell language and as a scripting language. This is one of the things that annoy me about the Windows ecosystem: it ships with no languages beyond a shell, which makes scripting for it very annoying, and you can't even install a language of choice via a package manager (one has to go find a download somewhere on the web).

I thought Windows shipped with .NET which includes a command line C# compiler, csc.exe.

The current C# compiler is no longer distributed as part of .NET. They're separate again (as is MSBuild).

Windows did ship with VBScript (and probably does still). You can also install languages via a package manager; an example:

- https://chocolatey.org/packages?q=python

After installing a third party bootstrapping program. I suppose that's as close as it gets.

Windows still ships with VBScript, but it's not a language I'd want to look into either. Everyone that has worked with it sighs when it comes up and nobody actually uses it for automation as far as I know (everything's either powershell, good old batch scripts, or custom .NET software).

This is a really strange argument -- he's saying that PowerShell was designed so that you could more easily move from shell to script to library ...

Your counter is -- you don't do that? Of course you don't, because your shell is bash. That's exactly the point. If your shell is bash, when you write a script you use a different language. What if there was a shell where you could actually script ...

You're absolutely right, PowerShell is made for both, and it has made compromises (like using comparison operators like -eq instead of ==).

Compromises are not inherently bad.

> Your counter is -- you don't do that? Of course you don't, because your shell is bash.

Yeah of course I'd use Bash because Powershell only just became available for other platforms, but that's not my point.

I understood his point like this: the grandparent of his post said powershell has some bad things for scripting. Someone responded to that by saying it's not just for scripting. He said something which I understand as 'that's not quite right, I see a command as a stepping stone to writing a script', To which I finally responded 'I enter a lot of commands but rarely write scripts in [my interpreter of choice's language] because that language just isn't that great, and that's fine because the language is great for its purpose: command line usage'.

And compromises are inherently bad (in a way), that's the definition of a compromise: there are two or more mutually exclusive options and different parties have different first choices. Since they're mutually exclusive, a compromise must be made which is acceptable for both, but it's the first choice of neither (or at most one of them). Compromises are necessary for a dual-use language, but they don't make it prettier for both usages either.

> it has made compromises (like using comparison operators like -eq instead of ==).

wat.

The use of -eq instead of == is a compromise?

I don't get that.

> If your shell is bash, when you write a script you use a different language.

The number of bash scripts I see for various -n-x related tasks suggests that this is substantially untrue; people do, in fact, do scripts in bash.

> What if there was a shell where you could actually script ...

I can't think of any shell that you can't script. OTOH, interactive use and scripting are substantially different use cases, so it really makes sense to have both interactive-optimized languages that may be usable in script and scripting languages that may be usable interactively, but focus use of each in their optimized role.

You could just use WSH (Windows Scripting Host) with either VBScript or JScript the way you've always been able to. AFAIK, Windows has shipped with that scripting solution since like Windows 2000 or possibly even earlier.

I noticed that on old computers (Windows 95, I think, and certainly 2000 and XP) where it would classify .js files as executables. I was about 11-14 so I didn't understand much of Windows at the time, let alone in a foreign language (I'm Dutch) when I wasn't even allowed to use the Internet (dialing up was expensive per-minute stuff), but yeah I remember that.

Still, I've never heard of anyone using Javascript or VBScript to automate anything. It's either powershell (these days), cmd scripts or custom .NET software. I've never even heard anyone mention using javascript in Windows (until apps came around in Windows 8) and vbs is only ever patchwork for legacy third-party software products as far as I've seen in my sysadmin days.

> Still, I've never heard of anyone using Javascript or VBScript to automate anything.

Well thousands of companies have done it, so. I don't know what to tell you.

> I've never heard of anyone using Javascript or VBScript to automate anything.

as just one example, It's common to include JavaScript or VBScript steps in Windows installers. You would never know it, if you ran an installer that used a script to automate a few things.

You're correcting Jeffrey Snover about the point of Powershell. I suspect that the guy that invented it and championed it through years of resistance at Microsoft probably knows what is or isn't intended with Powershell.

Besides mirroring the condescending tone, I was mentioning that in my (GNU/Linux) experience -- which is a lot more command-line oriented than Windows -- it's not true what he's saying. I use a lot of commands and very few of them become scripts, thus the conclusion that it's not merely a stepping stone. A command line language can have different features than a good scripting language. Further I was remarking on the limitation of Windows which ships with a "one size must fit all" language: the language used by their shell (and now there's a 2.0 version of that, called powershell).

I'm sure he knows a lot more than me about a lot of powershell-related things. Still, beyond him being (according to you) the inventor and "champion" of powershell, there's nothing that tells me they did UX research into this, so I added my experience because it differs from his viewpoint.

I think you're misreading what was exchanged. That is unless you're arguing with him about the intent behind the design of a product you had no part in creating. He was replying to "The point of PowerShell..." not, "The point of shells...". He was simply correcting someone who misinterpreted the intent.

I think you're also reading tone into his message that's not there. He's been very chill and helpful in this thread even in reply to open hostility.

I'm not sure what "according to you" means, but you can verify it yourself. Feel free to read Wikipedia or you can probably find one of his talks about it on youtube.

As one of those enterprise pseudo-sysadmin-scripting-guy, I do agree with the sentiment; and I think the fundamental object-ness of PS was nice and the emphasis on docstrings is nice. However, I still think a python-like syntax would have been miles better. As much as I tried, I just can't get into Powershell -- way too many special characters look awfully "dirty", and $vars give me PHP/Perl-induced PTSD. Deployment is also a bit of a pain.

I would hope it's also meant to put vbscript out to pasture as well.

The implicit return semantics are insane, since you can implicitly early-exit when you do things like call ArrayList.Add (which returns a bool to indicate whether the thing was added).

insane in a good way right? You aren't "returning" you are emitting to the pipeline

Not if you're in a script and declaring a function. So, no, it's a source of confusing bugs. Implicit return works better in, like, Ruby, where it has to be at the end and can't just randomly exit in the middle of a function declaration.

ArrayList.Add ... whatever ... | Out-Null

Its not implicitly returning. The return value wasn't assigned so it is emitted to the pipeline.

OK, sorry (there is an explicit "return" keyword that I thought worked the same way). Either way, yes, I know you can avoid it by piping to Out-Null, but that behavior is surprising.

    return {expression}

is equivalent to

    {expression}
    return

the only place you have an exit from a function, script block or similar is either at the end or at a return statement. Every value that isn't assigned or piped into nirvana gets pushed to the pipeline. I actually can't remember having used return statements more often than once or twice so far (and I use PowerShell daily, heavily).

Then I rarely found the need to use ArrayList. Sure, you can use it, but you can just as well do

    $array = @()
    $array += $item

A personal pet peeve of mine is actually that lots of people write PowerShell very similar to equivalent VBScript or C# code. And that makes for horrible PowerShell code (in my eyes). The pipeline is immensely powerful and not using it often makes for pretty crappy code.

Most people writing PowerShell scripts are likely dabblers though. Anyway, I have to admit I don't really appreciate the whole pipeline idea; I just know that `$something = Some-Function` sometimes wouldn't do what I thought it would because of these semantics.

I know the problem you are talking about - it has bitten all of us.

The new language mode you use when writing classes in PowerShell shifts the needle more towards programming semantics than shell semantics and addresses this.

Give it a try!

Jeffrey Snover [MSFT]

Interesting. I'll have to look into it. Thanks.

Our goal is to be incapable of sustained error. :-)

Jeffrey Snover[MSFT]

I really like the functional approach of PowerShell. The pipeline, while not the end-all-be-all of perfection, is still very, very good and very flexible. Once you get into the functional mindset, it is very easy to write terse, powerful code.

My real wish is that there was a syntax for writing function decorators and function generators. This would be similar to the [cmdletbinding()] syntax, but would allow me to write my own.

Yeah, I wanted to switch to PowerShell but then I had to call an exe with variable arguments from the PowerShell script. That was a quick way back to cmd.

Couldn't you use $() for this?

This was the hardest thing to figure out. I have a program just to ensure I'm sending the arguments that I think I am.

I was frustrated as well by this. There's a program echoargs.exe you can use to find out what you need to escape in your powershell args.

Usually everything works as expected, e.g.

    someprogram.exe arg1 arg2 arg3 $foo (Get-Item bar).Length

Variables get expanded in the argument list, arguments get quoted automatically based on their content. The only gotchas I can think of are:

    & "command with spaces.exe" args

It's a bit unexpected that the & operator is needed.

Arguments sometimes get quoted unexpectedly, or need (single) quotes depending on PowerShell syntax rules to avoid evaluating something.

The worst (but sadly common) thing people can do when faced with problems here, though, is heading for Invoke-Expression, sometimes in combination with all sorts of wrappers, each of them making the problem worse, not better (it cannot get better that way).

The most robust way I tend to use when necessary is to just prepare a single array with arguments and splat that when calling the program:

    program.exe @array

No surprises with PowerShell's parsing in command mode because we prepare the array in expression mode. Things are generally more predictable. Automatic quoting of the arguments still happens, though.

(Side note: My own little echoargs replacement is actually a batch file :-)

    @echo off
    echo.argv[0] = %~0
    set cnt=1
    :loop
    echo.argv[%cnt%] = %~1
    set /a cnt+=1
    shift
    if not [%1]==[] goto loop

)

They fixed this in later versions with the literal: --%

But you can also always use start-process like:

start-process app @'

your arguments and parameters go here

In my admittedly limited experience there seems to be at least 3 aliases for every keyword which makes reading someone else's code a real hassle sometimes. Our ops guys swear by it though.

You're supposed to avoid the aliases in scripts. Or that's what's recommended anyway.

Yes, this makes for much easier to read code for people down the road :)

I think OPS folks (myself included) sometimes create things thinking nobody but them will use them, then an occasion arises to share, and you're stuck with something functional but horrible to read.

It took just one such occasion for me to convert to writing everything from the perspective that 'other people may need to use this.'

There seem to be a few tools to do it for you automatically. ctrl+shift+a if you're using powershell studio for example. Or https://gist.github.com/DBremen/9db6632423d673ff18f6

This is cool, I'd not seen this before! Thank you for the link.

Yeah, it's a mixture of Perl and Bash with sudden C# calls interspersed.

Is this really so useful in practice? I have never used PowerShell before (nor do I have plans to do so in the near future), but I would imagine the downside is complexity, both for user and especially for developers. With text you just read and parse, while with objects you need to know the object type, its fields,... Whatever happend, I hope text stays here (and I have a bad feeling about it ever since I met systemd).

Objects are discoverable in powershell. Enter the object, and lists the object out as text.

Yes, this is so awesome. Can't remember the exact parameter/field? Just dump it to the console.

That what you would do with text? If you have no idea what its outputting, explore it by putting it to the console.

Sure, but with text you might have empty fields that are just omitted. With Powershell you can see that the field exists but is empty.

There's a bunch of subtle distinctions like that which don't sound impressive but end up being really nice when working with it.

What you can't do so easily with text is also just auto-convert that object directly to JSON or YAML or CSV, for that matter, and forward that easily into your favorite JSON/YAML/CSV exploration tool.

Ok, that sounds good. But what if some developer decided that some field would have prefix "ABC" in all values, then the value I am interested in, then some value I am not interested in at the end? (you say this can't happen? Clue: WMI.) Can I take the value, convert it to string, make some magic on it and convert it to another object entirely?

I don't know, I never had any problems with text parsing. I love strong types in programming languages, but in system administration I think it would only get in the way. But as I said, never tried it, so I might be missing something.

> "But what if some developer decided that some field would have prefix "ABC" in all values, then the value I am interested in, then some value I am not interested in at the end?"

I don't see why this would be a problem for PowerShell. Just use Select-String to strip away the characters you're not interested in. The advantages that derive from passing around objects aren't based on all data being automatically in the format you want.

Yep! There's a few ways to accomplish such a thing, but probably the most straight forward would be the Foreach-Object command (aliased as "%") to do a mapping from source object to something else. Free form objects/dictionaries are easy to make and consume.

Select-Object also makes it very, very easy to make new objects

There are many ways to accomplish this (in general). My favorite is that you can create 'computed' properties on the fly which means you can continue to keep the strongly typed object in the pipeline, and still have access to the data you care about.

https://technet.microsoft.com/en-us/library/ff730948.aspx

You use the exact same functions (first, last, substring, replace, etc) as you would normally. Then the rest of the pipeline has it available if needed.

I spend about half my job inside Powershell and had no idea bout this. Very cool.

The default regex functionality is absolutely terrible (maybe this can now be fixed with a PR). You'd write your own cmdlet[1]. A few things are demonstrated in the cmdlet: functions that take pipes, functions that return pipes, anonymous functions, actions to perform before processing a pipe and dynamic objects. The Powershell syntax gets ridiculed for being ugly, but it's beautifully expressive.

> Can I take the value, convert it to string, make some magic on it and convert it to another object entirely?

    Get-ChildItem *.js | Get-Content | Select-Regex 'require\("(?<Require>.*?)"\)' | Select-Object -ExpandProperty Require

In other words: find JS files | read lines | turn regex capture groups into an object (another object entirely) | select a single capture

Ultimately, Microsoft provides WMI cmdlets[2]. You'd have strong types from the get-go and wouldn't need resort to this string silliness.

[1]: https://gist.github.com/jcdickinson/cee4582448300c0d404bbff1... [2]: https://technet.microsoft.com/en-us/library/ee176860.aspx

The default regex functionality is absolutely terrible

How so?

Replicate the above without using [Regex]:: (i.e. use -matches and $matches). It forces you to radically depart from a typical Powershell mindset and approach.

I'm not certain what you're getting at; -match doesn't handle multiple matches so you can't replicate the above just with it? Yes, that's a bit annoying. You could match line by line:

    gci *.js | foreach { gc $_ | foreach { if ($_ -match ..) { $Matches...

which is ugly, but doesn't use [regex]::Matches. But you can do:

    sls 'require\("(.*?)"\)' *.js | select { $_.Matches.groups[1] }

or other variants of Select-String, depending on exactly what data you want, without departing into the .Net Framework too far.

More specifically, you can find out all of the type information about anything you get access to by piping it to 'get-member'

  get-netadapter | get-member

Which gives the following output http://pastebin.com/bM0cBeXb about the type information for the result of 'get-netadapter'. In addition to this, tab completion for properties is available.

Alternately, you can get all of the current properties of any existing object by piping to 'select' (to get a general subset) or 'select * ' to get all properties. Note that this gives all information for the objects in the pipeline.

  get-netadapter | select
  get-netadapter | select *

The first will provide a short list of fields for all network adapters, and the second will provide all properties.

'Select' can also be used to easily filter the properties of items in the pipeline to ones you care about.

  get-netadapter | select Name, MacAddress

You can also just generally filter the pipeline based on generic filters (give me all of the network adapters that are currently not enabled, and have a driver provided by Mellanox).

And ultimately, if you want to use powershell to do the stuff that typed objects are good at, and then leverage command line tools to do the rest, you can if you like.

  get-netadapter | select Name, MacAddress | exportto-csv

Which will take all of the network adapters, gather their name and mac address, and spit it out to stdout as CSV which can then be sent to any number of external tools, or using the '-expandproperty' flag to 'select' can give just a bare list of properties one on each line.

> With text you just read and parse, while with objects you need to know the object type, its fields

parsing is actually a huge pain in general. a lot of unixy tools produce different output when their stdout is not a terminal to be more pipeline-friendly. That output is generally easier to parse (often tab-separated or so), but you still need to know the same things about the output (the "type" could be whether it came from `ps` or from `ls`, and the fields are just in some static order that was usually chosen decades ago, and if it's extensible it's cumbersome).

If `ps`, `ls`, etc produced typed output, I would be sooo happy.

When I worked at google I thought about what it would be like to have a unix userland where all the parts communicate with protobufs instead of raw text. There are some problems (like portability of message definitions, which was a solved problem within google, but which is harder when you have multiple parties/entities/companies contributing message definitions).

edit: accidentally a word

I wrote a bash script to sort processes by age on an old redhat 4 machine.

What sounded like a very simple thing, quickly ended up being a complete nightmare of bash, python, and an unholy bunch of linux commands.

It was ugly, and a terrible way of doing it, but we got it working.

It would have been amazing if i could pipe PS into something that could sort by the dates on the 4th column, and return the last column in the sorted order.

You could probably hack something together with awk, but this sort of thing is exactly why I like powershell a lot.

Yeah i ended up using awk, some python for the date parsing and sorting, back to bash for grep and another thing i can't remember, then into the program that actually needed the data.

It was a mess, and I remember thinking it would be such a simple thing that was so much more complicated than i thought it would be.

It's entirely possible this is a recent development, but "ps -o cmd --sort etime" seems to do what you're asking for.

sadly that wasn't available, and researching an easier way to do it without that was met with pages and pages of people saying to do that...

There's literally an example in 'man ps' that does exactly this. It's right there in the output formatting section.

    ps kstart_time -ef

Pretty sure the section in the `ps` man page called OBSOLETE SORT KEYS indicates that even the `ps` on very old redhat machines could do this out of box... see the field `start_time`.

Failing that, consider the `etimes` field (seconds since process was started) used in conjunction with the -o feature (control output format) so you can put `etimes` up front then simple pipe the output to `sort`.

Those things being said, the situation you describe is indeed hairy. `ps` output was designed for human consumption, that's why the process age field format varies (1w vs 10:15) and that's presumably why you wanted to bring python into the mix!

...You might have had an easier time looking in /proc...

Indeed, this looks promising:

    cat /proc/[0-9]*/stat|sort -n|less -S

EDIT: hm, I don't know how old /proc is!

EDIT #2: But try doing ANY of this on an ancient Windows machine! The "peer" of your old Redhat machine would be what, a Win2k3 machine, or a Win2k? How old is PowerShell? :)

Honestly it was a quick and dirty one-time project, and I just needed to cobble something together.

I was more just commenting on the fact that with a "strongly-typed" shell some of this stuff becomes much easier without every single tool needing to add all these options.

It gets much more "Unix Philosophy" when every command doesn't need it's own sorting logic, and doesn't need it's own date display logic, and doesn't need it's own way of displaying it to the user.

It can output an object, which can be piped into another program to use what it needs which can be piped to something else to do some sorting which can pipe it out to another to display it in a nice table.

I hear you. I myself was impressed by some things PowerShell can do easily.

But let's not throw out the baby with the bathwater... The unix shell ecosystem is powerful magic. There are LOTS of things that can be done there with great ease.

Not to mention--and I understand full well that this stance is falling out of fashion--MICROSOFT ... KILLED... my PAPPY! :|

PowerShell was released after Win2k3, however you could install it on Win2k3/XP, even PowerShell 2 afterwards: https://support.microsoft.com/en-us/kb/968929

I find it massively useful in practice, even for mundane tasks.

For example, the other day I was trying to search the columns of a very large SQL Server table¹ but also needed information about the column type.

  …\dbo.site> ls Columns | select Name, DataType, {$_.Properties['Length'].Value} | where {$_.Name -like '*int*'}

  Name            DataType $_.Properties['Length'].Value
  ----            -------- -----------------------------
  internal_end    datetime                             8
  internal_start  datetime                             8
  interval_code   varchar                              5
  prod_intv_start int                                  4
  prod_intv_stop  int                                  4
  working_int     decimal                              5

Quick, easy. In a *nix shell that would be very tricky.

¹ SQL Server installs a Powershell extension so that you can essentially treat databases like a fancy object filesystem. It's wicked cool actually.

> Quick, easy. In a *nix shell that would be very tricky.

You'd probably need to make an explicit SQL query since you wouldn't have the database integration.

Well, yeah, but I mean the concept (grepping in tabular data).

If, hypothetically, bash had database integration, ls Columns would probably return something like

  foo	varchar	30	NOT NULL
  bar	decimal	 8
  baz	int	 4

Which would be a royal PITA to search for anything in—if I were to search for ‘int’ I'd get all int columns and any columns with int in the name….

I could've used an example involving process lists, but I didn't want to trigger PTSD in anyone who's had to grep the output of ‘ps’ for anything non-trivial. (Plus, the database thing was something I actually did a few days ago.)

But thats why there are tools like awk.

awk is rather similar in many ways to powershell, actually, but untyped. I agree most of what powershell does you can do with awk, but it's often less pleasant than powershell (in my experience).

ps aggregates /proc info for human viewing. Why grep the formatted output rather than the source data?

Maybe on Linux. OpenBSD doesn't even have procfs, FreeBSD deprecated it, DragonFlyBSD doesn't mount it by default.

/proc is a race condition waiting to happen, so I can't really say I recommend using that.

How would that be hard in a *nix shell?

    mysql > SELECT  column_name, data_type, numeric_precision FROM information_schema.columns WHERE table_schema = ? AND table_name = ? AND column_name LIKE "%int%"

Imagine for a second that the DB can't do it and you need to filter the STDOUT output.

Okay, I've imagined it. It does seem terrible.

Fortunately, that's not a world any of us live in.

OK, but now imagine it's not a database, because there are programs that do act this way.

And those programs usually emit proper CSV. You could use regex to match on what you want, but AWK was literally built for this. It's fantastic, it works great, and if you don't use that tool that was built to solve your problem, that's your own lookout.

I don't think awk is actually capable of parsing CSVs in all cases.

It doesn't parse escapes, IIRC, but those are rarely emitted.

Er, it's a world all of us live in. Say, for example, ‘ps’ or ‘ls -1l’.

Then you end up using awk, which is rather like a half-assed version of powershell to work around shortcomings with the unix everything-is-a-string design.

Parent comment was talking about parsing the raw output from a database in stdout. That's what I was responding to, and I'm not wrong.

AWK isn't a workaround. It's a tool for processing tabular data, in a textual format. The fact that it can solve so many problems is actually a strength of the EIAS design.

That doesn't sound hard at all. AWK, anyone?

Writing awk to do it is definitely more work than calling obj.Property though.

  $mysql_cmd|awk -F\| '$2 == "what you want the second row to match" {print $0}'

Not that much more work. And you can do better if you can coax mysql to emit proper DSV.

That kind of stuff adds up fast in an interactive shell.

...Which is why you write a script to do it if it comes up often.

Extendable tooling!

The reason I chose that particular example was not for the database integration (which is very cool) but for the fact that it's actually something I did.

Point is, imagine trying to grep anything tabular in bash. Hypothetically, the output of ls -la for files with execute permission that have been modified in the last day. Or, more realistically, anything involving ps.

Imagine trying? Sir/Ma'am, I have done so.

When boxed in-between the fearsome foursome--`bash`, `sed`, `awk`, and `emacs`--every problem involving tabular data (2D) is trivial. Hm, I'll give a shout-out to `sort` and `uniq`, too! When you go 3D, you might need to bring in `join`.

But anyway, you'd use

    find ~/bin /usr/bin -mtime 1 -executable -type f -ls

..to provide `ls -la` output for every executable, ordinary files (not directories, which are often +x) modified in the last 24 hours, under the directories /usr/bin or the bin/ under your home directory.

Hey, don't forget cut and xargs!

    *gulp* I've heard people say that, but I haven't delved into them!!  :-S

cut doesn't match awk, but it's handy for the little stuff. And xargs is the most usefull thing ever.

And paste for tasks requiring the opposite of cut, then there's tr, the lightweight sed. And jq for structured queries on JSON. But, all these are built in to poweshell, essentially.

No they're not. PS can't process text all that well...

Most users do not find awk trivial. Most people basically use it for {print $colnum} when simple columns are available.

PowerShell's integrated query syntax that you can patch into various things is a lot more consistent and reasonable. It is difficult to imagine volunteering people into a world of awk, sed and cut with good intentions in one's heart.

I didn't call awk trivial, I meant to say that the prudent application of powerful tools makes quick work of otherwise intractable problems.

Forgive my being frank, please, but I think that given the context of this conversation, it's appropriate: Most users don't HAVE non-trivial problems!

There aren't features in sed nor awk, nor regular expressions in general that weren't put there for some specific reason in the past.

When you wield your shell, well, you wield a tool honed through the ages by people who think like you! Do so with pride! :)

I have no time for tool worship, language worship, and other sentiment like that. These are tools, not people. I'll use the best I can buy, borrow, or make until I see another better one comes along.

A corporation has no incentive to give you (let you keep forever) the best possible tools.

If that is true, then it is equally true of the authors of shells.

But of course, it's not true. Many great and lasting tools have been founded out of corporate work in a bid to improve mindshare and ease recruiting by increasing the esteem of corporate engineering efforts. Many of the things we think of as "open source" and "free" were built subsidized either directly or indirectly by capitalistic interest.

But I'm not concerned about keeping it. Ossifying my process so that I can give myself the luxury of time of from learning is not something I'm prepared to do, because I think it will make me soft and less capable as an engineer. I regularly rotate my editors and learn new languages because I don't want to become like the sad specters of the previous generation of software engineers I see now, starched shirts and ties and a fanatical devotion to the dated technologies they stomach in order to collect huge consulting fees from corporations.

I've switched off Fish to Powershell as my primary shell for awhile. It's interesting.

I think this speaks to the difference in mindset I've seen a lot in this thread. It basically comes down to people who have gained and honed experience with these tools vs. people that haven't. Yes, for someone with a lot of experience with these tools something like Powershell is redundant... for them.

Some people just want to get shit done because this is just a footnote in a task that's a small part of a project. Utilizing gained proficiency in several tools is cool but if you have a choice between doing that and just taking advantage of a simple tool that streamlines things and lets you move on to your next task quickly the correct choice is obvious.

It could be that these tools are better and may be more valuable in some situations but asking someone to spend time gaining proficiency or mastery when another path is available is just premature optimization.

Pretty much all databases will support "describe tablename" to do exactly that.

I've done some Powershell work in my brief time as a Windows sys-admin. I personally struggled with it. The object interaction can be complex and the naming feels less scriptable and more directly out of the C# world.

That being said, I can see the appeal and I know people who love it. I have seen some amazing shit written in it, like this security tool that was just open sourced this year:

https://github.com/adaptivethreat/bloodhound

I have used Mono/GTK# for years on Linux and prefer it over other GTK wrappers. Powershell might be worth a try on Linux, but it will take time to build up the libraries and wrapper needed to interface with most of the Linux subsystems (unless Microsoft as ported a lot of useful stuff in this initial release).

People don't (or shouldn't) use bash/zsh/fish for really complicated stuff anyway. I have a feeling most sys admins/devops people will continue to use Python/Ruby for complicated tasks; and they both have a ton of libraries that help avoid the stdin/out/parsing situation you have with chaining together regular shell scripts.

Python and ruby for the big stuff, bash and the Holy Trinity (sed, grep, and find) for the small stuff. AWK for anything in between.

You can just use the Mono bindings for the Linux-specific API's, I hope.

It's nice when it works, but it's a pain to write a command-line utility that outputs into objects. Whereas in Unix, everything automatically outputs into text, so you have to work hard to prevent it from being chainable to the system.

There are also details of Powershell that are very annoying, such as the lack of < redirect, and the difficulties it has on binary output sometimes.

difficulties with jumping through hoops to consume and produce binary output are really the only "problem" I've had with powershell

Yes, it's quite useful in practice, because the alternative is knowing exactly the format of text some command is going to produce and munging it, which is hardly better

Pipe anything to Get-Member. For example:

(ls)[0] | gm

will list every field and its type on a File object. Wicked easy.

You can even just do

    ls|gm

and get the lists for both FileInfo and DirectoryInfo. There will be no duplicate types listed because Get-Member already sees to that there won't.

Could you elaborate on what the actual complexity is, rather than what you imagine?

For a developer writing a tool, exposing an interface as objects is far easier than having to constantly serialize data into a string. Keep in mind that once you serialize data into a string, you can never change the order of the data or else other tools will break. Not so with a object based interface.

In other system-admin type tasks PowerShell hides complexity. For e.g. finding the IP address of a network interface is simply

  Get-NetIPAddress | select IPAddress

compared to something horrendous like

  ifconfig | perl -nle 's/dr:(\S+)/print $1/e'

What about:

    ip addr show | grep inet

Edit: wrote back original command and replied below instead

That doesn't work on my CentOS box.

  [kapil@localhost ~]$ ip addr show | grep inet
      inet 127.0.0.1/8 scope host lo
      inet6 ::1/128 scope host
      inet 192.168.128.236/24 brd 192.168.128.255 scope global dynamic eth0
      inet6 fe80::215:5dff:fe80:501/64 scope link
  [kapil@localhost ~]$

Correction then:

    ip addr | grep "inet " | cut -f 3 -d " "

It still doesn't work (you forgot to add the tr back in your reply after your edit)

  [kapil@localhost ~]$  ip addr | grep "inet " | tr -s " " | cut -f 3 -d " "
  127.0.0.1/8
  192.168.128.236/24
  [kapil@localhost ~]$

To be honest, I don't consider this to be that bad, I was just replying to a person who thought that bash was "simple", and powershell was "complex". Also, I don't have a problem with either tool. I'm very open to the idea that Microsoft can and does create awesome tech, despite having other shitty products.

Thanks, too many edits indeed!

Nice way to prove the point that object based is far more reliable than text based.

ip addr | awk '/inet / { print $2 }'

That doesn't work either.

  [kapil@localhost ~]$ ip addr | awk '/inet / { print $2  }'
  127.0.0.1/8
  192.168.128.236/24
  [kapil@localhost ~]$

You guys are just proving my point. _I_ already know how to do it using bash. Its just that using objects and querying properties is far simpler. And I just took a random example because its a common thing that someone would want to do.

You really couldn't have asked for a better chain to demonstrate the validity of your point. Having been blinded by my own world view and biases in the past I sympathize with them but this is too funny not to have a little laugh at their expense.

I think what others might be demonstrated is that there isn't a reliance on what object Microsoft decides to ship to you, in *nix we can combine commands and builtins and get the information in the structure we need without having to wait 8 months for a system reboot (or worse a paid for upgrade) that will give us what we need.

Well, to be fair, what was demonstrated here was that people didn't actually extract the data after multiple edits and tries, regardless of what tools they had at their disposal.

I think the idea of an object pipeline is sound for precisely this reason, but I don't think it should be connected to any one language or platform.

First of all you are incorrect in your assumptions. You are reliant on the author of the tool in both cases. Just as there is no "UNIX Person" who decrees that all executables on UNIX must output text, there are vendors other than Microsoft who write software for Windows. You can write one too, it takes about 5 minutes to expose a powershell interface. And I'm not even a .NET developer. I write embedded software for a living.

But once again, you're simply scraping text, instead of querying an object for properties.

It is parsing text, but that text does have some consistent structure. Contained within it are properties. One can make the same comment of reading JSON or XML but I'll admit those are more like what you'd choose for an API. In general there is a not-entirely-obvious (to me) question of where does "formatted for machines" end, and "formatted for humans" begin? We can't say "it's for machines if and only if it must conform a schema that's defined in advance", not any more.

You would have to be aware of that fact in order to write the command. The question is about relative advantage, not about what the tools do.

> Could you elaborate on what the actual complexity is, rather than what you imagine?

I can only use this feature by using a .NET language. I can't take the strongly typed objects into a Node.js script, or a perl script.

>I can only use this feature by using a .NET language.

It goes without saying that a binary interface requires use of a system that understands that particular binary interface. With .NET being open source though, anyone is free to write a bridge to using the objects in COBOL if they wanted to.

>I can't take the strongly typed objects into a Node.js script, or a perl script.

Yes, because they have no concept of strongly typed objects. You can convert objects to strings if you're using an incompatible tool/language. I don't know what your point is. Sorry..

I guess the point is the point. Potential bridges that do not exist already and probably never will do not matter -- extending that idea of potentiality I could say that you are free to reimplement the CLR in Fortran, so PowerShell is compatible with Fortran? Oups sorry you already reached that ridiculous point, only with Cobol.

Virtually any language can already input and output text. That's the real lingua franca of computing (even if it is messy and dirty), not .Net objects...

>Potential bridges that do not exist already and probably never will do not matter

The difference between impossible and possible is important. Please read what I was replying to. Using your argument, Anytime a project is open sourced, you could reply with "who cares about potential since nobody has actually done anything with the source". Good job.

>Virtually any language can already input and output text.

Um, you seemed to have missed the obvious point that objects can also be converted to text through powershell if the other side can only accept text. So, I don't quite know what you're complaining about. Could you detail your actual technical complaints with powershell, rather than having a philosophical argument?

>That's the real lingua franca of computing (even if it is messy and dirty), not .Net objects...

I don't agree with your opinion, nor with the idea that only a two-choice system can exist.

The point is that this thing is less valuable by being limited to only being used by things that Microsoft created (in reality). Text, with all of its flaws, can be used by everything.

That's not really true, you can serialize as json in all of those

As someone who recently got his first exposure to PowerShell, I'll chime in that the case I needed to deal with was one where it added complexity.

In summary:

I need to pull data from a company we're partnered with, and get it into our systems. They ship it as SQL Server backup files. OK, no big deal, we can mount those up and run queries to dump stuff out to whatever format we like and ingest it.

Except part of that data is files, which were stored in the DB as binary blobs (in a table along with filenames and metadata). Need to get those out of there and into our storage as real files. OK, no big deal, I can just query for filename and file contents, dump out onto disk and then batch-transfer into our archival storage, right?

Except no matter what kind of obvious shell-idiom method I try to use, it doesn't work because PowerShell stubbornly insists that I must be working with text data when I'm using all those handy shell constructs, and so the files end up corrupt.

So now instead of just being able to shove data through pipes like I would on Unix (which, for all its faults, doesn't really care whether I'm sending binary or text data), I have to have my script go and instantiate objects for creating a file and reading and writing binary streams, and remember to clean up after them, and my script ends up way more complicated.

What would have been a pretty simple script anywhere else now has to know a bunch of .NET libraries and how to create and work with their types, and that's less useful to me.

Its true that powershell being a new tech probably doesn't have the large library of "known ways" to do things that the UNIX world has. And BTW I'm not even a powershell person. I write embedded software in C++ for a living ! Having said that, I'd imagine you would parameterize something like

  Invoke-Sqlcmd -Query "SELECT ..." and write it out to a file?

or avoid it and just use BCP "select blah" queryout "c:\file.jpg" etc etc

I just now have the chance to use Powershell on Linux, with that Powershell command you demonstrated, I get a command not found.

And how to delete files older 2 days? Get-Files / | select olderThan 2 | Delete-File ?

But it's not bash, it's coreutils

  find ./ -mtime +2 | xargs rm

    Get-ChildItem -File | Where-Object LastWriteTime -lt (Get-Date).AddDays(-2) | Remove-Item

If you prefer shorter (and similarly obscure if you don't know the language):

    ls -file|? LastWriteTime -lt ((date)+'-2')|rm

I actually find PowerShell's version more readable, even if more verbose. But code is written only once and read a lot of times. I may not know what the +2 actually means with the -mtime argument. How would I change it to look for the age of two hours instead of days?

> I actually find PowerShell's version more readable, even if more verbose.

Sorry, I can't agree with you.

> I may not know what the +2 actually means with the -mtime argument. How would I change it to look for the age of two hours instead of days?

If you forget or doesn't know what is mtime, you can open man find. For less granularity than days use mmin.

    ls|? LastWriteTime -lt (date).AddDays(-2)|rm

Just as an aside (and this probably applies to powershell as well), what if you have mapped folders from multiple servers into your file system and those servers are in different timezones? Something like

  /Mapped-Temp-folders/Server-Asia/temp
                      /Server-UK/temp
                      /Server-Japan/temp

I guess the script wouldn't work then if you wanted to clean out data older than 2 days, as per their timezone.

It's not about complexity; it's about robustness, and avoiding insane hacks like this:

    REASON=$(echo "$line"|tr ' ' '\n'|grep reason= |cut -d= -f2)

I really hope this drives a new wave of experimentation and development in shell interaction. Text as the lowest common denominator will never go away but having more powerful constructs available will be great for a lot of use cases.

Check out the language I wrote for that exact use case: http://tkatchev.bitbucket.org/tab/

I wonder what's the difference between an object-pipable shell and a lisp machine (only 10% troll here). Kalman Reti said that passing pointers as IPC meant zero perf penalty btw. Text serialization has design value sometimes, but the amount of cpu cycles allocated on string munging seems flabbergasting.

> Once you've piped strongly-typed objects around between your shell commands, text scraping seems barbaric by comparison

This is why Powershell is a great thing on Windows. Windows works with .Net objects and Powershell works directly with those objects.

It's also why I don't see the reason to be excited that it can run on OS X and Linux now, because neither of those systems work by passing .Net objects around. You parse text with bash/sed/awk/perl/python ... because *NIX works on text based config files and commands that output text.

> Windows works with .Net objects and Powershell works directly with those objects.

I'd say COM, WMI, and the registry are more prominent parts of Windows itself than .NET is. However, PowerShell seamlessly works with those as well.

I'm assuming that the Powershell port includes some subset of the standard cmdlets, which will handle serializing to and from objects.

A subset means it doesn't work as well or the same way.

No... It just means that all features may not be available. It makes no comments on how well the existing features work and how compatible they are with the Windows version.

I mean, `Get-ComputerRestorePoint` and `Get-WmiObject` don't mean anything on Linux. I assume those won't be ported. `Get-Process` is certainly meaningful; I assume that will be. The ports might also have cmdlets not available on Windows. I could certainly see cmdlets to interact with `apt` in Ubuntu, or `yum` in RHEL. Especially since PowerShell is focused on provisioning and managing machines.

Strongly typed cookies you say..

Maybe we all can just write our scripts in Haskell and eat some delicious type cake!

Along the same lines, for Linux and MacOS: https://github.com/geophile/osh

Interesting, nice work

"text scraping seems barbaric by comparison"

"can't use all that power for much, since the pool of PS-compatible tools and utilities is much shallower than it is for Unix shells"

The explanations are obvious though... aren't they. Text might be barbaric, but every programming language in the world makes sending text to stdout dead easy. The unix shell utilities are smart to support that case.

Elixir's REPL (iex) makes a surprisingly-good environment for "pip[ing] strongly-typed objects around between your shell commands." It could use a form with added convenience magic, the way Pry is for Ruby, but I've managed to do some decent interactive ops work on systems through an Erlang-remote-shell (remsh) connection to an IEx shell.

IIRC correctly, the major PowerShell push started with Win Server 2008, and headless server mode (no desktop). A lot of WinX sysadmins pissed their pants that they couldn't use GUI's to control everything, and PowerShell was pushed as the way to do server management w/o GUI's (crazy, I know).

I don't think it's too late for PowerShell, I think it's too late for UNIX. You need a big CLR ecosystem for this to play out.

If somebody resurrected Midori as a CLR Unikernel I'd be very pleased.

Care to post a few simple powershell examples that do the sorts of things those familiar with typical nx shell scripting might find useful?

It's junk. Tried to get something useful done and version incompatibilites and poor documentation made it a losing battle. Type theory is over-wrought for getting things done. My boss doesn't care that I have to cast an integer to a string.

I'm hoping this means running EF migration scripts on Linux will be less of a nightmare. It was one of the biggest issues I ran into developing a Web application in .net hosted on Linux.

Powershell is aweful

Back in the day I thought VAX/VMS was amazing. Particularly things like file versioning, however, I had to leave it behind and do things on UNIX (and then Linux) terminals. That change was relatively painless even if I did delete a file or two that I should not have. However, I have moved on and nowadays I would not have a clue how to use VMS beyond typing 'dir'.

If the VMS 'shell' was suddenly available on Linux, despite its 'proven' benefits in various scenarios, I just would not have the working knowledge to get going with it again. Furthermore, there would be no easy answers on StackOverflow to do basic things such as recursively renaming part of all filenames, or batch modifying the content of said files.

The problem PowerShell has for me is that it might as well be a souped up version of VMS (or even BBC Basic). It is not that I am adverse to learning, it is just that I left Windows World a long time ago. Much like my move from VMS to UNIX, there were some good things that I left behind (such as the file versioning thing), however, if you move to a new way of working then there is no need for the old stuff. I appreciate that PowerShell is new, but it does not suit my way of working, i.e. cribbing answers from StackOverflow and tutorial guides.

I appreciate that PowerShell is new, but it does not suit my way of working, i.e. cribbing answers from StackOverflow and tutorial guides.

PowerShell is ten years old and there's 37,000 StackOverflow questions tagged to it - http://stackoverflow.com/questions/tagged/powershell

Reply on news.ycombinator.com