Jeffrey Snover and the Making of PowerShell

If anyone from MS-PWSH team reads this, I'd love for y'all to add some basic GUI functionality that doesn't involve me having to write a bunch of .NET. I'm sorry, but the reason I like PWSH in the first place is it's a simple dynamic language with lots of easy to use commands that I can chain together. I'd love to have a new set of cmdlets for creating simple user interfaces and charts.

For example, something like the below would be so simple for Microsoft to add to the product and remove a page of boilerplate code that I don't really understand well.

Create-Chart -Type "Bar" -XAxis $Cities -YAxis $GDP -OutputFile "C:/Documents/ProjectAnalysis/CitiesBarGraph.png"

There are probably users in the millions that are ok at the basics of programming, but don't have the job role to where tools like Java or C# make sense. Python is usually a good fit here, but I really wish Microsoft had something written for us common folks and not just server admins and IT folks.

If Microsoft put some more effort into PWSH to where it wasn't turtle slow at things like parsing files and then started adding things like what I talk about above. Maybe even cmdlets for statistics and science...it could be something pretty amazing that your typical business analyst could quickly use to build some really amazing software to do their job better or a prototype for the software team to actually implement in a more robust manner. It's such a really cool technology that has a lot of missing potential IMO.

It seems like Microsoft assumes that the three options are full software developer with C#, IT stuff with PWSH, or Excel for the business folks. Excel is really great in a lot of ways, but it is also pretty limited and VBA+Excel is one of the most limited ecosystems I've dealt with. I guess third party languages like Python, R, and so on make for another fourth option, but sometimes I wish Microsoft had spent more time in this area.

And as far as I can tell its no longer a priority at MSFT, reading the complaints from the MVPs on Github about how MSFT promised a bunch of further investment that has not been realized, changes not implemented, good stuff just sort of left to rot on the vine.

I loved powershell with (some of) its weird warts, but I have moved on.

As someone who prefers to read interviews and articles rather than listen, I really appreciate you providing a transcript. Thank you!

I wonder why they chose to build something from scratch instead of using python or similar tools that existed before.

> I've gone from professional transcriptions to Descript and then a pass of GPT4 trying to find the right punctation breaks and then me doing a quick read through. I don't think its coming out as high quality as I'd like.

I tried reading the transcript and couldn’t reach the end, it’s a bit hard to read in my opinion. I assumed it was machine generated while reading it, but I can’t say why specifically. Maybe it needs a bit of editing to be easier to read.

Thanks for the effort anyway, it’s still better than no transcript :)

> the transcript has errors. I've gone from professional transcriptions to Descript and then a pass of GPT4... [...] I don't think its coming out as high quality as I'd like.

I wondered why there were so many unintelligible passages. I found it laborious to decipher. Thank you for explaining.

>PowerShell faced extreme opposition at Microsoft, and its creator Jeffrey Snover was demoted for pursuing it.

That makes me sad. The dude is absolutely brilliant.

Awesome post!

That was a great listen, thanks.

I've done a lot of Bash, read some books about it, I'm a firm believer that nobody should be writing anything complex in bash. Above the ballpark of 50 lines I consider it a code smell. I have this web page saved up in case I ever need to convince anyone of this.

http://mywiki.wooledge.org/BashPitfalls

I used Powershell recently and not having to wrangle with text (commands return objects) makes it a much easier scripting language, and command line language.

The fact that they have an official way to handle argument parsing is excellent, everything is unified and the commandline window is able to have autocompletion for literally every option of everything. Bash could never even dream of having that. It is incredible and makes you extremely productive.

But type coercion manages to introduce new ways to create bugs that Bash didn't have.

Honestly at this point I prefer PWSH but still kinda dislike both. I'm waiting for the new natural evolution.

PowerShell is more verbose than Bash, and has its idiosyncrasies (like sometimes automatically unwrapping 0/1-element arrays into scalars when you least expect it), but it is more productive and more readable. Object-orientation can be nice to produce pipelines.

For example: find potential file duplicates in a folder, recursively, by grouping by file size:

    Get-ChildItem -File -Recurse | Group-Object -Property Length | Where-Object { $_.Count -gt 1 } | Sort-Object -Property Count

No need to remember arcane `file` incantations, no need to parse textual output. You get real objects with properties, and tab completion can see their structure.

Need to parse JSON? No need to involve `jq`, just `Get-Content -Raw whatever.json | ConvertFrom-Json` and do it directly from PowerShell. Need to convert XML into CSV? ConvertFrom-Xml (or Select-Xml) -> do stuff -> ConvertTo-Csv.

Is `Get-ChildItem` too much typing? Use `gci`, or `dir`, or `ls` (by default only on Windows). Is `Where-Object` too much? Try `where` or `?`. And things are case-insensitive.

I started my career with a X term hosted on a SunOS pizza box, have written thousands of scripts, more than a few in the KLoC count.

Several years ago, I had to write a complex unattended robust data transfer system in PowerShell. (I know, I know, without more info these “requirements” beg many questions, but they are all out of scope for this reply.)

I enjoyed the experience so much I switched all my shells, on MacOS (my DD) and on Linux (my most common work environments) to PWSH.

What I liked most about it was the power of passing objects in pipelines, and being able to extract/manipulate some of the properties of an object in the first filter and still have access to others, along with those of objects created by that first filter, in filters later in the pipeline.

Immensely powerful.

The consistency of commands, of error handling, and of object properties was also very nice.

Eventually, as the nature of work changed I switched all shells back to bash, as the older muscle memory asserted itself. PWSH as a shell made sense when I was working and thinking so much in that space, but when I left it, it was more effortful to think in PWSH than to resume bash.

There are times I miss it. There is nothing else in the shell space that comes close, or at least not close enough to justify the effort of switching.

No expert in either but I've had to use both a fair amount. Powershell fell into a few traps that rendered it awful to use IMO:

* Trying to be a .NET language. I don't know why the one-runtime-many-languages promise of .NET withered on the vine while it flowered in Java-land without the promise even having been made, but it did. If you're writing .NET code, use C#. (And I say that having worked in one of the very few teams doing large scale development in F#.)

* Failing to get the basics of being a shell right. I can't remember the details as I've left it firmly in the past now, but its handling of redirection was just broken, to the extent that things that were trivial in bash were nigh on impossible in powershell. I got the impression its developers ignored the things bash etc. did well in their enthusiasm over building something new and powerful.

* Being fetishistic about stuff like requiring Verb-Object naming for everything. It's certainly subjective and has its defenders, but IMO it renders scripts ugly, is awkward to type and doesn't materially improve discoverability or memorability at all.

I don’t like powershell because it’s so surprising in a billion ways from bash. The tab being replaced with right arrow screws me up, and so does not having the normal commands, and it’s quite opinionated about naming.

That said, powershell seems more capable than bash if you know it well, because it has a better type system and arguments are easier to mess with, whereas bash things are amorphous strings.

https://github.com/bionicles/tree_plus/blob/main/tests/more_...

That’s a slightly out of date version of what I use to provision stuff on my windows machine to test things, and can show what’s possible a bit.

I find it is in a weird spot where it is more powerful than bash but you often don't need that power or if you do coding in c# might be better.

It is very good of you want to work with Azure though. You probably don't want to use bash for that. But lots of reading up each time I use it. Bash is my go to for small jobs or a command line experience.

I am not an expert in either but have used both plot to get stuff done.

I used a Powershell-script in a recent project and learned some lessons. It gets some data from an API. It is run dozens of times every minute, can run concurrently with itself, has some writing to a json file cache in a race condition free way. Works mostly ok. Development has been difficult because of low volume of forums talking about Powershell. Wouldn't have bothered to make the script without ChatGPT. One time it failed by inexplicably setting system+hidden+read only flags in a project folder. Quite a head scratcher. The worst experience was a bug I still don't understand: The script failed to get an access token when running in "ordinary powershell" but managed to do so when running in Powershell ISE. I checked every possible environment difference and concluded that there was only one: powershell.exe vs powershell_ise.exe. I am not going to be making any more Powershell scripts.

Well I used powershell a lot when I was managing VMWare infra. Ultimately the force of a object oriented shell is in the ecosystem, I haven't found any other use case where powershell was useful to me. I am pretty sure it is useful in an Azure context too but in the last few years I have never been in a situation where I would feel the need to install powershell on my linux workstation to use a specific module/use case.

Powershell is a huge improvement over cmd. However; much like Windows, it is full of complicated gotchas and edge cases. Bash has them too, but Bash makes concurrency and parallelization extremely simple, so despite it's absolutely bananas syntax and rules for hash tables and arrays, it is superior IMO. Bash also makes it simple to write your own more complex utilities in any language and call them from bash. Linux makes it simples to implement new kernel tech and Bash makes it simple to interact with the kernel.

Windows... You have to wait until someone in WDG decides some existing idea is good and then also wait for them to get support to implement the idea, then wait to see how that whole thing goes. So yeah Powershell is great for interacting with Windows. But Windows thinks you're a dumb shit so you can only do what Windows already lets you do.

I was extremely well versed in PowerShell early into its development.

It was indispensable when it came out. VBscript and Cygwin were the only options; both of them sucked. The ability to use native .NET objects in scripts was insane; I definitely abused that!

Nowadays, Git Bash and WSL2 replace a lot of the value that PowerShell provided.

It's purpose was to drive Microsoft away from the GUI, and, at that, it more than succeeded.

A shell is not a thing by itself. It's only as good as it integrates with the rest of the system.

So, the experience of PowerShell on Windows is unbeatable... on Windows. It's easily beaten on any other system.

I use *sh at home, and Powershell at work. Powershell has a lot of really cool and genuinely useful concepts. There's the obvious like piping objects instead of raw bytes, but there's also things like first class parameter validation, automatic array flattening, multi-threading, full .NET API interoperation, and more.

Unfortunately, it also has some real sore points for use as an interactive shell, and even some outright broken behavior. Some that I lament most frequently:

- Powershell has native tab-completion support, but the word-of-god dictated "Verb-Noun" command naming scheme means almost every command shares a common prefix with a slew of entirely unrelated commands, making tab-completion painful.

- Power shell has sparse built-in support for funtional style list manipulation or data structures beyond lists. There's a "map" command (%), but no reduce, zip, tail, etc. These feel like they'd fit in perfectly to the design of the language, but they're just missing. Also, pipes are lazy, but lists are always strict, which can cause friction.

- most operators are strangely named, such as "-eq" for ==, which hurts legibility.

- There're lots of unvoidable brackets, meaning you often need to jump around the line while writing out a command incrementally. For example, to get filenames, you either need to write `(Get-Item).Name` or `Get-Item | %{$_.Name}`. It'd be nice to have something like `Get-Item |. Name` as an option.

- It's impossible to pipe raw bytes between commands/executables; raw bytes are interpreted as strings. This means, for example, it's impossible to pipe a file to `patch`, because the contents will be decoded into strings, piped as strings, then re-encoded with line endings changed, which `patch` rightfully refuses to apply.

- Running Powershell scripts is disabled by default (WTF?)

By comparison, *sh's heavily favor interactive use. For example, you can put io redirections at the start or end of a line, tab-completion is often very good, it's often possible to avoid brackets (ex. with xargs), background/foreground job management, etc.

Overall, I often find Powershell more pleasant to use than *sh, especially when writing scripts. But for interactive use, I often wish Powershell was a little smarter and a little dumber at the same time.

I have used it for nontrivial stuff. The idea of streaming objects has a lot of potential, but I think the "enterprise crowd" had a really good crack at it - especially the syntax. Not having to resort to grep, jq, cut, and the host of tools required to (often fragile) parse out whatever specific format the tool you are calling is giving you.

Put another way, you could feel free to parse the output of ls in powershell - because it's already been done for you.

They should have called it PowerScript, because it's a perfectly decent scripting language and an absolutely horrendous interactive shell language.

I'm not an expert, but I use UNIX/Linux at work and only Windows at home (I have a dormant Arch Linux install that I haven't touched in a couple of years), and hopefully you think I'm qualified to respond.

I feel Bash is a 'glue language' in every sense of the word, and is optimised for interactive command-line usage because of its terseness. Being remotely productive on Bash presupposes the presence of an entire UNIX subsystem and coreutils (which is why Git Bash for Windows drags along an entire `/usr/bin` directory with all the coreutils in it). You can't really do much in it without the coreutils.

I find that Bash parseability and ease-of-understanding rapidly deteriorates in scripts of increasing complexity. After about a hundred lines of code I reach for other languages. I personally dislike its unintuitive syntax, global scope, and weak, dynamic typing. The terseness that lends itself to powerful interactive command-line usage means script readability plummets. Of course, that might just mean I'm inexperienced with respect to Bash and need more practice, but I find this a fairly common opinion amongst colleagues and friends. In my opinion, Bash scripts are write-only. It is a wonder that something like neofetch[1]—which is an eleven thousand-line monstrosity—lasted this long (if I recall correctly the author stopped maintaining it because—amongst other reasons—it had just become too complicated).

I find that people understand PowerShell better when they realise it is closer to Python than any interactive UNIX shell. By default, PowerShell is more verbose than Bash; for instance, the PowerShell equivalent of the two-character all-lowercase `ls` is the thirteen-character mixed-case `Get-ChildItem`. Arguments are by default also long and verbose, with things like `-FollowSymlink`. PowerShell is generally meant to be used in executable scripts, just like how most Python isn't executed in the interactive REPL, but by running scripts. Consequently, I find this verbosity lends itself to improved readability in scripts, at the expense of some interactive productivity. That being said, common PowerShell commands have UNIX-like aliases[2] (at one point the cause of much gnashing of teeth here and at /r/programming because PowerShell aliased `curl` to `Invoke-WebRequest`[3]) to improve interactive productivity. PowerShell also supports truncating parameter names; for instance, `Get-ChildItem -Recurse -Force` can be abbreviated to `gci -r -fo`, which looks remarkably like a UNIX command now. Even so, the PowerShell ISE and the PowerShell VS Code extension both suggest that programmers use the full unabbreviated commands and arguments in `.ps1` scripts[4].

PowerShell is also dynamically typed, though semi-static typing can be opted in to[5]. That said there is some controversy about arrays decaying to scalars when they have size 1 [6]. It is easy to set up both function and script parameters with as much power and expressiveness as `argparse` for Python, but natively without involving any additional modules[7].

The real power (pun not entirely unintended) of PowerShell comes when you realise it is just another front-end to the incredibly massive .NET ecosystem. Where Bash requires a full UNIX subsystem and a collection of hundreds of additional binaries to be productive, PowerShell employs pre-compiled cmdlets[8] written in .NET (usually C#) to augment the basic language. Most 'commands' in PowerShell are cmdlets, including `Get-ChildItem`. You can write your own 'back-end' in C#, F#, C++/CLI, VB, or any other .NET language, compile it, and expose a PowerShell interface for end-users. Pretty much like how people write fast but complex code in C/C++, compile it, and expose a Python interface (Numpy, Pandas, PyTorch, etc come to mind). You can additionally directly call .NET methods in PowerShell[9].

This also means that everything in PowerShell is an object. The output of `Get-ChildItem` is a list, not a string. Likewise for `Get-Process` (UNIX equivalent: `top`). These lists may be formatted[10], filtered, parsed, modified, dumped to JSON, and otherwise manipulated like any other list data type in most other languages.

PowerShell also lends itself to the side-effect-free/functional/monadic map-reduce paradigm very well, with pipelines[11].

In my opinion, it is not merely 'better than CMD'; it blows CMD out of the water. A little less so for Bash, because Bash is augmented by the UNIX core-utils. As should now be evident, in my view the real competition to PowerShell is Python.

[1]: https://github.com/dylanaraps/neofetch/blob/master/neofetch

[2]: https://learn.microsoft.com/en-us/powershell/scripting/learn...

[3]: https://news.ycombinator.com/item?id=12319670

[4]: https://learn.microsoft.com/en-gb/powershell/utility-modules...

[5]: https://stackoverflow.com/a/115815/1654223

[5]: https://superuser.com/a/414666/1132163

[7]: https://learn.microsoft.com/en-gb/powershell/module/microsof...

[8]: https://learn.microsoft.com/en-gb/powershell/scripting/power...

[9]: https://devblogs.microsoft.com/scripting/learn-how-to-use-ne...

[10]: https://learn.microsoft.com/en-us/powershell/module/microsof...

[11]: https://learn.microsoft.com/en-gb/powershell/module/microsof...

I just find everything about powershell tricky. Something just doesn't click for me. It's always a struggle.

Exactly my experience.

It took me about a decade to be comfortable with it :)

There's looseness around arrays because the API for emitting outputs from commandlets is also loose around arrays. There is just one function `WriteObject` that writes the value you give it as the output of your commandlet. If you call it once, then that's your output. If you call it multiple times, then the shell has no choice but to then make your output an array of all the values you write.

So if one invocation of your commandlet only calls `WriteObject` once and the other calls it twice, the shell in the first case doesn't have the knowledge that it should've wrapped that one output in an array too. And it can't always wrap commandlet outputs in arrays because that would be disruptive for commandlets that semantically only have one result and so always write only one output (like `Get-Date`).

And for whatever reason, they didn't want to complicate the API to let commandlets themselves be able to express whether they'll semantically write only one output or multiple, regardless of how many times they actually call `WriteObject`. Such an API can't be a static attribute on the commandlet because commandlets can have wildly varying output based on their parameters. It can't be an overload of `WriteObject` like `(Object, bool iMightWriteMoreValues)` because it has to work for empty arrays. So it would have to be a separate `IWillWriteMultipleValues()` function probably.

Edit: Also explained here: https://news.ycombinator.com/item?id=40874873

This is so annoying. I’ve started using the preceding comma hack to force an array.

  $Ary = @(, "value")

One cool feature for creating arrays (especially larger arrays, for performance improvements over using +=) is to assign your array to a for loop. It wasn’t an obvious method of populating an array to me, but is definitely handy.

  $Ary = foreach ($Obj in $Objs) { @{ foo = $Obj.foo } }

> Does anyone know WHY they did this?

That sounds like a classic case of software trying to be "too clever," nine times out of ten of which will invariably backfire.

As a software developer, one should always resist the temptation to make their software "too clever."

It feels like a half baked feature. The other (missing) half would be automatically converting a single value into an array of length 1 when the receiving end expects an array.

Niche languages seem to end up with insane quirks like this every time. It's like the tragedy of sisyphus that things like lua exist, but instead of using it directly or modifying it slightly, then having a dead simple, tight, small, elegant, consistent language, people reinvent the wheel and never make it round.

This is reminiscent of MATLAB where there is no distinction between a scalar and a 1x1 matrix.

Python's default REPL is godawful. It's also just not designed for the kind of console work that PowerShell excels at because you have to do things like managing file handles rather than just getting the file content and piping it to another command.

PowerShell is great because it's a swiss army knife that has a very nice REPL with autocomplete, no weird whitespace behavior, readline[0], and can do anything .NET can do if you need to do anything more complicated.

Plus it's object oriented so you can focus on doing the tasks you actually need to do rather than trying to figure out how to use archaic utilities to parse the text based output of other archaic utilities.

0: https://learn.microsoft.com/en-us/powershell/module/psreadli...

>> I often wonder why Microsoft didn't base it on Python, Node, or something else. I can't remember when PS was first released so I'm not sure what would have been ideal at the time.

They originally based it loosely on Perl and the Korn shell. In the first edition of "Powershell in Action" by Bruce Payette there is a sidenote that states:

'PowerShell uses the "at" symbol ("@") in a few places, has $_ as a default variable, and uses "&" as the function call operator. These elements lead people to say that PowerShell looks like Perl. In fact, at one point, we were using Perl as a root language, and these elements stem from the period. Later on, the syntax was changed to align more with C#, but we kept these elements because they worked well. In Perl terminology, they contributed significantly to the "whipupitude quotient" of the language.'

It also states:

'The core PowerShell language is based on the POSIX 1003.2 grammar for the Korn shell. Originally, Perl idioms were appropriated for some of the more advanced concepts such as hash tables. However, as the project progressed, it became clear that aligning PowerShell syntax with C# was more appropriate.'

Powershell predates Node by 3 years.

They probably didn't base it on anything else since they control .NET.

I doubt it's slow because of .NET, it's slow because of either its design or because of under-investment in performance.

However... which version of PS are you using? As far as I remember the newer versions are quite fast.

> Powershell was truly a product of Microsoft monopolistic confidence. To create a language that allows absolutely no syntactical carryover from any other langauge is wild. You couldn't guess or assume any command, parameter or flag. Even with Microsoft's ambition they should have realized that legions of admins and programmers had to learn and maintain scripts in both Powershell and bash for a few decades at least.

Um... no. Powershell was inspired by shell, Perl and a bunch of other languages, you see it in its design. The other part was just a desire for consistency, since *NIX knowledge is just brute forced. Yeah, -v is generally verbose and -h is generally help, but in practice you can't rely on anything.

> an easily composable and programmatic way to configure anything in Windows

There's an entire industry of consultants and software vendors that don't want that outcome.

> connecting with Remote Desktop and clicking around with a mouse

This generates a lot of hours.

> is horrendously difficult and annoying.

It is ironic that this is probably the main reason that alternative operating systems even exist.

Even crazier is that the “click around” mentality made its way into azure too - back in the day (a decade ago, to be fair) I had a serious suggestion from a technical support agent there that the best way to automate some setting was with Selenium…

You have long experience in computing and especially in the innovative currents that shifted the paradigm. Cheers!

Your first paragraph creates expectation that the second paragraph disappoints, though. Would you explain why you think PowerShell is "nice"?

You posted just at the same time I asked. What's makes PowerShell better than bash?

While I've never used C#, I agree with you. I've moved more and more to compiled or interpreted languages versus CLI tools for maintenance and administration.

Nushell is based on it, and it is picking up steam.

https://www.nushell.sh/book/

So in the end, PowerShell doesn't need to catch on.

Huh I always found that to be the opposite. PowerShell is clearly designed to be a programming language not an interactive environment.

E.g. 'wget' exists but run it on its own and it doesn't save the file to disk or print it out, it prints an object as a table, which is about the most useless output possible. I know why this happens but that doesn't make it helpful.

Reading about the dysfunction inside Microsoft makes it clear why it ended up this way though!

Does the ISE still take forever to spin up?

That's what they should have done. Have a C# interpreter with maybe a few extensions for shell stuff. Powershell is not a good .NET language and not a good shell. It's just a weird beast.

A while ago I worked on a project to do pretty complex configuration on Windows machines. Everybody thought Powershell would be perfect for this. It turned into a complete nightmare with tons of oddities to work around. I myself barely can make sense of the code and I bet nobody else can.

Next time I would do either Python or write the whole thing in C#.

Good news. You can write C# that powershell will compile on the fly. It's so ingrained in Microsoft applications that even MSSQL Server will do it as well, if you want to go down that route.

That is because `curl` is an alias for the builtin Invoke-WebRequest and not the actual curl program.

https://learn.microsoft.com/en-us/powershell/module/microsof...

To be fair, a GUI can be used in a very clever and skillful manner. It's called keyboard shortcuts (and, to a lesser extent, mouse gestures). You can't deny an Excel world champion uses its GUI in a highly impressive way. And features like multiple carets editing (as popularized by Sublime Text) makes it possible to perform some remarkable editing feats. :)