Hacker News Reader

Comment by brushfoot | Hacker News Reader

The name of its corporate progenitor may leave a bad taste in some mouths, but I highly recommend PowerShell for this sort of thing. It's cross platform, MIT licensed, and comes with excellent JSON parsing and querying capabilities. Reading, parsing, and querying JSON to return all red cars:

  Get-Content cars.json | ConvertFrom-Json | ? { $_.color -eq 'red' }

The beauty of this is that the query syntax applies not just to JSON but to every type of collection, so you don't have to learn a specific syntax for JSON and another for another data type. You can use Get-Process on Linux to get running processes and filter them in the same way. The same for files, HTML tags, etc. I think nushell is doing something similar, though I haven't tried it yet.

I prefer this approach to another domain-specific language, as interesting as jq's and zq's are.

View on HN

PowerShell "sends basic telemetry data to Microsoft [...] about the host running PowerShell, and information about how PowerShell is used" [1].

And since it relies on .NET, that also requires its own separate opt-out for its telemetry. There might be other components, now or in the future, that also send data to Microsoft by default and would have to be separately discovered and disabled.

[1] https://docs.microsoft.com/en-us/powershell/module/microsoft...

> And since it relies on .NET, that also requires its own separate opt-out for its telemetry.

Building a program with .NET does NOT cause that program to send telemetry to Microsoft.

You're thinking of the .NET SDK itself. Using PowerShell does not trigger any use of the .NET SDK.

Disclaimer: I work for Microsoft.

Ah, yes, my mistake. Although PowerShell sends its own telemetry, the additional telemetry from the .NET platform is only sent when you use the dotnet command [1] and, as a special case, not when you very carefully invoke it only "in the following format: dotnet [path-to-app].dll" and never e.g. "dotnet help".

However, presumably PowerShell requires at least the .NET Runtime if not the .NET SDK, doesn't it? The docs [2] suggest running "dotnet --list-runtimes" to "see which versions of the .NET runtime are currently installed", so it sounds like the Runtime also includes the dotnet command. Does running the recommended "dotnet --list-runtimes" command send telemetry, like most of the commands? Or are you saying that the Runtime, unlike the SDK, doesn't include telemetry at all?

[1] https://docs.microsoft.com/en-us/dotnet/core/tools/telemetry

[2] https://docs.microsoft.com/en-us/dotnet/core/install/how-to-...

> However, presumably PowerShell requires at least the .NET Runtime if not the .NET SDK, doesn't it?

Nope, these days .NET programs (like PowerShell) bundle the runtime. But even if they did a lighter distribution that depended on the runtime already being installed, there would be no .NET telemetry sent.

> Does running the recommended "dotnet --list-runtimes" command send telemetry, like most of the commands?

This is still an SDK command. I don't personally know if this one sends any telemetry.

> Or are you saying that the Runtime, unlike the SDK, doesn't include telemetry at all?

The runtime does not send telemetry.

So the "dotnet" command is only in the SDK, not in the separately downloadable Runtime? Does the Runtime have some other command to launch an executable?

Edit: Actually, the ".NET Runtime 6.0.4" [1] (not the SDK) definitely has a "dotnet" command included. Presumably with the telemetry?

[1] https://dotnet.microsoft.com/en-us/download/dotnet/6.0

When I say "the runtime", I'm referring to everything that would be bundled into a published .NET program. The base class libraries, the bootstrapper, etc. There is no telemetry here.

Yes, if you download a .NET Runtime distribution, it will include the `dotnet` command from the SDK so that basic commands like `dotnet --list-runtimes` and `dotnet --list-sdks` are available. These commands may send telemetry. But as you probably saw on https://docs.microsoft.com/en-us/dotnet/core/tools/telemetry , using `dotnet path/to/program.dll` to run an unbundled .NET program will never send telemetry.

To me a telemetry opt-out is a small price to pay for what PowerShell brings to the table, but to each their own.

> There might be other components, now or in the future, that also send data to Microsoft

Of course. Do your due diligence on whatever you install. No tool should be exempt from that.

> Do your due diligence on whatever you install. No tool should be exempt from that.

That's a ridiculous take. 99% of users don't understand what all that technobabble in a typical EULA means, they will just go for the option they are nudged to (which is why first the courts and now enforcement agencies are stepping up their game against that practice [1]).

The way that the GDPR expects stuff to be handled is by getting explicit user consent, the consent must be a reasonably free choice (i.e. deals like "give me your personal data and the app is free, otherwise pay" are banned), and there must not be any exchange of GDPR-protected data without that consent unless technically required to perform the service the user demands. Clearly, a telemetry opt-out is completely against the spirit of the GDPR and I seriously hope for Microsoft to get flattened by the courts for the bullshit they have been pulling for way too long now.

What I would actually expect of Microsoft is to follow the Apple way: have one single central place, ideally at setup and later in the System Preferences, where tracking, analytics and other optional crap can be disabled system-wide.

[1] https://www.hiddemann.de/allgemein/lg-rostock-bejaht-unterla...

> That's a ridiculous take

Then it befits a ridiculous state of affairs. It would be great to have the standards you suggest, and it's a shame that we don't. But that doesn't change the fact that we don't, and because we don't, we need to do due diligence on the tools we install.

The GDPR applies to personal data. PowerShell telemetry isn't personal data, so it's not covered by the GDPR. What is reported is documented here:

https://docs.microsoft.com/en-us/powershell/module/microsoft...

and is "anonymized information about the host running PowerShell, and information about how PowerShell is used". It sucks that it has telemetry, but anonymised information about whether a computer ran 10 .exe or 10 cmdlets pales into insignificance against Windows and Edge and OneDrive slurping up names, addresses, files, moving logins to Microsoft accounts, sending browser history to Microsoft, checking downloads with Microsoft, keeping a history of all programs run in Windows for timeline and trying to send that to Microsoft to sync it between devices, moving OneNote to the cloud, having the start menu search be a Bing web search, defaulting to Cortana being a cloud based voice search, sending pen and ink data to Microsoft, and etc. etc.

Even the fact that a particular piece of software is used by a specific IP address is enough PII that it's covered under GDPR by most viewpoints. The fact that Microsoft is collecting even more data doesn't excuse telemetry in PowerShell at all.

I would simply wish for no telemetry to happen at all without user consent. If Microsoft wants information about how people use their software or how stable it is and not enough people opt in, they should fucking pay people money for market research and QA.

> "Even the fact that a particular piece of software is used by a specific IP address is enough PII that it's covered under GDPR by most viewpoints."

I draw your attention to the link I posted, and the purple background call out box with the exclamation mark icon and the heading "Note" which says: "Application Insights uses the hosts IP address to determine the geographic location. The IP address is never included in the telemetry data or stored in the database."

> "I would simply wish for no telemetry to happen at all without user consent."

I would, too. So did someone on Github: https://github.com/PowerShell/PowerShell/issues/15722 "Change telemetry from opt-out to opt-in" where Microsoft said "we felt that by making telemetry opt-in it would bias and limit our telemetry in a way that would make less useful to our users."

> The IP address is never included in the telemetry data or stored in the database.

It nevertheless is transferred to the origin server (otherwise, how would TCP work), and that is enough for it to be considered under GDPR!

> we felt that by making telemetry opt-in it would bias and limit our telemetry in a way that would make less useful to our users

Well, that still doesn't override the GDPR. The GDPR is law, precisely in the spirit to prevent un-consented tracking!

> What I would actually expect of Microsoft is to follow the Apple way: have one single central place, ideally at setup and later in the System Preferences, where tracking, analytics and other optional crap can be disabled system-wide.

This is still GDPR non-compliant, you should have a central place to _opt-in_ tracking, analytics and other optional crap if you so desire.

So what? You can opt-in to tracking in the macOS System Preferences, pane "security and data protection", tab "Privacy" at any time you wish should you not have done so during the macOS onboarding process.

In Debian, you can opt-in at setup time or any later time with a simple "dpkg-reconfigure popularity-contest" (even though that one isn't fully GDPR-compliant as you can't easily read what exactly is being done from the same screen).

> So what? You can opt-in to tracking in the macOS System Preferences, pane "security and data protection", tab "Privacy" at any time you wish should you not have done so during the macOS onboarding process.

You cannot opt-in. You can go to `System Preferences > Security & Privacy > Analytics & Improvements` and opt-out, but the default is not opt-in.

> The beauty of this is that the query syntax applies not just to JSON but to every type of collection,

This is the best part of pwsh. Everything is standardized, you're not guessing at the idioms of each command, and you're working with objects instead of parsing strings!

My second favorite part is having access to the entire C# standard library.

PowerShell is "Python interactive done right". It's too bad it has a bad rap in open source community and it might never get the traction it really deserves. Sure it has it's downsides, which tech doesn't, but PowerShell has solved so many issues and annoyances with the shells that we've been used to, that it still comes out as the winner.

I've been using it since day one from 2006, every single day. It has come a long way and the current PS7 is the best shell experience there is. Hands down no contest.

Snover's passionate early presentation about the PS pipeline is a pretty cool tech video. https://www.youtube.com/watch?v=325kY2Umgw8

> PowerShell is "Python interactive done right".

Actually PowerShell is "Perl interactive done right" if you read what the designers say about their influences - the automatic variable $_ is straight from Perl and the array creation syntax @(a, b, c) is also a Perl-ism from @arr = (a, b, c). Which is funny as I dislike Perl intensely but really like PowerShell :)

To be fair there's not much Perl in PS, it's as much influenced by KSH, Awk, cmd.exe and VBScript as Perl. Thankfully "influenced by" isn't "a melange of", because a combination of all of those sounds like an abomination lol, and PS is wonderful in being about as consistent and simple as a proper shell can get.

I want to learn powershell, but I have an internal ick bias because I've been using bash for so many years. The tab behavior is the exact opposite of what I expect and it short circuits my brain every single time I press it. Having structured data in the pipes seems very useful and powerful though so I should probably just bite the bullet.

Tab behavior is configurable. I have mine set to menu expansion.

    Set-PSReadLineKeyHandler -Key Tab -Function MenuComplete

Thanks for that! I've recently been learning PowerShell. After 30 years of bash it is interesting.

.. or you can try Next Generation Shell (author here):

fetch("cars.json").filter({"color": "red"})

# or

echo(fetch("cars.json").filter({"color": "red"}))

Powershell's object pipes are more inspectable than any of Bourne's text-based decendants. But the tool itself occupies a niche between "write shell, dealing with esoteria of grep/sed/awk/jq/etc" and "write python getting constructs than handle complexity better than pipes".

Looking at the popularity of VSCode, I don't think Microsoft hatred blocks its adoption.

> Looking at the popularity of VSCode, I don't think Microsoft hatred blocks its adoption.

In-apt comparison. The people using VS Code are more likely to be migrating from proprietary tools like PyCharm, Sublime Text, etc or bloated offerings like NetBeans or roughly equivalent offerings like Atom.

The people that would use PowerShell would be migrating from the likes of Zsh, Bash, Fish, and other “hard core free” software.

I conceptually like pwsh, but even as your example shows, I don't have the RSI budget left to spend on typing that extremely verbose expression every day

jq and its unix-y friends allow me to trade off expressiveness against having to memorize arcane invocations

I hear that, I use and like *nix too. PowerShell aliases help a lot. It comes with some predefined, like `gc` for `Get-Content`. The above example could be rewritten:

  gc cars.json | ConvertFrom-Json | ? color -eq 'red'

`ConvertFrom-Json` doesn't have a default alias, but you can define one in your PowerShell profile. I do that for commands I find myself using frequently. Say we pick convjson:

  gc cars.json | convjson | ? color -eq 'red'

That's more like what my typical pipelines look like.

The nice thing about aliases is you can always switch back to the verbose names when clarity is more important than brevity, like in long-term scripts.

Edit: Seems I've been using too many braces and dollar signs all these years. Thanks to majkinetor for the tip.

You don't need $_ for immediate properties which looks much cleaner:

    gc cars.json | convjson | ? color -eq 'red'

TIL! Thanks!

jq can not only process JSON input but also emit JSON output. So on that note, has ConvertTo-Json stopped mangling your JSON yet? https://news.ycombinator.com/item?id=25500632

Agreed— PowerShell is really nice for this, as are some of the other shells it has inspired.

Reply on news.ycombinator.com