Comment by deafpolygon | Hacker News Reader

Comment by deafpolygon | Hacker News Reader

Parent Full thread

That is just a case of people trying to use PowerShell as a bash shell combined with the unfortunate decision by whomever at Microsoft to alias PowerShell cmdlets to well known UNIX tools.

It doesn't "silently mangle" binary data - it assumes you're piping PS objects or text streams through. If it is a text stream, then it will pipe through as Unicode by default.

Invoke-WebRequest (curl) pipes out an object, with the content in (object).Content. Using | uses Unicode by default which "mangles" binary output. So it tries to convert the object into a Unicode text stream then into the file.

The correct way to save an image would be to simply set -OutFile (-o) in Invoke-WebRequest.

If you MUST pipe it through, then you need to access the object's .Content, then use Set-Content -AsByteStream (for PS7+) or use Set-Content -Encoding byte (PS5)

PS5> $(Invoke-WebRequest -Uri https://www.w3schools.com/html/w3schools.jpg -ContentType 'image/jpg').Content | Set-Content -Encoding byte test.jpg

PS7> $(Invoke-WebRequest -Uri https://www.w3schools.com/html/w3schools.jpg -ContentType 'image/jpg').Content | Set-Content -AsByteStream test.jpg

This is useful if you wanted to put the image inside a variable (rather than create a temporary file for the image) and pass it around until you figure out what you want to do with it.

>> That is just a case of people trying to use PowerShell as a bash shell combined with the unfortunate decision by whomever at Microsoft to alias PowerShell cmdlets to well known UNIX tools.

>> It doesn't "silently mangle" binary data - it assumes you're piping PS objects or text streams through. If it is a text stream, then it will pipe through as Unicode by default.

The problem is that PowerShell uses aliases that match UNIX command line utility names and the same pipe syntax, but exhibits subtlely different behavior without any indication of the difference.

If someone is not aware of the difference and they use PowerShell to process production data, their data could be corrupted without them realizing it.

> The problem is that PowerShell uses aliases that match UNIX command line utility names

Agreed.

> If someone is not aware of the difference and they use PowerShell to process production data, their data could be corrupted without them realizing it.

Someone coming from UNIX-land shouldn't be making PowerShell scripts day1. They really should be diving into documentation. Microsoft's documentation on PS is _excellent_ and here is one about pipelining: https://learn.microsoft.com/en-us/powershell/module/microsof...

Notably:

> To support pipelining, the receiving cmdlet must have a parameter that accepts pipeline input. Use the Get-Help command with the Full or Parameter options to determine which parameters of a cmdlet accept pipeline input.

then further down:

---

Using native commands in the pipeline

PowerShell allows you to include native external commands in the pipeline. However, it is important to note that PowerShell's pipeline is object-oriented and does not support raw byte data.

Piping or redirecting output from a native program that outputs raw byte data converts the output to .NET strings. This conversion can cause corruption of the raw data output.

As a workaround, call the native commands using cmd.exe /c or sh -c and use of the | and > operators provided by the native shell.

---

To work with external commands and pipelining, you can refer to this SO (top answer explains it very well): https://stackoverflow.com/questions/8097354/how-do-i-capture...

> Piping or redirecting output from a native program that outputs raw byte data converts the output to .NET strings. This conversion can cause corruption of the raw data output.

And this behavior is terrible and one of many "gotchas" that PowerShell has. Fortunately, there is a chance it will be "fixed": https://github.com/PowerShell/PowerShell/issues/1908#issueco...

Reply on news.ycombinator.com