Hacker News Reader

Comment by vips7L | Hacker News Reader

Invoke-WebRequest most certainly runs on linux.

This comment made me go and try out Invoke-WebRequest because I'd forgotten what it was like.

Being able to do simple web scraping like this in a "shell" is pretty great:

    $hn = Invoke-WebRequest https://news.ycombinator.com/
    $hn.AllElements | Where {$_.TagName -eq "a" } | Where { $_.Class -eq "StoryLink" } | select "innerHtml"

IWR and ConvertFrom-Json are what got me hooked on powershell. No more learning complex switches for curl or crazy JQ syntax. It's all built in and makes sense.

FWIW, I already have a command line tool that can do that installed in regular Unix: https://github.com/ericchiang/pup The docs even use the same example of scraping HN.

How does Invoke-WebRequest decide if it’s looking at HTML or something else?

Well, you use Invoke-RestMethod for JSON/XML stuff.

Or decide yourself for Invoke-WebRequest to pipe it with whatever the data: ConvertFrom-{JSON,XML,CSV}

With Invoke-WebRequest you get back headers, raw response, etc - and you get to decide whats next. With the RestMethod - just the content as an object.

There is an param to declare response type parsing and if you want to use different parsing. Older versions of powershell use IE and webrequest will randomly barf. It really sucked. You can also use a rest request method that uses json.net to deserialize I believe .

I just attempted to prove you wrong, but it does look like it's fixed.

So is creating secure strings.

Cool!

Creating secure strings isn't exactly fixed; the type exists but they aren't encrypted in memory on Linux because it doesn't have the DPAPI (Data Protection API) from Windows:

https://github.com/PowerShell/PowerShell/pull/9199

Reply on news.ycombinator.com