Extracting web page content using Readability.js and shot-scraper

Here's the terminal one-liner:

    shot-scraper javascript https://simonwillison.net/2022/Mar/24/datasette-061/ "
    async () => {
      const readability = await import('https://cdn.skypack.dev/@mozilla/readability');
      return (new readability.Readability(document)).parse();
    }"

To first install shot-scraper, run:

    pip install shot-scraper
    shot-scraper install

This looks cool.

In the past, I've had good success with using GoOse. There are implementations in Scala as well as Golang.

https://github.com/advancedlogic/GoOse

FWIW I offer a (paid) webpage content extraction API:

https://www.goldenhillsoftware.com/webpage-text-api/

Thank you @simonw thanks to you I get to learn Python and pipelines :p