> If folks want to help, it would be great to have a "youtube-dl for open access papers".
I think Unpaywall is already trying to do this? Or at least, for every DOI they index, they try to include a link to the direct article if known.
I think Unpaywall is already trying to do this? Or at least, for every DOI they index, they try to include a link to the direct article if known.
Having an URL to something that is supposed to serve a PDF doesn't mean that you actually get it. Publishers like Elsevier or Wiley require a cookie/JavaScript dance and/or place heavy rate limits; whatever PDF hosted by them is never really accessible, even if it's ostensibly open access.
The "youtube-dl for papers" would do what youtube-dl does, i.e. take the URL and execute whatever JavaScript or other trickery the target page/website requires before it serves the content.