OpSci: URIs, indexes and RSS for 72 closed-source publishers and scraper code. | Hacker News Reader

OpSci: URIs, indexes and RSS for 72 closed-source publishers and scraper code.

github.com

26 pointsdelinquentme12 comments

I don't understand what this is... It doesn't scrape the full published articles/material on hidden/private URLs (unless you have access, paid for a subscription)? So what would one use this for?

Its basically a 1-shot index of as much information as one can LEGALLY currently index.

Do you mean it just indexes the short "abstract" that's provided of each paper? Or does this even go less, and just builds an index of the URLs only.

It gets to the journals ... Perhaps you're right ... the abstracts would be plainly available as well as valuable.

It should be noted that this only indexes the journals and does not scrape the research PDFs themselves.

That was not immediately clear, wish that had been in the title. I was curious how you were scraping private PDFs and open sourcing such a thing.

So one is clearly somewhat legal grey area, while what is here is a little more legal. Perhaps someone will take this library and put it to good use to topple that tower =]

IANAL, but I would think the wholesale scraping and republishing of entire copyrighted research papers would be a pretty dark shade of grey, no?

Its your tax dollars that someone else has locked up. Why don't you tell me.

"Ethical" is not necessarily the same thing as "legal."

And I think it's rare that there's a paper that is entirely funded by tax dollars. I agree that all research receiving government grants should be open access, but it's not quite as clear cut as you seem to imply.

A very impressive achievement. Where can I find an example of how to use it?

OH! Good call! Just uploaded a few edits to the Readme. Note: You'll need bundler installed

https://github.com/delinquentme/OpSci

Reply on news.ycombinator.com