Hacker News Reader

Comment by artembugara | Hacker News Reader

I totally agree with the first part where you say it's fine to scrape. It is just an automation.

But you don't mention what you do with this data after. 1. You store it somewhere (not in your brain) 2. You extract value out of it (directly or indirectly)

That's why I understand why it is a problem for those who publish this data.

One thing I always try to say to everyone who argues about web scraping: web scraping is not a problem, problem is what you do with the information you scraped.

Disclaimer: We crawl the web for news (https://newscatcherapi.com/)

View on HN

> That's why I understand why it is a problem for those who publish this data.

I still don't understand why it's a problem for those who publish the data.

If the "data" is facts (e.g. lists of values associated with objects, like the colors available on automobiles), this data is not protected by copyright, there is no remedy if it's republished, and if a business relies on limiting access to this data, that business needs a new business plan. Charge for access to cover the costs of obtaining and organizing the data, but understand that clients are legally allowed to make copies and use them however they see fit, even if that impacts the supplier's ability to charge for access.

If the "data" is prose, it's under copyright and republishing without a license has remedies under law. Maintaining copies of articles for the purpose of processing them to obtain other data (e.g. how many nouns? what adjectives are near those nouns? etc...) isn't protected.

Databases are actually protected under some copyright laws in some countries.

For copying the database wholesale, yes. But the individual records, formatted on a web page, and scraped out by a bot is not copying the database files or its schema wholesale. They're still individual facts.

So if somebody compiles the data, charges an N amount for access, it's fine that some entity buys a one time access and then releases it for free?

This is usually explicitly prohibited.

If it's not prohibited for some reason, e.g. because the term of the prohibition, such as copyright, has expired, why not.

Are news just facts? Or do they also incorporate the bias of the author?

Sure, you could argue that news are facts, and that they should be free from copyright. But that would only work if your news is just bullet points of the factual stuff. But if any element of bias is present, it become a distinct property of the author, hence it should be copyrightable.

Of course, with today's tech, it's easier to remove the bias element from the factual data.

I'm agree, 100%.

Reply on news.ycombinator.com