Hacker News Reader

Comment by iav | Hacker News Reader

Scraping is legal, but the legal framework requires scraping to abide by three rules:

1. The page is not behind a login. If you have to create an account to access the content, then you have to agree to abide by the terms of service which may ban scraping.

2. What you scrape is still protected by copyright. If you scrape a clip from Star Wars from a web page, you still can't redistribute it without a license.

3. Your activity may not impede access to the site in anyway i.e. no DDOSing. If there is a robots.txt file, you are supposed to abide by it but the courts do not literally say that. Must rate meter your requests, scrape at night, so on. Gray area if the robots.txt bans every page and you end up having to ignore it.

View on HN

The Supreme Court decision referenced in this decision involved a case where the information accessed was protected by a login and accessing the information violated the terms of service. The court ruled that is not a violation of the CFAA. I fail to see that it is therefore necessarily required that the page is not behind a login for scraping to be legal. There may be other issues regarding the type of information or repeatedly creating accounts to skirt account terminations which affect whether it is legal, but simply violating terms of service for accessing a page behind a login doesn't appear to make it illegal by itself.

This case was about users public profiles. LinkedIn makes users profiles accessible to the Googlebot and other public visitors without the need to login. There was an additional wrinkle in that HiQ actually had a commercial agreement with LinkedIn for some time that explicitly allowed them to scrape. They were arguing that canceling this agreement was anticompetitive and would destroy their entire business. Because of that pre-existing agreement, they likely didn't dwell on the distinction of public or private as they already set the precedent of signing up to an agreement that authorized them in the past, so they can't now say they never needed one in the first place.

I'm not sure why you are referring back to the specifics of the case since your comment was about what you put forward as a general framework required for scraping to be legal and that was what my comment was regarding.

Since when are terms of service legally binding?

They never weren’t. After all, they’re contracts that you agree to when you check the box. Some clauses may be unenforceable, but voiding a clause does not necessarily void the whole contract.

Always have been. See: https://en.wikipedia.org/wiki/Terms_of_service

Legally binding =/= enforceable with penalties or illegal to break (excepting revocation of service). Breaking TOS is not a crime [0]

[0]: https://www.eff.org/deeplinks/2018/01/ninth-circuit-doubles-...

Reply on news.ycombinator.com