Don’t most websites prohibit someone from scraping and harvesting the data on them? Most recently, I can think of Yelp, Amazon, and GitHub prohibiting this, as well as the Aaron Swartz case.
While finding that link for you, I learned that since that date, the Supreme Court vacated the decision back to the lower courts in light of a new decision of theirs.[1]
Now I don't think it's clear one way or the other just yet. Any lawyers here with an opinion on how this is going to go? I haven't found any analysis.
[0] https://en.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn
[1] https://www.reuters.com/technology/us-supreme-court-revives-...
It's probably very bad strategy to allow the privacy leak, and then hope that the FTC agrees with your decision later. No one will be sanctioned for adhering too strictly to consent decrees, but you could be sanctioned for being too loose. So the choice is obvious in that light.
As a website owner, beyond that, there is no need to tell someone they cannot access your site if you simply block them from accessing your servers instead using a multitude of techniques.
Where does caching come into play at all here? You cannot cache content to begin with if the server is blocking access in the first place. And if you have already cached it in the act of violating said website’s terms of service, then you are still not in compliance.
According to many rulings the last few years, continually and systematically accessing a third party’s data under the clear expectation that you are aware of (as well as agreed to) their terms is definitely a meeting of minds.
Not to mention the fact that terms and conditions are not contracts. I don't think they carry the same weight, although someone please correct me on this if I am incorrect.
Judge: “what defendant? There is no one here.”
Plaintiff: “Oh he was anonymous, so I am not sure who it was…”
Judge: “Hmm interesting, so you seem to think an implicit contract exists that you want to enforce with no documentation at all with a party you can’t name, because you are not sure who it is?”
Plaintiff: “Exactly.”
Judge: “Feel free to come back when you aren’t going to waste the court’s time”
Also…you missed the word “public”, again.
https://www.upcounsel.com/are-website-terms-and-conditions-l...
This is heavily supported by Case Law.
If you don’t want me to have a copy of it for any reason, don’t let me have it at all.
If you publish content on a website that willingly provides data to anonymous users of your site, even with a TOC on the site, the TOC is not enforceable if you cannot prove that the user explicitly agreed to the TOC. If you don’t know who the user is, you can’t prove that they agreed to your TOC.
Having a TOC is basically legal theater if you allow anonymous users. The implied threat is basically “IF we find out who you are” and you use the site in a way that is contrary to our published TOC, we will take action against you.
Your only recourse in that case is to pursue sites that are republishing your copyrighted content…because only at that point can you actually identify the party that may be misusing your site and it’s content.
This weired mindset where corporations make law shouws up again.
The websites have no power to prohibit anything. If they make bytes avaliable, we may do with them as we please so long as its legal.
If they'd try to prevent you from scraping third-party sites it would be making laws; setting up ground rules with their ToS and enforcing them is absolutely fine.
Actually no.
If a platform provides a generally available service they are (in many countries, idk. about the US) not allowed to arbitrary exclude some people they don't like without a legal valid reason.
And braking legally not valid/binding terms in a ToS is not a legal valid reason. Just because you write something in your ToS doesn't mean it has any legal relevant meaning, there are limits to what you can put in ToS. And limiting (properly done, privacy respecting) research is often not valid. (Through depends a lot on the country.)
Nobody is legally forcing websites to allow access to everyone, and accordingly, nobody is altering the law by blocking access to people (crawlers, hackers, spammers, malcontents, or anybody really) that they feel are not welcome. So exercising one’s existing rights isn’t an act of making or altering laws.
I suggest reading up on what robots.txt is to further understand this.
Either Crawling does not belong on that list, or google exects should be in jail.
Given that crawling is not malicious, what we are discussing now is 'someone is crawling my website in a way I dont like' which is a different gripe.
It mighthave some merit, but robots txt is not legally binding.