Hacker News Reader

My view of course also says that I'm free to download the data at http://secret.somecompany.com:8081/secrets/data.bin Because they exposed it at a public http endpoint.

That's a poor example, because they most likely did not intend for that information to be publicly available, and just did a poor job securing it.

Real-world analogy: if you happen to find someone's house with an unlocked front door, that does not give you permission to go looking around inside. It doesn't matter that you don't steal anything, and it doesn't matter that nobody knows you were there. You're still not allowed in and by going in you're breaking the law.

If your example was for content that's retrieved as the default web page for a public DNS entry, or a page that's reachable by following links from the website's default page (including urls in meta, link, and script tags), then I think you're on solid standing as far as being allowed to retrieve the content. At that point, copyright laws and ToS govern what you're allowed to do with the content, with the safe assumption being that you're allowed to view it, but not republish it (except for fair-use excerpts.)

I agree - I suppose it matters mostly how I find it. If I could be shown to have been poking around (walking up to every door to find an unlocked one) then certainly I have some dubious intent.

The question is of course in what situations I should have known I stumbled across something they didn't intend for me to see.

Googles scraper certainly walks up to every door, and then publishes what it finds. If you accidentally publish something and then google index it, and someone reaches it via google, it's hard to say it wasn't public.

No, that's not how Google's scraper works. They only follow publicly-exposed links, and they check your robots.txt file to see if they should ignore any links they might have come across that you intended to be private.

If you accidentally put a link on your public website to content you didn't want public, and Google indexes it, then someone else fetches it, yeah, that's your fault for exposing it. It's also your responsibility for telling Google to remove it from their index. But if someone is poking around to see if urls are valid (eg: the /.ssh/id_rsa requests my website gets) that's breaking and entering, even though I'd be negligent for leaving my keys on the webserver like that.

I know - but let's assume that since I and google could reach the document it was inadvertently put on a public server. Since it was a mistake it won't be filtered in robots.txt either.

For example, in the weeks before an Apple event, people tend to visit presumed unannounced product pages (by guessing) etc.

Actually, I don't think they state that is the only way they find content. I've had plenty of sites with obscure domain names for development and they always seem to find their way on to Google some how.

I imagine in addition to public URLs they also use URLs entered into Chrome (which isn't that public) and since they host DNS they probably crawl domain names requested through them too.

Legally I'm not sure what they stand on to do that but they're big enough now so I guess they don't worry too much about that nowadays.

> if you happen to find someone's house with an unlocked front door, that does not give you permission to go looking around inside.

But in this case you are asking the server for permission to come in. Unlike a lock, a server has situational awareness to decide if it says yes or no. We even call it a server, à la servant. If the maid acting as an agent for the homeowner at this hypothetical house invited you in, against the wishes of the homeowner, are you still breaking the law?

Like I said, someone who does a poor job securing their content, but who are not exposing it directly via publicly-exposed links, are not granting public access.

Your example is more akin to the maid wandering off leaving the front door open. That's not an invite to come in, that's a poor choice of maid.

I respectfully disagree. We don't use terms like 'request' and 'server' because the mechanism is akin to a door wide open.

We can agree that a maid that lets the wrong people in is a poor choice of made, but the maid is the one responding to the request. A door/lock does not deal with requests at all.

In this scenario though the maid is not choosing to wonder off, the homeowner is telling the maid to leave the front door open.

To extend this analogy a little bit, lets say that the homeowner told the maid that they can "give the old tv, the toaster or any other item in the garage to anyone who asks." A stranger comes and asks the maid for the golf clubs in the garage that the homeowner forgot about.

In this case, the homeowner did not explicitly expose the golf clubs to the public, but gave permission to the maid to give them out. Did the homeowner not grant public access to the golf clubs?

Well, if you go to another country and you don't know the laws, you will still get arrested if you break the law.

Sure. That's the starting point in that case. The question is whether it can be seen as the starting assumption of someone making a http request that it's a "service" for which I'm assumed to find, read and follow the terms of?

I'm guessing that varies and is decided by legal precedent only once it becomes an issue.

Most countries apply intellectual property law to anything published on the internet. In other words, when it comes to something you pull down via http, you should assume that, if you attempt to republish it in any form, someone out there might decide that it's worth their time to use the court system to destroy you, so you should take the time to read the ToS.

Yes - Like I said I think it's perfectly find that the material I download from the internet has terms and laws associated with it. That is, what I do with what I download is subject to laws as soon as I e.g. try to publish it.

But my point is this: what service have I used, and what terms can I be assumed to have agreed to if I do this

    > wget $random_url > /dev/null

That's what web scraping is. What I do with the scraped material is something completely different.

In that case, you are bound to these terms of service: https://tools.ietf.org/html/rfc2616

IANAL, but it came up in a computing science/internet law course and in the Netherlands the current jurisprudence is that if you use (and continue to use) a service you are reasonably expected to read, understand, and adhere to the rules and conditions set by the service.

In this case it thus doesn't whether or not you even saw the contract. You are bound by the ToS because it can be expected of you to search for it during continued use of the service.

What happens if the TOS state that you are not allowed to use an ad-blocker while viewing their website? (Honest question)

Disclaimer: Pure conjecture.

I'd say the TOS have to apply to the "service" you're consuming and not the website itself. Otherwise, what happens if you use just a text based browser? Or any other means not foreseen by the TOS?

An ad-blocker would be just the same, it's something that your browser does.

Intent matters. Ad blockers are not the same as text based browsers.

There is another side of ToS in the Netherlands that I haven't mentioned: The ToS has to be reasonable. There is a blacklist which cannot appear in a ToS under any circumstance because it is unreasonable to ask from users. For example, it is illegal for a company to require users to agree that when they buy goods from their shop they cannot return it whatsoever. There is also a gray list: these are conditions that companies may ask users to agree to if and only if they can prove it is reasonable condition. Notice that this a reverse of the burden of proof: normally a user has to prove that some terms in the ToS are unreasonable which is hard. Now the company has to prove that their terms are reasonable, which is also hard but better for the user.

For example, it is illegal for companies to document the religious beliefs of a person BUT if this is a service for certain religious community (e.g. a forum for a church) then the term that they have to disclose their religious belief is reasonable (given that they don't share this information with anyone, of course).

As for the ads, I don't know. It might be unreasonable to have users agree to subject themselves to third-party ads in order to use the service. But again, I am not a laywer.

However if a ToS attempts to remove enshrined rights or includes terms that are particularly onerous then this can invalidate the section or sometimes all of the ToS.

I think this differs from country to country then, because I'm pretty sure that in e.g. Germany a contract is not valid or enforcable without prior explicit agreement from both parties. In Germany most ToS and EULAs are void anyway, because they are not presented to the user or buyer before the purchase or start of the transaction. Also, if ToS contain a serious error, the whole ToS is invalid.

However, in other countries such as the US contract law is totally different, so this does not generalize at all.

Also, IANAL.

There was a case in Denmark, where a large real-estate agency, was forced to pay damages, caused by keeping a large real-estate listings website, from scraping their listings.

If you publish the information, you might be breaking copyright rules. In some jurisdictions a collection of non-copyrighted items (like some catalog) might be protected by law.

If one wants to separate the question of scraping from the question of publishing something scraped (Which are entirely different things) then the thought experiment should be e.g. that you scrape a catalog and your page only displays "right now there are 10224 articles in the IKEA catalog" or "right now 1234 bikes are for sale on craigslist".

That material has been reduced to a number - which certainly can't be considered protected as a material.

This is why I've always had a bit of a trigger when I see signs that say, in a publicly accessible place, "No Photography!, as I, personally, am of the opinion that, if I have been able to freely and legally walk to a spot where I can see something with my eyes, I should be free to photograph or record it. It is My experience after-all, and it should not be infringed on me experiencing seeing something that is legal to see.

It's kind of shocking that you can't "see" any difference between a picture and looking with your eyes.

Also disappointing that you're (perhaps willfully) ignoring the many reasons why photography is banned/strongly discouraged in some public places.

Oh I can see it - I just don't agree with you or those that would sag that they should not be able to photograph such things...

That's all - I just don't agree with you