Hacker News Reader

Comment by falcolas | Hacker News Reader

Scraping the MTA site against the TOS, and intentionally obfuscating requests to limit the chance of the MTA identifying and shutting it down? Yeah, no surprises here.

EDIT: Oops, missed that he didn't end up going live with the scraping.

It would still be great for the MTA to publish this data intentionally; but I'm sure the potential for trains to switch tracks and the associated backlash when they do is what prevents them from doing it.

Perhaps they could adopt the airline model, where there's a best effort to reach a particular pre-announced track, with notifications ringing out when they can't.

View on HN

The author didn't do that, though. As far as MTA is concerned the app was always using an API he had been given a legitimate key for.

As an aside, fuck websites that prohibit scraping. If you send me some bytes I'm gonna do whatever I damn well please with them.

In that case screw your API, if you allow DDOS I may as well do it for the lulz.

Isn't it the same approach? If you can abuse a site you will, because they opened themselves up to the public?

That's an illogical extreme. I don't think anyone thinks that, rather, the question is whether this app was an abuse of MTA's system. They claim wide distribution of the app is a hazard. Really, it's not like MTA is stopping anyone with the inclination to do this themselves. If it's as reliable as the author says, you could do it without any automation if you were patient enough. Downvoting because that kind of exaggerated strawman reduces the quality of discussion with no benefit.

Thanks for explaining the downvote. But this is not exaggerated nor a strawman. It mirrors the extreme "screw your website" reaction and investigates whether the logic for such an extreme disregard holds.

So a website allows you to "download some bytes". Yes you can do anything you want with THOSE bytes. Does this mean you should make an app to let people scrape the website in an automated way and possibly overload it?

If airlines, restaurants give something free or discounted, does that mean you can make an app to systematically let millions of people take advantage of the arbitrage?

XKCD even has a comic for exactly this situation: https://xkcd.com/1499/

I wasn't the one who said "FUCK WEBSITES". I used a nicer word and questioned the logic. Yet I get downvoted while the parent comment with expletives is upvoted.

Pretty sure that if you read a little further, you will find that the author decided to NOT go with the TOS violating process, and instead used an official MTA API.

> Scraping the MTA site against the TOS

Where does it say it was against the TOS?

They could publish the log of passed trains, though. This will hurt nobody.

Reply on news.ycombinator.com