Hacker News Reader

Comment by ryandrake | Hacker News Reader

Out of all the zero-sum tech arms races (increasingly complex DRM, SPAM senders/blockers, software crackers vs. copy protection, code obfuscation) this one seems to me to be the stupidest. Here we have people putting data out in public for free, for anyone to access, and then agonizing over how someone accesses it. If some data is your company's secret sauce, your competitive advantage, don't put it out on the Internet. If your data is not your competitive advantage, then why bother wasting all this development effort stopping browsers from browsing it? So much waste on both sides.

View on HN

I agonize about this every day, since a large part of my job is aggregating data from many sites that seem hell-bent on not letting anyone access it without going one-form-at-a-time through their crap UI.

The thing is, we would gladly pay these companies for an API or even just a periodic data-dump of what we need. We've even offered to some of them to write and maintain the API for them. They're not interested, for various industry-specific reasons.

I often wonder how much developer time and money are wasted in total between them blocking and devs working around their blocks.

Sometimes when I'm thinking about it and what 95% of developers are working on, it feels like a planet-wide charity project against unemployment.

I think it's a fairly well known thing that 'junk jobs' tend to spring up in response to supply. I think it's a bizzare cultural thing.

The travel industry is highly protective of its data. It's my understanding that they consider it proprietary and only sell it to those who they deem worthy.

who says this is zero sum? in the limit, it seems like a loose/loose situation to me. or at "best" modest rewards for [spammers|scrapers|...] at tremendous (spread out over the population) cost in loss of usability and compute cycles and development effort

This is like saying, if you're going to give people free samples, why not give away the whole grocery store?

Wanting to give out limited free samples inevitably leads to making sure you are giving out samples to people and not bots and not too much to each person, and that leads to user tracking.

Compare with the arms race between newspapers and incognito mode:

https://www.blog.google/outreach-initiatives/google-news-ini...

Reply on news.ycombinator.com