Craigslist, LinkedIn, Netflix, and others don't owe us anything.
monkeymace.com
monkeymace.com
Padmapper made craigslist apartment searches much better, instead of making their own apartment searches better craigslist shut Padmapper's access to craigslist data down. It shows that they value their own business more than they value the experience of their users, which to a lot of people here isn't exactly "good".
Companies are free to do with their data whatever they want and they're free to restrict access to it however they like, sure, but it's still lame when a company does it because they don't want to improve their product and don't like competition.
my 2c
For example, It's probably fair to say that Monsanto is not a constructive part of the marketplace and community, but they sure do have a sustainable profit generating engine.
Edit - if you're going to downvote, how about some discussion as to how this comment does not add to the conversation... since I'm sure there was some other reason for the downvote other than not agreeing with me :)
Just having a business plan and making money should not protect you from criticism or absolve behavior detrimental to the general community! Yes, legally Craigslist has no obligations; that's why nobody's litigating. I argue that morally (at least from a utilitarian standpoint), Craigslist does have an obligation to play nice. And so I welcome critical blog posts and bad publicity.
That being said, in this specific case I think that whether or not there is a moral obligation to let a third party scrape and mix-in your data is very debatable (given that Craigslist does not provide an API).
Sure, people should play nice. And some may interpret playing nice as 'don't scrape other people's stuff for your own startup'. Or at least don't be suprised when they get pissed ;)
In reality, private companies are personality driven.
The consumers absolutely have a right to express that they are dissatisfied with the exchange being made.
My guess is that the site is generally mediocre enough that it would be hard to get more people to open their wallets except for select listing types.
If they charged to view the listings for example, I'd wager that very few people would bother and they'd lose their network effect.
My guess is that they're getting about as much money as they can without offending too many people as preservation of their user network is highest priority, as it is what allows them to keep charging money for the few things they charge for currently.
Just by putting un-offsensive elements such as a single Adsense box on each site they should be able to generate serious additional revenue without experiencing that much of critique. With that kind of traffic and legitimate profile, the un-intrusive revenue options are endless. Just additional fees for highlighted listings, dynamic pricing for a slower decay of a posting, etc. But the beauty of Craigslist is its absolute no-frills approach and consistency over years and years. Reminds me a bit on the success of Drudge. Similarly weathering all storms by concentrating on content and simplicity to the extreme.
I was naively thinking of charging users more money to increase revenue, and while you've pointed out a few very viable ways that they could easily get more wallets open, the real gold mine is likely ad revenue that nobody would bat an eye at.
The targeting/CPM would likely be fairly low, but by sheer volume alone, they'd rake it in I suspect.
Thanks for the insight.
It's true that if they do something highly awful, then that might create enough demand for a competitor to take over. But it would have to be a high degree of awfulness to overcome their existing momentum. There are many shades of not-quite-awful, where those companies can make decisions with market impunity.
So considering that we're kinda stuck with those guys, I think the question becomes, why shouldn't we complain if they are doing something that we don't like? Doesn't matter who owes who what. It's our world, we should try to improve it.
I think perfect market choice is a funny idea. Why would anyone assume the market would function on such a limited plane of understanding, that all you had to do was poll people, you could trust their verbal response, and decide that's what the market should do?
People very often do things in contrast with what they say, even what they believe consciously. There is so much more at play than what people would "vote" for in the sense of a "perfect market choice".
Unfortunately, us humans have a hard time banding together and forming enough consensus to 'jump ship' enmass to a new service. Especially, when the a lack of 'innovative features' on a given service barely registers on the pain scale.
Look at the internets SOPA protest, the entire integrity of the internet was threatened and we did something about it. But how can you generate enough support around: we need better mash-ups to view craigslist postings!
A) destroy the network effect with legislation. (How is this good?)
B) select another company The Board feels has a better platform, crush CL with legislation, and establish the new company with legislation. (How is this good!?!)
We need legislation to stop people from abusing legislation. Err.
I presume that is what we are talking about, when people here complain that nobody has been able to defeat Craigslist and that Craigslist ought to die (in so many words).
In the past, do you think anybody ever looked at the classifieds page in a newspaper and thought "this data is owned by the Times"?
In the PadMapper case, who are the losers? Renters benefit from a radically different UI and de-cluttering of daily reposts. Owners benefit from the increased traffic to their listing. Craigslist benefits because it's more of a 1-stop-shop for owners: post to craigslist (and pay them), and get some level of syndication to different services and apps. They also aren't losing out from reduced mindshare of renters: when you use PadMapper you know you're using Craigslist. It's always obvious and all links point there.
In the past, do you think when you submitted a classified to 1 newspaper, they called up all the other local and regional papers to submit your information too?
Nobody is expecting craigslist to push data to 3rd parties. They expose programmatic ways to access listings, and PadMapper used them.
Also, do you have any examples of this hypothetical happening and newspaper A not having a problem with it? Outside of tabloid culture, ye olde newspapers had much more in the way of principles, and using another's content without permission was reprehensible.
Its a loose loose scenario, much better to allow the reproductuon for everyone.
But even if the labour was free and paper B was willing to print ads/classifieds for no money, it's still a drain on paper A's resources - running a classifieds department required staff taking calls for placed ads, plus editorial work and similar. If paper B takes those ads and sells them at a lower price, then paper A will lose out on revenue for value created by their staff.
Advertisers are still paying paper A, so there is no loss of revenue.
It's kind of funny, isn't it? On the one hand we have the newspapers, who are being slowly killed off by the internet, and on the other hand we have internet startups like Padmapper who are doing things that are absolutely and plainly stupid when you imagine them being done by newspapers rather than web startups.
Here we have one company that allows free classified ads for almost everything, everywhere (Craigslist) on the one hand, and another company that scrapes free classified ads for rental housing and accepts no payment from anyone (Padmapper). Very strange.
FWIW, to me, Padmapper drives traffic to Craigslist rather than competing with it. Whether that traffic is valuable to Craigslist or not I can't say.
Don't look at me. That flawed analogy wasn't mine.
I've posted a couple things to Craigslist. Maybe it's different for rental listings, but for what I've posted, Craigslist did not demand exclusivity. If I wanted to post my listing to some other service, I was free to do that.
Similarly, regardless of what Craigslist's policies are with regards to third party access: it's still your data, right? You are free to post it to an open classifieds service?
That being the case, from where do you derive this idea that Craiglist has somehow taken custody of your data, and thus owes the rest of the industry some measure of access to it? Why is that Craigslist's job? If you want there to be multiple competing listing services, push your data to multiple services.
How about the convenience and enjoyment of their users?
I kid, I kid.
Maybe they realized the path to greatest profitability was to act like they don't care about profits.
Just a thought.
If you value users, don't hurt yourself to show it. Rather, don't shut off sites that save them hours of their limited and precious lifetime.
I hope that Craigslist changes their mind and decides to allow websites to license their API. If they don't, however, then Padmapper should just adapt and become mobile-exclusive, or try to generate their own apartment listings instead of depending on Craigslist.
It seems abundantly clear to me that the assertion that "A competitor to Craigslist who got $x right would easily get all Craigslist's users", for any of the usually mentioned values of $x [web design, UI, UX, customer service, search, API, other services using their API, …] - is nothing more than a head-in-the-sand misapprehension on the part of startup/web design/UI people. Craigslist _users_ have not "jumped ship" en-mass to any technically or graphically "better" alternative - and it's hardly from lack of trying by people who're _sure_ they will.
I think sometimes we need to get out of the HN/startup/bay-area echo chamber - I'm pretty sure Newmarket and his crew have a much different view of what their user base wants than any assumptions made by the HN zeitgeist…
I, personally, save several hours every time I look for an apartment by using padmapper over Craigslist. Aggregated, that's human lifetimes of time saved every year by padmapper.
UI isn't about round corners and fancy color schemes. It's about using computers to automate repetitive behaviors. Newmark's defenders tell me what I really want is to waste my limited lifetime clicking manually through entries on his shitty site. But no, I really don't.
Fuck Craigslist. Seriously.
> Fuck Craigslist. Seriously.
It's a bit disingenuous to say fuck Craigslist when the data you are using above...comes from Craigslist. Shouldn't you have switched to Zillow or another service by now if you hate Craigslist so much?
"Fuck Craigslist. Seriously."
Because they "cost" you "several hours" everytime you choose to use them to look for an apartment? Which is what - twice a year maybe? And this cost to you is somehow greater then the newspaper classifieds market they disrupted? Without some more backstory - I can't help but think you're _seriously_ over-reacting…
I think my point still stands - if there's a better way to do it that's so obvious, why hasn't somebody _done_ it and grabbed all the users? Google did it to Alta Vista. Facebook did it to MySpace (who did it to Friendster who did it to Tribe…)
It seems to me as though Newmark has balanced the often conflicting requirements of "people wanting to advertise stuff" and "people wanting search through advertisements" better than anyone else. Craigslist wouldn't have any data worth publishing in an API if they didn't address the needs of the advertisers. Padmapper doesn't have any data at all - why should Craigslist give them their apartment rental data just so Padmapper can compete with Craigslist without having the constraint of having to satisfy the advertiser well enough for them to provide the data in the first place?
My prejudice is that PadMapper probably wants to flip the company, and even if they don't, they probably have investors who want them to.
I trust Craigslist, and I don't trust PadMapper.
By charging for only a small number of listing, CL accomplish two things:
1. They create a large marketplace, which by virtue of its size provides more value than if it were a smaller, more diversified set of markets.
2. It "sucks the oxygen out of the room" for any competitor attempting to create a similar service. There are a few other commerce-related online sites -- eBay, Amazon, iTunes store, etc., and even a few competing specialized listings services (jobs, apartments, etc.) -- but none that specifically serve the general classifieds market.
I do suspect that there's a genuine component to CL's community service mission. But that doesn't mean the self-serving aspects of its behavior are any less valid.
Padmapper is the Mozilla.
If I place a classified ad it is to find a buyer for the services or product I'm trying to move. CL currently does that better than the competition. Yet the comments I see being handed around are about UI, design, automation and other items that might, but don't actually increase my chance of selling. To replace CL those sites need to provide the person posting the ad with a chance of finding a buyer that's equal to or greater than CL.
If those new start-ups can't provide that, but only have a shinier UI to offer, then they're competing on irrelevant details that aren't the main drivers behind people's decisions to post/view CL.
Others have mentioned marketing approaches and techniques to help improve those aspects. But most of what I've read is summarized as: "we'll provide a better app/ui/api" and that's a noble goal, but the reason for existing of these sits is to connect seller with buyer, what do those things bring to the table if a new app lacks a sufficiently large viewing audience? That's the problem that needs solving, not the app/ui/api one.
This implies that part of the quality of a product is NOT the ability to gather data. This would be like me walking into a Walmart, setting up a cash register next to theirs and start ringing up customers and arguing that Walmart can't compare with my customer service. Ignoring that customer service is just one small part of the business. Walmart had to find the property, buy the property, do the legal work to get the product on the shevles, the negotiation, the supply chain management, etc... I can't just focus on the last mile and then argue that I'm entitled to everything else they did.
Craigslist built its name, it's reputation, it's backend, it's API, vetted a business model for its services, scouted out regions to focus on etc...
Furthermore, people are arguing the data is the users, not CLs. True, but the user gave it to CL, not to you. When I loan books to a neighbor I assume they aren't going to create a library of my loaned books (unless they specifically told me so).
I'm not sure if it's a sense of entitlement fostered by all the seemingly easy money around tech startups or simply sheer laziness.
Housing is 1 of the 2 things (job postings in certain cities is the other) that Craigslist charges for. So if Padmapper gets popular, starts listing from it's own website and people go directly to it, bypassing Craigslist, Craigslist takes a Massive hit in revenue.
THIS is a huge weakness. If you want to kill Craigslist start with Housing and Job postings in the cities that Craigslist charges for. Those 2 things ALONE are what are funding all of Craigslist.
That's human nature. Users in marketing communities conclude that it's the best marketing that wins. If you're an executive you think the best management team is all that matters. Designers think it's design. PR folks think it's press. Financiers think it's money.
I have bad news for everyone. The world does not work in the way our echo-chambers would seem to model it. It's almost impossible for the human mind to process such a fact. Yet, it's true.
I might be inclined to agree, if it weren't for network effects.
Once a company reaches a certain size and market share, like Craigslist, would-be competitors are at a competitive disadvantage. Lots of people have created objectively better sites than Craigslist, but they fail because of network lock-in. Normal capitalistic competition is failing.
This is the same thing that happened with railways, utilities, and other natural monopolies. In these cases, it is necessary and proper for the government to step in and regulate access and interoperability.
Sites like Craigslist and LinkedIn have become the new natural monopolies. And demanding that they open up their data is the natural, and right, action to bring back natural market competition that benefits consumers and the world at large.
Craigslist disrupted a huge classified industry without any legislator saying that papers had to "open" up their data. Markets work over time.
I'm not so sure (for reasonable values of "forever"). From what I see, once some software hits a critical mass, it's nearly inextricable.
Look at Microsoft. Before they became a major player, the dominant OS and computer system would seem to shift every couple of years; Amiga, Commodore, Apple, etc., but once Microsoft became a major player they never really lost their grip. They continue to maintain that grip simply because so many people were familiarized on their systems and so much software was written for those systems. Does anyone really believe that someone looks at OS X and Windows these days and decides that Windows is objectively superior? Most people, even technical people who hate Apple's MO, would admit that OS X is better on its merits alone.
The reason that MS has dominated and continues to dominate is because of the software available for the platform. It's all about lock-in. People don't use Windows because it has more compelling features than OS X, they use Windows because they want to play games, buy software off the shelf and know it will run without any problems, or run their company's internal software.
Let's take Facebook as another example. For years there was a pretty constant, roughly bi-annual shift in the most prominent "social networking" site; LiveJournal, Friendster, Xanga, etc., until Facebook. Around 2008 Facebook began eating MySpace's lunch and Facebook remains undisputed social networking champ today with no signs of slowage.
I'm concerned that Facebook is the new Microsoft -- it has become the de-facto standard, and despite any number of superior features that may be developed in other systems, it doesn't look like anyone is going to be able to challenge Facebook seriously. Some would say that Google was perhaps the most worthy challenger, but even they have failed to really make a dent despite the fact that most of the world visits their search and email services daily.
I'm not sure exactly where that critical mass gets hit but I think it's hard to reverse once it happens. Maybe it's something like, "Once your grandparents use a certain program because everyone else does, it's relatively safe to call it 'ingrained'." I know that my grandma had no compulsion to create a MySpace profile even though she constantly [attempts to] check[s] Facebook now, and I also know that it'd be extremely difficult to train her to use a non-Windows interface.
Ahem. I may not use it because it has more compelling features than OS X, but I did not rule out OS X because of lock-in.
Craigslist disrupted the classifieds industry because it had first-mover advantage in a new medium, the Internet. It now abuses that advantage in order to not innovate or even compete.
The market is not working over time -- proof is how everyone complains about how crappy Craigslist is.
But a solution, for example, could be something like: legislators could force companies with over, say, 25% market share, that are mainly based on user-generated content, to not be able to sue other companies if those companies re-display information originally input by users, that is publicly visible on the site.
This wouldn't force Craigslist to alter its behavior one bit. But it would allow entrepreneurs to actually fight once again on the basis of their ideas, as opposed to whoever had first-mover advantage and subsequent network lock-in.
Two - you have way more faith in our legislators than I do. Remember SOPA, PIPA etc from way back in the day. Oh wait - that was a few months ago.
Inviting legislators in - be careful what you wish for.
Then their ideas are simply not better enough.
An idea can't "beat" Craigslist. It's the whole of a product and team mixed up with constantly shifting market dynamics and human preferences.
It's just way more complicated, and legislation shouldn't even come into the conversation here unless the company is provably violating another's civil or property rights. Are they?
Here's an idea that can beat Craigslist: put up billboards all over town, offering $500 per ad to property managers and individual landlords in return for giving your site a 4-week exclusive on those ads.
That's the only way people will stop putting ads on Craigslist first, which in turn will be the only reason people will stop looking there first.
Network effects as powerful as Craigslist's can't be fought with a better product alone, as you point out. (See eBay for another example.) You either have to spend some money to take their market from them, or you have to be there when they fuck up.
And whatever else you can say about Craigslist, they are very, very good at not fucking up.
There is no lock in. A buyer or seller is free to move place their listing or make their purchase from any number of alternatives regardless and Craigslist can't stop them.
> Sites like Craigslist and LinkedIn have become the new natural monopolies.
Mark Zuckerburg would probably disagree with that statement. In 2005 MySpace would have been the equivalent of Craigslist, massive network, shitty design and no data portability. If the value provided by alternatives to CL were as superior as most HN readers believe them to be then END USERS would see that as well and migrate just like MySpace users migrated to FB. But they don't so CL retains its dominance, the failure of alternatives is not indication that CL is cheating or doing anything even remotely monopolistic.
> And demanding that they open up their data is the natural, and right, action to bring back natural market competition that benefits consumers and the world at large.
The data is open, just not in a format you think you are entitled to it in. It's open in that the same poster that put their listing on CL can post the same listing on alternatives, they just choose not to. Convince the seller or buyer of the value you bring to the table and they will open their data to you as well.
Many HN readers have reached the conclusion that convenience and competition are the same thing. They aren't.
If the value provided by alternatives to CL were as superior as most
HN readers believe them to be then END USERS would see that as well and
migrate just like MySpace users migrated to FB.
This type of dilemma is known as a collective action problem. When these problems arise the best way forward is often to have a superauthority such as the government step in for everyone's benefit. For example, even though it is generally accepted that free trade increases the world's prosperity when it is practiced around the world, individual countries occasionally see immediate advantage in adopting mercantilist or protectionist economic policies. The GATT is an attempt to provide a superauthority to stop such practices, because the immediate gains possible when a country embargoes an enemy are not as valuable as a system where the majority of actors behave in open and unbiased ways.
The free market doesn't work on its own. When there are systemic problems in the way actors conduct business, judiciously applied regulation can help promote the public interest.
That's not necessarily true. I see network effects as something of a negative externality, one that is hit particularly hard by the tragedy of the commons. The problem is that even if every single person on Facebook decided that YourBookSpace was a better website that provided more value, no one would leave because until everyone else followed, your network wouldn't be there.
This type of dilemma is known as a collective action problem.
What most on HN don't want to admit is that the brilliance of their code or the beauty of their design alone are not enough to guarantee success. We still have to market our wares and if the value derived from our code+designs isn't greater than the value provided by CL's network + usability + cost then that doesn't mean we call our parents or a referee or a senator and tell them we need the playing field leveled because the end users are too lazy to see how deserving we are of their business.
Paper classified still exist, web based CL alternatives still exist (Backpages.com is a big one), CL is not a monopoly and a "superauthority" is not the answer.
And how would we know when a "superauthority" needs to step in, "judiciously applied" is subjective. Successful passing of SOPA would have been "judiciously applied" "superauthority" in the eyes of the entertainment industry.
The free market doesn't work on its own.
I do think there are web companies who could really do some anti-competitive things that we should watch out for. Facebook is probably the most capable, or maybe Google if they went off the deep end and started holding companies hostage through their search. The argument still stands, I think, even if it doesn't justify intervening on behalf of a company that wants to improve Craigslist search.
1. The goal of almost every rational business is to achieve a monopoly in a certain space. If the public or the government had the power to systematically break up every company who comes close to success, there would be little incentive to start a business. Perfectly-competitive industries are awful for business owners. You have to draw a line in the sand, before-which monopolies as tolerable. I don't think Craigslist is anywhere near that line.
2. A huge part of anti-trust laws means that companies engage in anti-competitive practices. To the best of my knowledge, Craigslist isn't doing this. It's simply refusing to willingly participate in its own destruction.
3. I question your usage of the words "normal" and "natural". First you claim that monopolies indicate the failure of normal capitalistic competition. Then you claim that monopolies naturally occur. Then you claim that we need to act to bring back natural market competition. So which is it? Are monopolies "natural" or is competition? And why does something being "natural" make it good?
So, so many people have forgotten this. Thanks for pointing it out.
1. No, the goal of most businesses is not generally to achieve a monopoly, the goal of most businesses is market share or just profit. A company can have 100% market share and not be a monopoly, if the barriers to entry for new market participants are comparable to the original barriers to entry for the current winner (so there aren't network effects, or prohibitive capital investment relative to divided market size).
2. I never talked about anti-trust laws or anti-competitive practives. I talked about natural monopolies [1], which have nothing whatsoever to do with those. You're thinking of artificial monopolies, something totally different.
3. I said "normal capitalistic competition" in the sense of competition outside of special cases (like monopolies). And "natural monopoly" is an economics term [1]. I used neither in a normative sense.
Consumers do not get to choose alternative railways. They can chose alternative classefeids services; those services are just inferior (maybe they have a better UI but they aren't as good at selling things)
And yeah, these companies don't "owe" us anything, but consumers (even the subset of developer-consumers) can still get pissed off at them and say shit and that's the fucking market. And maybe it won't mean anything, maybe it'll all fade away... and maybe it won't. That's the risk those brands are taking. It's our market-based prerogative to bitch about whatever the hell we want to bitch about, using whatever hyperbole we want, with any sense of entitlement that we've acquired.
If someone starts sending mail bombs to these companies because they aren't tending their APIs as we'd like, then that's coercion. But there's no coercion here, this is just people offering their opinions. So quit yer whining about our whining, the internet has entitled us to whine as much as we fucking want to, and has offered market-based ways to lift or bury that whining.
It would be great if there was regulation over certain types of data that would require companies to post back to a central database if they take certain kinds of information. It would be great if there was a device that could help level the playing field.
The point of my post was to highlight that complaining, or being shocked by these companies behavoir is sort of missing the point.
How can we unify and provide access to what we feel to be 'universal' data types?
My god, I hope not. The last thing we need is legislation that determines, because I went through the work of building up and creating a successful product and now have a massive amount of valuable data, I have to share it in the interest of 'leveling the playing field'.
I've been on the side of being a content creator with a very successful site, that was promptly scraped by a competitor because I had ammassed a large amount of very useful data. This data that was, through my site, freely available to the public, and the ad revenue helped pay the bills.
So I really don't have a lot of sympathy for someone who had, as part of their business model, the practice of scraping content from someone else's site, nor do I agree that we, as developers, should have free and unrestricted access to someone else's API... they are the ones gathering/storing the data, building the network, etc.. and any use they decide to allow us is at their discretion and a gift. Nothing wrong with that.
The idea of mandating any such rule is not the solution. It's antithetical to how markets should work.
There's been no company that's managed to dominate forever. LinkedIn, CL, Netflix will be no different. Someone or ones will attack them (perhaps orthogonally) and ultimately they will lose their dominance. That's what happens to incumbents. The market takes care of them. Note: this disruption may or may not be quick.
I'm not so sure. Communication tools are fundamentally different than most other products, because their primary value is the number of other people reachable through the service. This isn't true of most other industries, where you can drive a Toyota and I can drive a Ford and everyone is happy.
When your a whole market is based on the ability to interact with other people in some way, and a single company can flip interoperability on and off like a light switch, it has the potential to be a dangerous monopoly. The government stepped in and broke up Ma Bell because of this.
If we characterize them as communication tools, we are using a pretty loose definition. Moreover, inviting regulators (via their involvement in some sort of monopoly breakdown) into the proverbial henhouse is a nice mix of anti-free enterprise, anti-entrepreneurship and misguided/naive.
"I'm from the government and I'm here to help!" - music to every Internet entrepreneur's ears.
Google: man up and build your own database of everything. Stop indexing our data when we already offer our own search mechanism.
I get that everybody wants to build ajax web apps and ios apps and android apps and so on, but what's the point of the internet except for being the default client-server protocol if we can't create mashups? The whole point of the internet is that data is public and linkable and interconnected. If companies don't want to allow scraping and they don't provide an API for their non-user-specific data then they're defeating the whole point of the internet.
The vast majority of their users don't give a hoot about whether or not other parties have a way to wrap up craigslist data and do stuff with it.
The whole point of the Internet is that it's a collecation of independently run networks that can communicate. The point of the WWW is to serve hyeprlinked pages to people. There IS NO POINT to the internet - it just happened, and here we are.
Sure, they don't owe third-parties anything, and personally I think it's folly to base all or even a significant portion of your business on data or functionality that you don't control.
But that doesn't mean that the community corporations do business in must always just shut up about it when, for instance, one company pretends to support a third-party's efforts for two months, just long enough to get a good hard look at their user experience, financial standing and business model, and then summarily shut them down without recourse.
"It's just business" is fine, but it's not a get-out-of-shame-free card. Just as third-parties can't complain when an API is yanked out from under them like Lucy with a football, so corporations shouldn't get butt-hurt when the community they work with looks at them and says, "the way you did that really stinks."
Internet is still relatively young, but I think eventually we're going to have to have laws that regulate how companies that collect user details allow access to 3rd parties.
Monopolies are illegal for a reason. Companies who build infrastructure have to manage it fairly.
APIs are nice, but legal hassles aren't worth dealing with.
Legal hassles are totally worth dealing with if the cost of managing them doesn't exceed your profit. If you're the right company, legal red tape can be a very effective barrier to entry.
And for that data, they provide you with services. That doesn't mean they should be required to hand that data out to any service that requests it?
I mean, are you really asking that companies like LinkedIn allow free access to it's api's and user data to any 3rd party that requests it?
> Monopolies are illegal for a reason.
Well, no. Monopolies just require additional oversight. Microsoft, for example, didn't get in trouble for being a monopoly, it got in trouble for unfair business practices as a monopoly. More importantly, a monopoly doesn't exist with any of the companies presented in this article.
Really though, your comment fits with the picture at the top of the article.
It does, however, mean that it's up for negotiation, if the consumers want it to be.
So, either you are saying something else, or just stating the status quo.
I think less people would be concerned about the issue if it was just Craigslist putting up technical measures (e.g. blocking API access) to competitors, but competitors were free to rent VPSes or pay people for use of their IP address to access the data for scraping. If there was no law against scraping information from websites that is available to arbitrary members of the public (provided that you comply with copyright laws post-scraping), then there would be no issue.
So the real issue is not that Craigslist owes anyone anything, but rather, that the government is enforcing laws on their behalf to entrench the network effects of existing businesses against new entrants to the market.
I mean that PodMapper was a service for Craigslist's users, first and foremost. And banning PodMapper means that Craigslist doesn't give a shit about us, its users. Do you really believe that PodMapper did any damage to Craigslist?
You seriously think Craigslist doesn't give a shit about users? Think about what you are saying. Think about the world before Craigslist. It's a free service that even my mother uses. This is the _exact_ sense of entitlement that the OP is talking about.
So for me personally PadMapper provided 50% of the total CL positive experience.
I've used ebay over 100 times. Ebay is ripe for disruption, imho, but it's still okay.
You sound very nervous.
First and foremost because you make wild assumptions and accusations. They don't give a shit about users? Have you talked to the Craig? What about other employees? You are throwing a company under the bus because you don't like its design and don't like them controller their API. I hate their decision with respect to PadMapper, but I reserve doesn't give a shit about users for Monsento, tobacco and alcohol companies. You aren't their only user, you might not feel like they don't give a shit about you, but any grander statement is just big meaningless talk.
Secondly, as I originally stated, because you think Craigslist owes you something. They don't. Deal with it.
> you might not feel like they don't give a shit about you
May I, nevertheless? Thanks in advance!
What's the difference between CL and Monsanto (Philip-Morris) in this context? All these companies sell something which is legal and have a demand. Users pay for the goods (seeds, listings, cigarettes). Users appear to be happy. Users want to get some extra value out of the goods/services (put the PadMapper on top of CL, breed their own seeds from Monsanto modified plants, dunno about cigs, though :-)). Oops, lawyers abound! Now Monsanto owes something to farmers, but CL doesn't owe anyone anything. Hmmm. You're throwing Monsanto under the bus because you side with farmers. That's okay. But who told you that you're absolutely right?
I can easily deal with CL not owing me anything - I don't do anything on CL.
Please, calm down.
Whether or not we think PodMapper is a better service for "us", it's not our choice to make on behalf of Craigslist.
Wouldn't we all better off as a whole if technology wasn't solely based on profit only?
I think that's the real question, and what started the whole "API" "open XXX", not "OMG I WANT IT /CRY/CRY" that the image suggests. Remember that most devs nowadays participate in open source projects and start to have different ethics than they used to have (which was, money > everything, and justifies all actions)
Or the dev-became-CEO, too, I guess. Not all of them of course.
I just wish it would keep on being "more open" for these reasons, and it would be sad to go back to an "all proprietary, closed, etc" world.
As a provider of data from which these companies make their profits, I believe it is entirely reasonable to demand that they make those data accessible to other services I'd like to use. Sadly the trend of the Internet has been away from open systems and toward siloed, proprietary data stores.
What you're upset about is that you've already given them your data, and now you want more for it. You've upped your price after handing over the product, and them having delivered the product.
Am I somehow mystically bound to never ask for more in return for the data I provide them?
Of course we can ask to change the terms. It's a business transaction.
This is silly, and you probably think you are being smart.
That's not what's being argued. Read the parent comment, and there is an expectation that it's your data and you have the right to decide what to do with it, and that you essentially should be able to dictate to other companies how they have to share it.
I call bullshit.
> What about the posts I put on Craigslist in the future?
What about them? T&C. It's your data. You are choosing to give it to Craigslist.
> The ratings I give to Netflix?
What about them? Those are your rating? You are choosing to give it to Netflix so Netflix can give you better recommendations.
> Am I somehow mystically bound to never ask for more in return for the data I provide them?
That was never suggested. Nor is it what is happening here. People are acting self-entitled, demanding that their data, which they given to other companies, somehow is still theres by some non-existent right.
> Of course we can ask to change the terms. It's a business transaction.
Which has nothing to do with what's being discussed here.
Here, let me help explain: if you sign up and agree to use a site, and provide it with data, then the business relationship already exists. If you suddenly want more, you need to renegotiate. But that doesn't mean the company should be required to hand over that data (which is what many are suggesting) that they own (and just because it's about you, or you provided it, doesn't mean it's yours to control).
So yeah, you can ask them to change the terms. And when they don't, you stop using the service.
I swear, this is like reading a bunch of posts my self-entitled, egotistical nobodies who, in all truth, never really cared about their data in the first place.
After all, if you did, this wouldn't be a problem for you.
What I'm reacting to is this idea that things are the way they are and they can't change. That the companies have the sole power to set the terms for what they do with data provided to them by their users. And of course they don't. It's an exchange, and if the users, as a group, want to ask to change those terms, they are completely withing their rights to do so.
There's nothing self-entitled or egotistical about that. It's simple business to request the terms of an exchange be changed.
No, you're demanding change. Demand has a meaning to it. You are demanding they do something. You aren't asking. You aren't negotiating. Your are demanding!
> I believe it is entirely reasonable to demand that they make those data accessible to other services I'd like to use.
Here is some good advice: Condescension doesn't make your point any more valid that it would be otherwise. You'll actually find people take you more seriously without it. =)
> That the companies have the sole power to set the terms for what they do with data provided to them by their users. And of course they don't.
You are wrong. First, your framing this wrong. It's their data. You've shared that data with them, they've collected it, stored it, processed it, and done what they will with it. Outside of a breach of the terms you agreed to, I don't see any reason they are beholden to you or anyone else.
> It's simple business to request the terms of an exchange be changed.
But that's not how you are framing it. You aren't requesting anything. You are demanding. And, more to the point, you are demanding you have control over all aspects of that data, even, apparently, data you generated using their product.
So again, the problem is you see this as data that is yours. It's not. You've paid them already, with that data. It's like giving someone money and demanding they do what you say with it.
It was your self-entitled attitude that reeked from your comment. If I was wrong with that, sorry.
In that case, your demanding they give you something for nothing.
> I'm not sure what makes you think that customers can't demand things of those they do business with.
Demanding things while offering nothing in return doesn't help. Also, "Please point to the part of my comment where I said customers can't demand things."
> Also, who's saying I've upped my price?
You have. You're asking for control over their API's.
Seriously, people like you think these companies have your data. They don't. You still have your data. You've shared it with them. It's theirs now. If you really cared about controlling your data, you'd have worried about this when you first signed up.
You're just acting the part of a self-entitled egotistical internet drama-queen "fighting the big guy oppressing the little guy", and frankly, that song-and-dance get's old quick.
If there is API access it should be enforced with more care or not made available at all. Personally, I was skeptical that PadMapper would go anywhere (expecting CL to kill the scraping early on). Much to my surprise it stuck around and furthermore was really useful. I'm not mad that it is gone: I read the TOS from CL - it's totally their right to do that. It just hurts end users which is the problem.
And sure, I do believe it's dangerous (and can be misleading to users [like when your data access gets pulled]) to build a product entirely on a platform, but it's almost unavoidable. (That's why you are the platform). Companies like PadMapper are going to try to improve where you have left off. (They probably didn't even register on CL's logs until they queried a lot which might explain the delayed C&D).
Let our entitlement / disappointment be a message to these companies on how to improve their services (ideally they could improve in a way that makes them more money). Or hopefully fuel for others to build competing products.
That's a two-way street.
As crazygringo pointed out, a very large part of the value of these firms comes not from their technology but simply from their market position, dominating a specific niche, and generating network effects as a consequence of size.
In a market in which there's a tendency toward monopoly: utility services, telecommunications, desktop/personal software, business office software, business systems software, advertising networks, publishing, broadcasting, pop music/entertainment, agricultural middleman, major chip foundries, pharmaceuticals, a large part of the value accruing to the firm/organization is a consequence of the interest / business which society as a whole has invested in the monopoly.
At the same time, twin abuses of monopoly pricing (which accrues a greater marginal profit to the monopolist) or anti-competitive actions (driving out competition by way of locally (time or region) undercutting prices, contractually discouraging or prohibiting customers from doing business with competitors) exacts an additional cost on society as a whole.
I've long been a fan of Craigslist. I think that their business exemplifies some of the best of how to conduct an online business (leverage technology massively, but also rely on community goodwill to conduct many business operations, including removing of bad/fraudulent listings). While I'm not a particular critic of CL's page format -- they've held to the KISS principle in the extreme -- it's become increasingly clear that for many types of listings, there are some very evident improvements which could be made.
My feeling is that CL are squandering an opportunity in their dealings with Padmapper. The Padmapper interface, for apartment and real-estate listings, is vastly superior to CL's existing listing format. While there is a business risk to CL in allowing another party to utilize its comprehensive listings (for which CL would reasonably be able to assert compilation copyrights, answering some here), I really believe it would be to CL's benefit to find a reasonable business relationship under which it could utilized Padmapper's interface. Whether that's a data exchange agreement, a purchase or leasing of the technology, or simply a reinvention of it, I don't particularly care (I also owe Padmapper nothing in particular).
And if that's not the case, then, well, CL show a case of a market which, if a feasible business case can be made, is ripe for disruption.
Moral is if you quit, you are a quitter. If you are useful to people find some other solution implement it asap and keep rolling. I am not yet living in the sates but if I truly believe that my service is useful to people I'd just go and try to do it differently and move on with that. Never give up, especially if you are doing something that is good. If you're causing rucks and are being useful maybe that might be an opportunity of a lifetime...
my 2c
The point is that Craigslist is not the only side in the transaction and should not be presumed to be able to set the terms on their own.
This is also how one destroys markets. Say for example dumping cheap corn on markets to destroy the livelihood of local famers, then when they are gone jacking up the prices far above what would have been possible when the local farmers were still operating. This and other monopolistic practices are illegal and generally accepted to be morally reprehensible in most of the world.
There is a implicit deal that is being struck between suppliers and consumers. That what we get today we can get tomorrow. TQM style supply contracts often call this out explicitly building long term positive relationships between supplier and consumer. Unilaterally changing the deal is bad for buisnes by leaveing the overall environment unable to reliably plan or predict future actions.
And, if you go against that approach you shouldn't be surprised or indignant when you get shut down. You will have a much better chance building a truly complimentary product, rather then rallying users to boycott a service or demand changes in policy.
To me it's sort of like someone who always drives over the speed limit by at least 50mph, and one day they finally get pulled over and are given a ticket. But, because they were never 'caught' before, they just feel like they were entitled to always drive that fast, and instead of just paying the ticket, they try to get the speed limit laws changed, or to have a judge throw out their ticket and let them keep driving however they want.
When you use another company's API, you are driving on someone else's road, and for better of worse you need to play by their rules.
So if you want to drive fast with no consequences, build your own road. And if your interested in 'exploiting' or piggybacking on someone else infrastructure, don't do something to call too much attention to your self.
PadMapper - started providing other listings that were not from craigslist. GoodFilms - providing information and data to other movies services besides netflix Pealk - were undercutting the price point for LinkedIN premium features.
Both sides had a point, and while my bias is with [Bidder's Edge], I don’t mean to deny the plausibility of a different regime. What I do deny, however, is that the answer to this question is obvious.
One is "a website like website x does anything it does to generate profit". That would be the side the OP takes. The other is "a website like website x does anything it does to make the world / the internet a better place", something like that. I think an open source enthusiast would take this position.
Obviously people value these ideals differently (otherwise this discussion wouldn't be there) and therefore voice different opinions. With that in mind, I think it's easier to evaluate the opinions of others on this matter.
On the other, I think that when you have a businesses creating industries around themselves, that a significant amount of that autonomy has to be abridged.
Take for example Ebay and how they strongly encourage Paypal. If you run into a problem with Paypal, and your business has a significant presence on Ebay, that's a pretty heavy cost, and I don't think Ebay should be able to say "so long, tough luck."
While unfortunate, the move by Craiglist isn't surprising. They have done this repeatedly. The situation PadMapper is in is inevitable for any entity that isn't self-sustaining..
PadMapper should be working as quickly as possible to capitalize on the added attention this has caused and start letting users input their listings directly..
If you got a C&D as well and isn't happy about it, you should go ahead and make a better product. Otherwise you can go to work for one of their competitors. That's the beauty of being a hacker.
Just make it sure that online classifieds is your passion.
I was watching Ghost Dog: Way of the Samurai last night. In it there are a number of Samurai codes that are highlighted in title cards. One of them was this:
"If one were to say in a word what the condition of being a samurai is, its basis lies first in seriously devoting one's body and soul to his master. Not to forget one's master is the most fundamental obligation for a retainer."
Craiglist is available by RSS and would not be too hard to scrape. You want "API access", you got it. Unlike Facebook, I haven't heard of anyone tossed off Craigslist for using a script to access the data Craigslist is happy to give out for free.
Ah, but you don't want "API access", you want a license to resell the data - and for free! You want different terms of use. Etc. Wah, wah, wah.
Guess what? That's different. Obviously.
Other companies' data, or other companies' users' data?
I really would like to stop hearing the whine "but they have a right to do X" every time a business's practices are criticized here on HN.
My point wasn't so much that they have the right to do that, but more about when your playing with another company's API don't expect to disrupt them.
Choose a start-up that is already successfully and profitably using a company's API and make your service even better. AKA don't compete with the mothership.
Your own linked articles about that topic are a user only complaining about the legal takedown of a useful tool [1] and another complaint about UX with an included blurb asking startups and users to put their time and data into something better [2]. Nothing about API changes or right to a company's proprietary data.
[1] http://blog.garrytan.com/save-padmapper-craigslist-is-wrong-... [2] http://pandodaily.com/2012/05/30/craigslist-is-squashing-inn...
If Microsoft decides tomorrow to say "if you want to write Free/Open Source Software, you can't use our API's" do they have a right to do so? What if it is competing with Microsoft software, i.e, "no use of our API's to build competing web browsers and office software" or the like?
I am genuinely asking because I think it's as clear a line as one might think.
The line is clear, the source of the service has the right to do whatever they want with it. Unless you want to claim that a company's property belongs to the people.
So I think in Microsoft's case I would argue that they have a) no legal or moral right to impose such and b) no effective mechanism to enforce such conditions. They make practical tools, and people may use those tools in whatever ways they see fit. To the extent Microsoft can limit this through a EULA, they are subject to all sorts of judicial scrutiny.
For example, I have a hard time imagining that a "you may not run a web server on this edition of Windows" would be enforcible. Client access license requirements might be in some cases but I don't know what the dimensions are that they would be.
The question with Craigslist only becomes harder because to interoperate you are interacting with Craigslist's infrastructure. This is of course covered arguably by a different set of laws which may give Craigslist a bit more freedom. But we have laws in many states that restrict what, say, shopping malls can require of people entering (a shopping mall in California, for example, cannot prohibit pamphletting).
I would like to see similar rules passed regarding the internet equivalent of shopping malls, to be honest.