Dear “API providers”, I don’t want a relationship.
tommorris.org
tommorris.org
P.S. the code samples are there help all the people that are not great developers as you.
I could harm your service by making too many API-key signed, OAuth-signed requests too. I could harm your service by hitting your website a lot too. We have ways of dealing with people who intentionally or unintentionally launch the equivalent of denial of service attacks: you block their IPs and move on. There's no need to have a special magical way of doing it with an API.
The point is the whole concept of an API should be unnecessary. We have a way of saying where data is: URLs. We have a way of specifying what format the client wants it in: content negotiation (Accept headers). We have a way of retrieving that data: HTTP GET. There's a reason why BugMeNot exists for websites. API keys are basically pointless registration pages for access to the same data that is being published on the web.
As for the code samples: if there's little more needed than "here's the URL of our data", I don't need a code sample. I only need a code sample when it's been made ridiculously over-complicated.
(My favourite API recently: clockworksms.com - all of the other SMS providers I've looked at want me to talk to some salesman and/or read complicated docs. Clockwork just let me send an HTTPS POST message. They have an API key, sure, but they required only an email address to get it. And I can pay for credits with PayPal. Ridiculously simple. I like.)
As for arrogancy? Guilty as charged. I'll say in my defence that it's more that the data I wanted to retrieve from the service in question (which I won't name) was exceptionally simple, had no commercial value in itself, but would send referrals to the site that they could monetise (and there's no affiliate scheme, I wasn't gonna profit off this). There literally is no business reason to lock that kind of API down. It's just cargo-cult API design: everyone else has API keys, they must have a reason, so I better have that too.
If thinking for yourself is "arrogance", then we need more "arrogant" people involved in web design.
It's not the "API" that is the method of providing and restricting access. It's the IP address. Any reasonably smart user can figure out the "API". They do not need a lengthy manual. A few examples, mere hints, is all that is needed.
This helps stop developers from putting lazy API calls on the client side instead of caching, for example.
For instance if my website includes a feed of my latest posts to a third party service I should be caching them myself and serving direct to visitors, rather than using a client side request coming from every visitor's browser.
Obviously this sounds like common sense but you might be surprised :)
The point that websites without API-key get hit by this anyway is valid though and legitimate users won't be annoyed too much by IP-blocking abusers of the API for API access only (block all Tor exit nodes for the API too, who cares ...).
You could also mention that authentication, registration etc. probably make APIs slower and more of a burden for the servers too ...
Another solution is to actually use User-Agent strings. Nominatim, OpenStreetMap's reverse geocoder, recommends that legitimate users put an email address in their User-Agent string. So it might read "My Craptastic Mashup v1. Maintained by: <whoever@gmail.com>". If there's a legitimate problem, email them.
When the site is down, people can't do any work. If they can't do any work, they lose clients. If they lose clients, I don't have a job anymore.
If the point of using SMS were to actually build an SMS service, I'd spend more time worrying about it and choose one of the more fully-featured SMS providers. And if it were to be a commercial thing, sure, I'd talk to their business development people. But everything starts at the micro level: what could be a business idea in six months starts as a crappy little hack now.
Here's a point of comparison. Esendex are usually considered one of the better SMS providers. How much do they cost? Oh, let's have a look at their pricing page. Doesn't tell me. Lots of blather. And a nice phone number I can ring and talk to a salesman or I can sign up for a no-commitment free trial. I don't want to talk to a salesman, I want to send a flipping text to myself in this crappy little Python script I'm writing.
It is a way of naming a data query.
HTTP GET is not a way of retrieving data.
It is a a way of launching a query.
In the general case, the data is going to be in an enterprise datawarehouse divided over 200 tables each of which could be terabytes in size.
The schema design will mean that each table individually is meaningless, and you will probably not understand it all without a lot of documentation.
If you were allowed to pull a table using a GET it would almost certainly be very expensive for the service provider, and might kill your download connection.
APIs were invented to deal with these cases.
Whatever complex enterprise data warehousing you are doing to get it in my browser as an HTML page is very impressive, I'm sure.
It can be as complicated as you want it to be if you want to offer something more complicated than that. But for the basic use case, I fail to see why exactly it has to be more complicated than using HTTP to GET things that are on the damn web. And all the talk of APIs makes people think it needs to be complicated when it doesn't.
But in your comment you say "The data is there in an HTML page. ... I just want to stick .xml or .json on the end and see the exact same thing in a machine readable form."
In the first case, you are asking for the absence of an API. In the second case, you are asking for an API that apes the user interface.
I just wanted to point out that that's an important difference.
EDIT: It is not correct to say the data is in the web page already, it is more accurate to say the needed query result is in the web page.
It may be necessary to have big complicated APIs some of the time. Great. Mostly, it's not necessary: you have a URI structure, you have pointers between records (hyperlinks), you just need a machine readable representation of the data. HTTP and web architecture already does that. You don't need a special fancy API with things I need to learn and understand.
On reading your clarification: okay. You term what I call "data" as a "query result". Then it's a merely semantic distinction you are drawing. And, I'd suggest, probably an irrelevant one from the perspective of the data consumer. There's a URL for a resource. I want the stuff there. I don't care whether it's in a database or how it's stored. That's just plumbing.
Currently, no API key is required, but this will likely change so we can better monitor usage and enforce the Terms of Use (below).
I apologise for the lack of insight.
I wouldn't advocate implementing an API to those bullet points, but it's certainly worth remembering when you are making an API that you aren't making it for you, the service provider.
As nothing but a reminder of that, the post is valid. It's certainly why I upvoted it... not because I think it's deeply insightful, but that it's good to have the reminder that on the other side of an API is a developer who I might be frustrating and causing problems for.
Also, on one hand you ask for unadulterated access to their data, but you explicitly don't want their free code samples? - "Give me free stuff, but not that, I don't want that!"
If the complaint is that there are needlessly complicated apis out there, it's a valid one. But the solution completely ignores any problem that does't fall into a very simple data provider model.
At some stage, people started to get weird ideas about URL schemes (aka "API's") like it's some sort of marketing thing. API keys. WTF? Imagine if early Google was required to get an "API key" for every site they crawled. Who came up with this silly idea of "API keys"?
In the 90's, I remember decipering URL schemes in order to do crawling long before anyone used the term "API's" to describe URL's. They are usually quite predictable, since almost every website used the same software. And they're even more predictable today. Crwaling got easier. Hmmm, I wonder why.
I also remember how most URL's used to be non-descriptive. Not true anyomre. Google has changed everything. Sites _want_ to be crawled now. But funnily enough they think they can make money from it. Heh, good luck with that.
If you want to charge money, then put the data behind a password and a paywall. And watch traffic plummet.
Example: http://wiki.eveonline.com/en/wiki/EVE_API_Functions
Oh god, I'm being tempted back to exploring New Eden again.
For non-EVE players: http://wiki.eveonline.com/en/wiki/3rd_party_tools
I don’t want to build an application for your service.
I don’t want a relationship.
I don’t want to talk to any business development people.
I don’t want to read your developer blog.
I don’t want to participate in your “ecosystem”.
I don't recall any API agreement that dictates any of those. No one is forcing you to read a blog or change your relationship in facebook to 'in a relationship with twitter API'. I don't get this.
I don’t want to get an API key.
Fat chance. Some APIs need rate limiting / authentication. What do you have against obtaining an API key ? Its not that complicated. Most APIs provide BASIC auth or let you get OAuth tokens.
I don’t want to read your code samples.
ಠ_ಠ
What likely happened was that you were pissed with the API that someone provided. It had complicated requests where HTTP methods and simple structures alone were not enough to describe the request and to add to that you were given a freakin complicated way to retrieve auth tokens. You then decided to take it out on all API providers without providing enough context on your rant.
[EDIT] - formatting
Tom, a grumpy developer.
I think you are tapping into the negative sentiments of some other recent articles. Namely the "Do You Really Want to be Doing This When You're 50?" article.
Perhaps people are laughing at you, not with you?
I'm not very familiar with Mozilla's Persona. Been meaning to try it but I've got to find the time and a suitable project.
It's a case of building my site from scratch, and I wanted a super-light-weight authentication system. So I picked Persona. (Incidentally, I picked it because unlike Twitter or Facebook or whatever, I didn't need any API keys to get started.) There are a few bugs with my implementation. But as it's just to authenticate me to access the admin panel, it's no biggie.