Why isn't your API specification public?
apiful.io
apiful.io
Psychologically, looking at a formal specification is pretty intimidating. How many people do you know who enjoy sitting down with a cup of coffee/tea and their favorite time-worn leather-bound copy of RFC7231?
Most developers I know love getting something to work. They love the feeling of having mastered something to the point that just reading "GET /widgets" is enough for them to go off to curl and implement the rest ad-hoc as they're visually inspecting the JSON and occasionally referring back to the docs.
Creating formal specs is probably the most efficient way to go in a purely computational sense, but reading specs is hard work and it's not especially inviting or rewarding, especially to a class of workers that would much rather jump in, get their hands dirty, and figure it out themselves. It's why developers love frameworks/APIs that just feel simple and familiar. The low cognitive overhead, and low barrier to entry gets you into a wonderfully endorphin-driven flow that's hard to replicate.
Formal specifications are great. But they won't likely get read by people. And people, not machines, are going to drive your profit as a business offering a shiny API.
So should you focus on your formal spec first, or human-readable API documentation? Probably the latter. And good luck convincing the business owners that a spec is a high priority now that you technically have everything covered.
However, what if the specs were able to capture what is in such human-readable pages (for example https://docs.cloudant.com/)? What if API spec could be automatically derived out of HTML pages? Embed some metadata in HTML page and then we have both cake (human-readable) and eat it (machine-readable)?
We ended up using Swagger, which doesn't solve as many of our problems (having documentation generated from code isn't as robust from a testing POV, particularly when your RESTful API is versioned), but my "perfect" is the enemy of the company's "good" :).
Scrape HTML to swagger: https://github.com/bobby-brennan/scrape-to-swagger
Swagger to HTML docs: https://github.com/lucybot/lucy-console
We also have a commercial version of the latter which you can see at any-api.com
I agree the API provider's primary concern should be Developer Experience, and understand why creating a spec often takes a back seat. But I suspect that soon it will be seen as critical for any production grade API to have a machine readable spec.
I view specs as analogous to unit tests - it doesn't get you closer to a deliverable product, but it makes the development lifecycle much less painful.
And I disagree that formal specifications are intimidating. A formal specification is just code, so if you have trouble understanding something, you can enlist the computer's help. On the other hand, if an informal specification isn't clear to you, you are more or less screwed.
Therefore, I am trying to find or create systems that do this without this requirement, in a mostly trustless p2p fashion, starting with an extension to the OpenAPI spec:
http://i.imgur.com/4mmckZ9.png https://gist.github.com/void4/523f23b50e1572e8ef60
I wonder what comes after that. Most probably different transports and patterns of interactions. http://iris.karalabe.com/ might show how it will look like.
Having the API specification cover more of the business/service aspects is very interesting (could you possibly define cost per call per endpoint or per payload size for instance).
The only costs that would occur would be transaction fees for changing state in the Ethereum network (which is only required once for registering the service and at the order of half a cent at the current scalability level) and a similar amount for opening and closing the payment channels. I'm currently implementing a prototype extending http://hug.rest. If executed correctly, the following three lines of Python:
@hug.get("/pay_world", requires=pay("$0.001")) def pay_world(): return "You paid for this"
would result in the framework generating the OpenAPI spec amended with the payment metadata in the background and making it available under the service root. A middleware layer would check every ingoing request for payment signatures, closing the payment channel if necessary. Interesting things would become possible, although it is not clear if it all will succeed. Many different kinds of payment mechanisms would be conceivable, from pay-per-call to subscription based. Hypothetically, you could even pay APIs with gold backed cryptocurrency or tokens you issued yourself.
See here for a similar project and reasoning: https://github.com/etherapis/etherapis
IMHO key component here is automatic integration, you simply publish a link to your spec and you magically have integration with a number of 3rd-party tools/services.
I currently work on catalog which does exactly this, and you can see list of integrations here: https://github.com/APIs-guru/api-models#existing-integration...
I think realityking nailed it in terms of the API community needing to define compelling use cases. The catch 22 here is that it is hard for people to innovate around API specifications when they are so hard to get! Per the SDKs ... what if there was a NPM.org or rubygems equivalent of high quality SDKs that get automatically generated whenever I push a new API specification? That seems like it could be cool.
And your example of automated composition is fantastic! Curious if you are able to share how much work it is for you to get access to the specifications in order to make this possible? If we invented a common practice to publish something like api.company.com/spec.yaml could you have spent a lot more time on making cool integrations vs mining specs?
If you've a Swagger/OpenAPI spec, you can use Swagger Codegen (https://github.com/swagger-api/swagger-codegen), a free and open-source project, to generate API clients (SDKs) in C#, Ruby, PHP, Java, ObjC, etc.
Here is an example: https://github.com/wing328/petstore-php
You can also use Swagger Codegen to generate server stub in SpringMVC, PHP Slim and more.
Disclosure: I'm a top contributor to Swagger Codegen.
(To convert RAML, API blueprint, WSDL, etc to Swagger/OpenAPI spec 2.0, please try https://github.com/lucybot/api-spec-converter - an open-source converter written in NodeJS)
Yes, this is exactly the problem which I try to solve with my collection.
> Per the SDKs ... what if there was a NPM.org or rubygems equivalent of high quality SDKs that get automatically generated whenever I push a new API specification?
You can run the same tool on API owner side you don't need to publish your spec for that. I did a couple of interviews with API owner and they fear to loose control over SDKs, Docs, etc.
> Curious if you are able to share how much work it is for you to get access to the specifications in order to make this possible?
No, it's public info since my entire work is open-sourced under MIT license. I try to make process automatic as much as possible, so it starts from scraping. For example, I scrape Github for Swagger specs: https://morph.io/IvanGoncharov/API_specifications When I need to filter test, example, etc. APIs and find specs for real-life APIs. But get spec source is just first step, I need to fix errors in it(~80% have them), get additional info(logo, link to API key registration, etc.). But most time consuming is update them every day. My update/convert/validation scripts not ideal so I need to manually inspect all changes.
> ould you have spent a lot more time on making cool integrations vs mining specs?
Actually not I learn a lot from it, for example, you can fix all mistakes in API owner specs. Instead, you should give them a tool which will automatically inspect API spec and output errors/warnings/recommendations/hints. Together with my friend we working on OpenAPI/Swagger linter.
If you want to discuss more here is my Skype(ivangon4arov) and Hangouts(ivan.goncharov.ua@gmail.com) or APIs.guru public chat https://gitter.im/APIs-guru/api-models
Contentful's API specification (in the blue print format) has been open for a while as a side effect of opening up our documentation. I'm not aware of any user ever using for anything but the generated documentation (and there have been bugs in the past that would have made things like generated test servers break quite nicely).
What (potential) use case am I missing?
With regards to the bugs, were they a result of the implementation and the specification getting out of sync? Would automatic means for creating / maintaining specs be able to help here?
In my opinion, API specs should probably be generated from code and its documentation, and not the reverse. This is because it is the code and not the spec which is subject to most change and only the endpoint interfaces/signatures can be generated. API specifications could also be more easily transformed by middleware (e.g. when renaming endpoints, combining APIs) if they were machine generated.
As was described above, only few people are acquainted with service description languages, and deeper changes in their semantics over time or the choice of an alternative would require a manual rewrite. Also, some SDLs seem to impose their own higher level ontologies and name spaces, which is another major obstacle to usage for simple systems.
Second, I like your enthusiasm for specs from code ... the problem (opportunity?) today is that the specifications intertwine human readable descriptions / summaries with the actual endpoint / payload information.
I would hate for my devops pipeline to kick-off because somebody fixed a typo in some human focused in-line documentation. I personally don't think it belongs in the code but absolutely agree with the spirit of your comment (if you keep them separate it is more work and they will get out of sync!).
The best luck I have is working within the specification (i.e. Swagger) and avoiding doing things that don't model well. This seems backwards, but for most of the development I've been involved with to date it has served the projects well.
Interested if others have had luck embedding API specs into code.
Swagger-Codegen (https://github.com/swagger-api/swagger-codegen) is a pretty mature, free and open-source project for generating API clients (e.g. ObjC, Java, C#, Ruby, Python, etc) and server stubs (e.g. PHP Slim, Java Spring, etc). Please give it a try and let us know your feedback by opening a ticket via https://github.com/swagger-api/swagger-codegen/issues
Disclosure: I'm a top contributor to the project.
I lead the API Harmony team at IBM Research and this information would be extremely valuable.
Yes, the API is public, but you need a Cert bound to your user data to get anything from it.
This is an orthogonal issue to whether or not the API itself is publicly accessible.