An experimental, automatically generated set of AWS clients in Go | Hacker News Reader

An experimental, automatically generated set of AWS clients in Go

github.com

153 pointsbowsky44 comments

Really happy to see these projects for automatically generated code for based on service descriptions. Minimal code to maintain and near instant updates for service providers. I was thrilled to see Amazonka (https://github.com/brendanhay/amazonka) a little while back. Great to see this done in Go as well!

This is from HN contributor codahale, originally mentioned in the Atlas from Hashicorp thread.

"(Apologies for the self-promotion, but it's at least on-topic.)

I just opened up this repo yesterday: https://github.com/stripe/aws-go.

It's really raw, but it uses the JSON API descriptions from botocore to generate Go clients for all 40 public AWS services."

https://news.ycombinator.com/item?id=8729740

Excuse my ignorance, but isn't this essentially a tight(er) coupling between the client and the server, something we have been trying to avoid with REST and, especially, HATEOAS? What happened to API “discoverability”? As I understood HATEOAS, the point was that the client just had to know how to work with particular data types (that's the point of an API, really), but was not expected to know or construct URLs beforehand, thus loosening the coupling to the server. Did I get it wrong? Did that path prove to be unrealistic?

That was not the path taken by Amazon as they made their nearly 40 different APIs. Engineers from there could certainly speak to the circumstances behind their choices.

As for aws-go, I had some APIs I needed to use and a machine-readable description of those APIs. The choice was pretty clear.

Feel like I'm in college and missed a class lecture - what's the standard for JSON API descriptions and where can I learn more so that I can upgrade my APIs to have a usable JSON description?

Follow-up: Looks like JSON Schema is emerging as the standard, in part thanks to Swagger and Google adopting it for their APIs. References to Amazon and Heroku doing so as well (not exhaustive list, just what I've found while searching).

Helpful links: https://github.com/swagger-api/swagger-spec/blob/master/vers...

Powerful array of tools to enable wrappers for Go, Scala, Flash, Java, Objc, PHP, Python, Python3, Ruby, Node, and more. https://github.com/swagger-api/swagger-codegen

Not sure if there's really a _standard_, but there are examples of projects[1] and generators[2] that encourage best practices. Presumably if you can document your API as JSON, it should be reasonably well-formed.

[1] https://bgentry.io/blog/2014/01/09/auto-generating-a-go-api-...

[2] https://github.com/pksunkara/alpaca

Update: Here's a link from [1] that may be closer to what you're looking for: http://json-schema.org/

https://stackoverflow.com/questions/12806386/standard-json-a...

Swagger

"Swagger is a simple yet powerful representation of your RESTful API." http://swagger.io/

What an awful name for such a useful project.

Thanks. Here's GitHub link for others interested like me: https://github.com/swagger-api/swagger-spec

"Kickin' APIs to the curb unless they look like Mick Jagger" :)

This reminds me of the Google APIs client libraries for Go. They're also automatically generated from JSON.

https://github.com/google/google-api-go-client

All of the Google API client libraries are automatically generated. There's a discovery document that's published for each API to allow for this: https://developers.google.com/discovery/

We have come full circle: implementing WSDL code generation with JSON definitions.

Amazonica (https://github.com/mcohen01/amazonica) for Clojure automatically generates a client based on the AWS Java API

There is a similar Perl implementation that we use at my company (Semantics3), which uses auto generated code from the botocore - https://github.com/pplu/aws-sdk-perl

It's a strange thing that Amazon doesn't have an officially written and supported golang SDK.

Especially since they have bet so heavily on Docker which is built with golang.

I wonder if Amazon don't support golang directly because it's the Google language.

To be fair, it took them ages to start getting serious about Python. It's still not what I'd call a first-class citizen, in that boto (even with paid AWS employees on it) lacks support for things like their ElastiCache discovery protocol (last I checked earlier in the year).

Source: I heavily contributed to boto in years past. It has progressed by leaps and bounds since then, but I still don't see it mentioned in the AWS blog posts when Ruby/Java/PHP get new shinies.

Well, AWS CLI is built on botocore. I've used boto sinne 2012 and never got the feeling it's a second class citizen. Mitch's leaving Amazon didn't slow things down. Nowadays when new services are released there is usually a new version of boto available immediately.

Or good Python3 support for that matter, sigh...

http://aws.amazon.com/blogs/aws/boto-python-3/

Boto has supported Python 3 since early August :)

Oh wow, finally! Totally missed that.

After spending so much effort trying to get boto 3 to work with python 3 I completely gave up and re-wrote all my scripts in python 2. August isn't that far away.

I thought Python 3 was fully supported by boto?

there are languages inside of Amazon that are first class citizens (java & ruby mostly, node.js more now). Go is too new, moving too fast, and hasn't seen the same kind of adoption yet to warrant an SDK.(despite what the folks here might think..)

This sort of argument is very odd. Amazon can't justify putting the resources into building a golang SDK for AWS but it appears that Stripe can. Perhaps even more strangely, someone who works at Stripe can justify doing it on their spare time, but Amazon what hasn't got the money to justify it?

Companies that provide programmable systems should be the first to provide SDK's, even for technologies not yet widely adopted. That goes for any company that provides something programmable.

Amazon appears to have no trouble at all justifying the massive development effort required for every new back end web service they build, but front end programming API access just isn't valued as much.

Honestly, is that really a surprise at all?

REST web services work well. Most languages have frameworks with excellent support for them.

Historically, the community has provided wrappers for popular APIs pretty quickly.

In the specific case of Go, it's still in early adopter territory. That means they are unlikely to have a problem using the raw APIs.

> Amazon appears to have no trouble at all justifying the massive development effort required for every new back end web service they build, but front end programming API access just isn't valued as much.

Their back end services make them money. Creating a Go API does not.

Makes it hard to justify investing development resources.

I didn't do this in my spare time.

I did this in rough a week and a half of full-time work.

Not widely enough adopted, and yet Docker is built with it. So golang ultimately is a technology that Amazon has bet the farm on.

For the downvoters, I assume this is a reference to the Amazon EC2 Container Service.

http://aws.amazon.com/ecs/

I think Amazon when comes to money especially on the web service side of things doesn't think in this way.

I'm more oriented that the language is new so is still hard to find devs without consider that is a very a big work port this huge amount of api.

I guess they also have a lot of software running that they rely on and is written in C or C++. And still, there is no SDK - there is also no (Node)JS SDK. Using Docker does not force you to support golang in any way.

Java is the absolute priority in Amazon, while strangely, though Python is widely being loved and accepted, is absent in the official support list.

Huh? http://aws.amazon.com/sdk-for-python/

For AWS, not so long time ago NO, right now it is a YES. If you look at boto, it still doesn't have built-in client side encryption, while Java has. It is trivial to implement your own with Python, however it does tell there is difference here.

For internal projects, situation is even worse and many of them are community-maintained.

Disclaimer:　Amazon Employee.

Does anyone know to what extent are the botocore JSON service descriptions similar to Swagger service descriptions? Or are they entirely different?

The SOAP dream is alive, after all!

Hah, I've been building a similar general approach for any rest service: https://github.com/boourns/restgen

Why is it so many of these submission titles tend to include "in Go" at the end?

There are many, many submissions of things written in many languages and they don't generally append "in Scala", "in JavaScript" or "in Julia", etc., unless it's a post specifically about the language features or experience of writing the given software in that language.

When it's a tool such as this, is there some reason why it matters to a user if an app was written in Go specifically? Are Go users just particularly zealous? Honest curiosity here.

Really? Code generation tools shouldn't mention the language they generate code for?

(Put another way, you might want to take a moment to understand what the tool is before you use it as an excuse to complain about Go.)

Because otherwise it makes the object of the title impossible to understand. There's already plenty of libraries of this kind of other languages. Not for Go.

very excited about this, given the breadth of aws services, this seems the only sane way to get solid coverage. thanks

Reply on news.ycombinator.com