The AWS SDK for Go is now 1.0 | Hacker News Reader

The AWS SDK for Go is now 1.0

aws.amazon.com

142 pointsGeneticGenesis30 comments

I've used the SDK in the past (before the 1.0).

I am far from being a Go expert but you can see this has been written by Java developers.

The usage is so verbose and you need to pass pointers around all the time. Even when it's clearly not changing the original instance (when passing by value would be enough).

If you compare it to another lib like goamz you can clearly see how easier it is to use comparing to the Amazon SDK.

There's also an issue from June 4 that discusses this: https://github.com/aws/aws-sdk-go/issues/265

In my lib I ended up going with goamz, it was much simpler to use and required much less boilerplate.

I totally get why they are using structs rather than positional arguments, and pointers instead of literals.

Go doesn't have function overloading, so if they used positional arguments, they could never change the signatures in a backwards-compatible way. With structs, they can add new fields.

And the reason for the pointers is that Go doesn't have optionals or sum/union types. My own experience is that the lack of optionals, or some similar construct, is extremely annoying when writing clients and servers for APIs where parts of the requests and responses are optional. You have no choice but to dick around with pointers.

goamz is more idiomatic Go, but it's going to have a problem with backwards compatibility when Amazon changes their APIs. I'm with Amazon here, myself. goamz has a bunch of functions that take 4+ arguments, and given the lack of named argument support, that's when you really want to refactor your arguments into structs.

I'm all about passing structs and objects into methods and not make the arity too complex.

However, there's passing the struct and there's passing a pointer to the struct.

Also, if you look at the structs, you see that instead of just passing strings, they pass aws.string("some-string-value") which actually just takes the string and makes a pointer of it. That's what I'm talking about when I say it's too verbose, it's just a lot of unnecessary things like this.

It's not unnecessary, though. In Go, a string cannot be nil, as you know; it is initialized to the value of an empty string. That's not true in any other languages, and even the Go AWS library cannot use that as a signal of intent, because you may have intended to use an empty string.

Granted, I don't know if there are any calls where an empty string is an actual valid parameter. But it seems Amazon is erring on the side of explicitness here.

I agree that the AWS library isn't ergonomic. But Go is forcing their hand, in my opinion.

The original code was autogenerated, which is why the style was very terse and verbose. It's still like that:

https://github.com/aws/aws-sdk-go/tree/master/models/apis

The fact that it feels very Java-esque comes from two things, in my mind:

   - the AWS API has Java-esque RPC calls
   - the request/response paradigm is easier for 
     code generation

Personally, I don't like the style of the SDK, but at least it's maintained by AWS; you can always wrap the parts you use to get a nicer interface.

What do you mean by "terse and verbose"?

Ugh I failed at english ='(

I mean, "terse" in the sense that it is very mechanical, raw, not very elegant. Verbose in that you need to type a lot of characters, or read a lot of instructions, to understand the meaning.

raw instead of terse?

I am not native in English too. anyone can think of a better word.

Maybe "inelegant" or "rough".

Haha, those words mean the exact opposite.

What was used to auto generate the code?

Unless things have changed drastically, they use internal tools to generate the code. I used to work on one of the SDK teams.

They have a makefile target called generate

https://github.com/aws/aws-sdk-go/blob/v1.0.0/Makefile#L31

Reading the code, it looks like the most important lines are around here: https://github.com/aws/aws-sdk-go/blob/v1.0.0/private/model/...

This is the result of auto generated code. Pretty much every auto-generated code library is going to be fairly verbose, and behave this way.

Despite this, I think it's perfectly fine to use. I wouldn't even consider this verbose by most standards. The fact that it uses structs for it's only parameter instead of a lot of arguments is a huge win for API backwards compatibility. It's also significantly more flexible when you want to omit values.

I've used goamz in the past. It's good code, but I always had problems with lack of connection reuse on S3. Apparently coaxing S3 into not closing KeepAlive connections is non-trivial, and with goamz they'd always get closed.

This is a huge problem when you're trying to upload several million tiny files as quickly as possible.

When I switched to Amazon's official S3 library, this problem went away.

I have no doubt that Amazon usage of their own advantages is better than goamz.

SDK usage is all about the getting started and the documentation. For me, building an open source project around the Amazon SDK was just too much.

Obviously, documentation will solve a lot of this. I did not visit the project in months, maybe they improved on this.

For the record I'm a professional Java programmer and while Amazon's library works great, it often feels like the 'enterprise Java' people try to make up to mock enterprise Java.

Not trying to be a troll but I really do not understand what you are saying.

> I am far from being a Go expert but you can see this has been written by Java developers.

The same was true of their Ruby libraries. Thanks to the auto-generation, super annoying to debug when things went sideways :)

There have already been a lot of comments about the structs and generated nature. But consider that CloudFormation also appears to have been created by Java developers. It seemed(though maybe not?) very evident to me that much of AWS is written in Java. The types spill out into their API's, like CloudFormation. It can be verbose, but this type information is actually very nice when you want to hook into it from a typed language(csharp, go, java, etc).

I was just starting out with building a Go app on AWS and didn't even consider finding an alternative to the official SDK. Thanks for this comment - I'll now go and checkout out goamz.

Having used goamz and one of the other forks off canonical's original SDK, I'll stick to the verbosity, consistency, and general reliability of the official one.

> I am far from being a Go expert but you can see this has been written by Java developers.

Hi! As of writing, I am currently the top code contributor to the AWS SDK for Go (NB: I've since moved out of Amazon). I've never written a day of professional Java in my life, but thank you for the compliment! Would you mind endorsing me for Java on my LinkedIn page? This will go far to getting me that enterprise Java job I never once dreamed of having!

But seriously, it's worth noting that the SDK was originally created by Stripe, and most of what you're likely referring to (struct inputs) worked in the exact same way when it was Stripe's codebase. As far as I know, Stripe is also not a Java shop.

I've heard this exact same comment at the release of almost all of the AWS SDKs I've been around for, so I'm not too surprised to see it again. I think this comes from people incorrectly assuming that (a) most Amazon developers only use Java (this is far from true), (b) that complex APIs are inherently "Java-like", and (c) auto-generated code is somehow evil, even though it yields fewer bugs, more consistent usage, better documentation, more performance optimizations (compare boto's issue tracker to Ruby, for instance, and count the number of defects).

The reality is that sometimes complex APIs are just what they are: complex. Goamz is "easier" because in most cases it completely ignores the complexity of the full API by only exposing a small subset of it. For many use cases this is fine, but I've spoken to plenty-a-customer who told me stories of trying to hack in extra API calls to Goamz when it couldn't do what they needed. That doesn't sound "easier" to me.

Unfortunately, when you're writing a generalized library meant to account for the simple and complex use cases alike, you can't optimize only for the users who touch a small portion of the API's surface area. You have to start with a base that supports all use cases. If AWS had released a library like Goamz, you would hear the other set of users complaining just as loudly that they can't do X with the SDK, talking about how poorly designed it is for them. Over time, you may see the Go SDK add specialized APIs to make certain simple use cases easier (just like has already been done in other SDKs), but this won't happen right away-- it requires a lot of effort to identify and prioritize those cases. The S3 upload/download managers and the DynamoDB marshaler abstractions are a good example of this. Instead of yelling into the wind, I would recommend opening issues to ask for similar simplified APIs for use cases that are genuinely complex with the current SDK. But note: they should be genuinely complex, not just "this takes 4 lines of code and it should only take 2".

> The usage is so verbose and you need to pass pointers around all the time. Even when it's clearly not changing the original instance (when passing by value would be enough)

Passing by value is not enough. Go cannot differentiate a string value of "" (or int value of 0) from one that was omitted by the user, which is why we (very painfully) had to make the decision to use pointers. Making this decision sucked; we literally spent weeks trying to whiteboard workarounds that wouldn't completely gut the core functionality of the SDK. Unfortunately, @lobster_johnson said it best when he pointed out that Go forced our hand here. The lack of overloading, named parameters, default arguments, nil-by-value, really are the reasons why it's just not possible to make this more elegant in Go. Using positional args, as pointed out, wouldn't even last 2 months before you would see breaking changes in the API calls for many of the AWS services. Things really do change that fast in the APIs, it's the underlying reason why all of the AWS SDKs moved to auto-generated code. It's the reason you now all get SDK updates on the day of the API release-- with way fewer bugs along the way. That's a good thing.

I think the team did a great job getting this out given the circumstances. Go is a solid language for app development, but it's not really optimized for generalized libraries. The lack of common language features (literally, the lack of "generics" for generalized interfaces) means you're effectively writing code with one hand tied behind your back. With a foreign keyboard layout of your evil coworker's choosing.

aws.String just to get a pointer to a string rubs me the wrong way. I guess that lets it be nillable but I still don't like it.

I have been watching it's development fairly closely, seeing their strange decisions, and hoping they would rectify them. They haven't, and as a whole the API is not written "the Go way". It's written like they are fighting the language.

I assure you that in Go there are better solutions to much of this. I pray V2 to be better but if it's anything like their other SDKs they'll avoid breaking changes to the interface until its simply no longer reasonable.

Their PHP SDK is largely powered by magic getters, setters and method calls, and the methods therein are largely defined by a file describing the API. This works but is hardly ideal particularly for static analysis. I was at able with a PR to talk them into putting the method hints back at least.

You can really tell they were missing the magic in Go and working very hard to try to make it work in a somewhat similar fashion.

Full points to Amazon for not treating client SDK's like some sort of third class afterthought. And double points for not just palming off client SDK responsibility to "the open source community".

A cloud feature does not exist unless it is supported by the SDK that you program with.

The crappy design of this library screams after thought. The google cloud go sdk is far better.

https://github.com/minio/minio-go is a nice S3 Client library in Go

Now introducing random 500s in Go!

Great. I've been using it for the past few months. Documentation is terrible and there was quite a lot of breaking changes, but other than that it works... so I am okay with that.

Yes, but does it play Doom?

Reply on news.ycombinator.com