Hacker News Reader

Comment by ryandvm | Hacker News Reader

AWS S3 is a testament to just how successful a service can be if it is SIMPLE and RELIABLE. I recall using it in its very first incarnation in the 00's. From an API surface perspective there was barely anything to it, but it nailed being easy to use, easy to pay for, and as reliable as the sun.

View on HN

I agree that S3 is fantastic, but not sure I would consider it simple considering how much sensitive data is left wide open in publicly accessible buckets all the time. Clearly, applying correct permissions seems to be quite a challenge for even experienced folks. The S3 console doesn't really make life any easier either and terminology can be confusing. Also, applying IAM/CORS policies, object headers, etc isn't exactly simple for the layperson.

You have to jump through hoops to make buckets public and even then you get a clear label saying “public”.

The "public" label was somewhat recent (I think that happened in the last 18 months), but I think many of the biggest S3 leaks predate that labeling.

The S3 console now makes it quite apparent, in my opinion when things are left public. Public is no longer the default, and theres big scary warnings whenever you make something public.

Buckets are private by default and always have been. A bucket needs to be made public. Given that, I would hazard a guess that buckets are often made public (when they shouldn't be) for either testing purposes and never reverted or because creating proper access to the bucket took too much time/knowledge.

That's most certainly the reason.

I sometimes have to support freelancers working in some of our Wordpress websites. Their first instinct when something is wrong on their end is asking me to run a chmod or chown command they found on Google on the whole directory. Not that it matters – we're using Docker.

Security seems to be secondary when the priority is to just deliver.

> Security seems to be secondary when the priority is to just deliver.

Security is always an afterthought to those who don't actually understand it.

Security should be secondary. I know that sounds wrong. And that’s not an absolute, sometimes security comes first. But in general, everyone is trying to get stuff done and security gets in the way. My passwords get in the way of using my devices. My keys gets in the way of coming home.

That’s different from saying “security should be an afterthought”. Security is something you should consider consciously and prioritize against your other goals.

Putting security first is kind of weird if you think about it. Imagine building a house and prioritizing the locks.

Several billion people don’t have easy, immediate access to your house locks.

Further, sometimes users aren't truly aware of the implications of what making a bucket "public" means. Heck, maybe the original use-case wasn't too concerning, but it changed over time. The folks writing/reading from the bucket are also likely distinct from those setting up the infrastructure - although that's (thankfully) not always the case these days, it certainly can still be.

This reminds me of how React has a dangerouslySetInnerHTML[1] API - sometimes it's the hammer you need, but the name should (hopefully?) cause pause.

[1]: https://reactjs.org/docs/dom-elements.html#dangerouslysetinn...

A shotgun is pretty simple.

It'll still blow your head off.

Do one thing, but do it very well.

It's also at it's heart distributed state management, and that is generally speaking very hard to do for anyone to do correctly.

You don't distribute state mgmt in S3 alone since there is no lock mechanism.

If there is but one direction of distribution, e.g. when you deploy config changes, and never replace objects, this suffices.

Eventual consistency is a valid distributed strategy.

Unix philosophy for services encourages the user to model their problem rather than their architecture. Edge computing is a brewing shitstorm at companies which are not trending in this direction.

Care to elaborate? Like every sentence you wrote?:-)

Implementing a S3 client from scratch is no longer simple. V4 signature and payload signing is a pain and you have to read the doc a dozen times before you can successfully implement it.

>V4 signature and payload signing is pain

Don't be overly discouraged :-)

I have a reliable (if a bit ugly) implementation in under 3kb of sparse PHP OOP code. Yes it took a few hours to iron out the last kinks, and one extra refactor to free up a mistakenly bound variable. It might have helped that it was the first version I implemented, thus no need to match impedance of older interface.

On the plus side, the signing is designed as self-contained, and neither implementation nor API calls spill to too many parts of request handling.

>you have to read the doc a dozen times before you can successfully implement it

Very much so - it does require a small 'eureka' moment. It's hard to put your finger on what exactly makes it hard to get right the first time around. At the very least it is written from a somewhat peculiar viewpoint.

Fortunately the primitives used are common enough to be available in typical standard libraries.

I'm constantly reminded of Rich Hickey's point about good platforms providing few but powerful verbs.

Put another way, if you look at your service's API and it's complicated, you probably don't have the abstraction right.

Reply on news.ycombinator.com