AWS CodeArtifact: A fully managed software artifact repository service
aws.amazon.com
aws.amazon.com
Most dependency management tools have some kind of hacky support for using S3 directly.
Full fledged artifact management tools like Artifactory and Nexus support S3 backed storage.
Interesting to see that the pricing is approximately double that of S3, for what I imagine is not much more than a thin layer on top of it.
Honestly the fact that they only support javascript, Python and Java is pretty bare bones compared to what the others on the above list support, and again as you say, for a fairly high price.
Haven’t looked carefully, but is there a difference in the guarantees it provides? Might be a performance or SLA difference.
All I'm seeing this does is give the proper http endpoints so you dont need the wagon. Is it worth ~2x the price, no, but it's better than the other enterprise-y solutions.
Avoid criticizing HN staff or related companies. Gentle / kind disagreement is fine, but err on the side of keeping it private.
There's a lot of necessary complexity in the backing platform. Encrypted package blobs are stored in S3 but there are a bunch of other distributed systems for doing things like package metadata tracking and indexing, upstream repository management, encryption, auditing, access control, package manager front-ends, etc... that are not immediately obvious and add cost. The platform that backs CodeArtifact is far from what I'd call a thin layer on top of S3. There is also a team of humans that operate and expand the platform.
Source: I lead the technical design for the product as well as a chunk of the implementation but left the team around mid-2018.
GCP has a similar offering[2]. And GitHub[3].
[1] https://docs.aws.amazon.com/codeartifact/latest/ug/python-co...
It is common for developers to use Git to store source code, in a hosted service like GitHub. It is common to use SSH keys to access Git. Frequently those SSH keys are generated without passphrases. Those are non-expiring credentials stored on disk. If HTTPS is used to access Git, it will likely be with non-expiring credentials.
I'm not saying short lived credential are bad, not at all. I'm pointing out how this service differs from similar services, requiring a change it workflow, which might be annoying to some people. Not everyone is operating under the same threat model.
The short lived passwords is a non issue and a good thing. Your dependency resolver should handle fetching the new password and most orgs I’ve worked at had scripts dealing with short lived passwords/iam.
According to AWS's documentation, none of the supported dependency resolvers will fetch the new password[1][2][3].
If they were capable of automatically fetching the new password without human intervention, it would mean they have credentials for generating credentials. If this isn't on an EC2 instance (where an IAM role can be used), that means there are long-lived credentials (probably written to disk) used to generate short-lived credentials.
This would be the case if you are using a hosted CI service that doesn't run on your own EC2 instances. You would probably be providing an AWS key and secret, which would then be used to generate the short-lived credentials. But the key and secret won't be short-lived, and will have at least the same access as the short-lived credentials (probably more access).
> Your source code may reference a shared library at a specific version from a trusted source to build. This trusted source is CodeArtifact.
HTTPS is what forms the trust between you and the artifact repository. Short-lived passwords don't do anything to ensure you are talking to the real trusted source. They may make it so the artifact repository can better trust you are who you say you are, but I don't see what they has to do with safely getting a specific version of a library.
[1] https://docs.aws.amazon.com/codeartifact/latest/ug/python-co...
[2] https://docs.aws.amazon.com/codeartifact/latest/ug/npm-auth....
[3] https://docs.aws.amazon.com/codeartifact/latest/ug/env-var.h...
In terms of local development experience, most mature organizations will have these "long lived" credentials still require an MFA at a minimum of once per day and locked down to particular IP addresses to be allowed to get the temporary credentials.[1]
> This would be the case if you are using a hosted CI service that doesn't run on your own EC2 instances.
Typically you'd want to see third-party platforms leveraging IAM cross-account roles these days to fix the problem of them having static credentials. Granted, many of them are still using AWS access key and secret.
This is still not a "solved" area though, and a point of concern I wish would get more aggressively addressed by AWS.
[1] https://github.com/trek10inc/awsume, https://github.com/99designs/aws-vault, and a few other tools make this much easier to deal with locally.
If you care about the security of these artifacts, why is their home directory (or their full disk) not encrypted? If they have access to the repository, they probably have artifacts downloaded on their laptops, so if the laptop is compromised, the artifacts are compromised anyway.
Edit:
Not saying temporary credentials are bad. But the reasons you gave seem a little suspect to me. A better reason is that you don't have to worry about invalidating the credentials when an employee stops working for you.
The problem is that your home directory is accessible to a ton of apps on your computer, and you have no idea what each of them is doing with that access. You also have no idea if any of them can be / are being exploited. The most recent case being Zoom – if that server they had running on localhost and responding to anyone on the laptop had file system access APIs (which is reasonable if Zoom had offered file sharing on calls) an attacker would have been able to read all your credentials.
And ideally these credentials should have similar controls applied around them as well (only temporary, using passwords to unlock the SSH keys, etc). If you don't have that, that's your choice, but just because some of your credentials lack security controls is not a reason for other credentials to lack security controls, too.
> you should be auditing every single package or isolating (docker, vms, Bare metal if you’re super tin foiled), anything short of that is fake security.
Which is exactly the reason that many orgs do specifically audit every package and disallow unapproved software. But again, even if some of your desktop apps are allowed unaudited, that is not reason to lessen your security elsewhere.
For the specific use case of the developer box and the Docker registry, resetting the credentials every 12 hours doesn't offer any more security than not on its own.
The reason for that is after you try to login to ECR after the expired time, the way you authenticate again is to run a specific aws CLI command to generate a docker login command. After you run that, you're authenticated for 12 hours.
If your box were compromised, all the attacker would have to do is run that aws command and now they are authenticated.
Also, due to how the aws CLI works, you end up storing your aws credentials in plain text in ~/.aws/credentials and they are not re-rolled unless the developer requests to do so. Ultimately they are the real means for Docker registry access.
There is absolutely no reason with the tools that we have available that we should be creating long living AWS keys. It's a major security risk if those keys ever got out.
Also the lack of nuget is a major issue.
Out of curiosity, what would you want from this service for the "plain binary" use-case when S3 already exists?
It's not advertised in the documentation, but HeadObject(Bucket, Key)['Metadata'] is a neat dictionary of custom values.
0: https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObje...
Perhaps I don't understand what you're saying fully though--as I don't fully understand your comment.
https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadat...
This is where S3 really shines. You can give developers access through group membership while servers using instance profiles. We have implemented a fine grained access control for the S3 repos that works really well. Of course you access the content via HTTPS.
IAM is on the AWS repo aswell isn't it? I guess it wouldn't be so bad then.
Are others not packaging their code in intermediate packages before packing them into containers?
We end up having a lot of libraries that are shared across multiple images.
For people that disagree with this model: where do you think the the software comes from when you apt/apk install things inside your Dockerfile?
Whenever I see a product announcement like this missing something I need to use it, I immediately ping our Technical Account Manager to get the vote up for a particular enhancement.
Source: I lead the technical design for the product as well as a chunk of the implementation but left the team mid-2018. I don't have any specific insight into their plans, not that I could really share them even if I did.
The git server you use supports artifacts already. You could also just put all of your artifacts on an S3 bucket if you needed somewhere to put them, which is exactly what this is but more expensive. I don't understand when this would save you money or simplify devops.
Not so on enterprise networks.
A lot of people won’t find this useful but for some it’s a big blessing.
Skim the docs and you will see it is not “just S3”.