Hacker News Reader

Comment by riking | Hacker News Reader

To be clear: the "blowing up" would occur when a client observed the new maven-metadata.xml file, but old ("does-not-exist") records for the newly uploaded artifact, correct?

With this update, ordering the metadata update after the artifact upload means this failure is now impossible.

View on HN

For some context, S3 used to provide read-after-write consistency on new objects, but only if you didn't do a GET or HEAD to the key before it was created. So this access pattern is all good:

  PUT new-key 200
  GET new-key 200

However, this access pattern would be unreliable:

  GET new-key 404
  PUT new-key 200
  GET new-key 200/404

This last GET could return a 200, or it could return a cached 404. Unfortunately, Maven uses this access pattern when publishing maven-metadata.xml files, because it needs to know whether it should update an existing maven-metadata.xml file or create one if it doesn't exist yet. So when publishing a new version, it does something like this:

  GET com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 404
  PUT com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 200

And then downstream builds would try to resolve version 1.0-SNAPSHOT, and the first thing Maven does is fetch the maven-metadata.xml:

  GET com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 404

So when publishing a new version, you could get a cached 404 and fail loudly. However, when updating an existing maven-metadata.xml file you could silently read an old maven-metadata.xml, and end up using an out-of-date artifact (which is even more concerning). Here's what one of the maven-metadata.xml files looks like: https://oss.sonatype.org/content/repositories/snapshots/org/...

Because updating an existing object in S3 didn't have read-after-write consistency, we could have a publishing flow that looked like:

  GET com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 200 (v1)
  PUT com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 200 (v2)

And then downstream builds would fetch the maven-metadata.xml:

  GET com.example/example-artifact/1.0-SNAPSHOT/maven-metadata.xml 200 (v1)

So downstream builds could read a stale maven-metadata.xml file, which results in silently using an out-of-date artifact.

We ended up just switching to GCS because it was relatively straight-forward and gave us the consistency guarantees we want.

Reply on news.ycombinator.com