On the client side, you just have to pin to an older version of the AWS SDK till whatever compatible service you're using updates, right?
Also, this is the first I've heard of OpenDAL. Looks interesting:
It's had barely any discussion on HN:
Back when Microsoft started offering CosmosDB, I was working at MongoDB on the drivers team (which develops a slew of first-party client libraries for using MongoDB), and several of the more popular drivers got a huge influx of "bug" reports of users having issues with connecting to CosmosDB. Our official policy was that if a user reported a bug with a third-party database, we'd do a basic attempt to reproduce it with MongoDB, and if it actually turned out to be a bug that was with our code, it would show up there, and we'd fix it. Otherwise, we didn't spend any time trying to figure out what the issue with CosmosDB was. In terms of backwards compatibility, we spent enough time worrying about compatibility for arbitrary versions of our own client and server software for it to be worth spending any time thinking about how changes might impact third-party databases.
In the immediate week or two after CosmosDB came out, a few people tried out the drivers they worked on to see if we could spot any differences, and although at least for basic stuff it seemed to work fine, there were a couple small oddities with specific fields in the responses during the connection handshake and stuff like that, and I think as a joke someone made a silly patch to their driver that checked those fields and logged something cheeky, but management was pretty clear that they had zero interest in any sort of proactive approach like that; the stance was basically that drivers were intentionally licensed permissively and users were free to do anything they wanted with them, and it only became our business if they actually reached out to us in some way.
Just because you can get something to work doesn’t mean it’s supported. Using an Amazon S3 library on a non-Amazon service is “works but isn’t supported.”
Stick to only supported things if you want reliability.
I stopped giving amazon the benefit of the doubt about any aspect of their operations about 8 years ago.
Do these 3rd parties get veto power over a feature they can't support?
Can they delay a launch if they need more time to make their reverse-engineered effort compatible again?
It seems a hard to defend position that this is at all Amazon's problem. The OP even links to the blog post announcing this change months ago. If users pay you for your service to remain S3-compatible that seems like its on you to make sure you live up to that promise, not Amazon.
Clicking through to the actual git issues, it definitely seems like the maintainers of Iceberg have the right mental model here too. This is their problem to fix. After re-reading this post this mostly feels like a click-baity way to advertise OpenDAL, which the author appears to be heavily involved in.
If your service no longer works with the AWS SDK because you crash at `headers["content-md5"]` just because "it seemed a good way to make things more correct" - it is on you to fix it, IMO.
Like, this changeset https://github.com/minio/minio/pull/20855/files#diff-be83836...
Why does Minio mandate the presence of Content-MD5? Is it in the docs somewhere for the S3 "protocol"? No, it's not. It's someone wanting to "be extra correct with validating user input" and thus creating a subtle extra restriction on the interface they do not control.
https://aws.amazon.com/blogs/aws/introducing-default-data-in...