pentlander

Hi Tim, I sent you a message from the email in my profile about two weeks ago but haven’t heard back yet. I’m definitely interested!

I’m pretty sure that Lucene is exactly the same, the segments it creates are immutable and Elastic is what handles a “mutable” view of the data. Which makes sense because Tantivy is like Lucene, not ES.

https://lucene.apache.org/core/7_7_0/core/org/apache/lucene/...

Roc is trying to fill this space: https://www.roc-lang.org/

I started upgrading some of the Docker images from Java 8 to Java 16 at work without touching the compiler. The changes I had to make to the code were fairly minimal, bump the version on a couple libraries and change the JVM flags on the Helm chart. The title is a bit clickbaity, but I agree with the premise for containerized workloads because the difference is huge.

The faster startup is nice, but the biggest advantage is in the memory usage. G1 past Java 8 actually releases memory back to the OS instead of keeping it forever. In addition, I've seen applications use 1/7th of the heap space after upgrading. This translates to being able to significantly lower the memory limits for applications without worrying about OOMing. I haven't moved enough services to see the difference in CPU, but it looks promising there as well.

Some quick feedback on the mobile layout, there isn't a lot of the actual blog post text visible due to the margins. I would suggest allowing the url and text to span the full width of the page. Also I would increase the margin between the title and the icon and star button, and potentially remove the word "star" to give more breathing room. It might also be nice to make the sidebar buttons an overlay to give more width as well.

Is it PG? If so, I was an intern at the time and you were gone once I joined full time.

As another ex-Glacier dev, I disagree with it not being "all that hard" but I agree with everything else. Now I'm curious who this is :)

Though I no longer work for Amazon, I'm reasonably certain they use it from the description. Especially given I know for a fact that other more foundational services use it.

Why is it a "relic of years gone by"? Consul uses a similar, though more advanced technique[0]. Consul may not be as widely used as etcd, but I don't think most would consider it a relic.

[0] https://www.consul.io/docs/architecture/gossip

Yeah Amazon still runs on Coral, there were some recent (release a few years ago) advances on it under the hood and ergonomically. I think the "replacement" for it is Smithy[0] though it will likely just replace the XML schema and codegen and not the protocol. Honestly at this point I think it would be in Amazon's best interest to heavily invest in Java Project Loom rather than trying to convert to async.

[0] https://awslabs.github.io/smithy/

The hand rolled gossip protocol (DFDD) is not used for consensus, it's just used for cluster membership and health information. It's used by pretty much every foundational AWS service. There's a separate internal service for consensus that uses paxos.

The thread per frontend member definitely sounds like a problematic early design choice. It wouldn't be the first time I heard of an AWS issue due to "too many threads". Unlike gRPC, the internal RPC framework defaults to a thread per request rather than an async model. The async way was pretty painful and error prone.

Ah right I misspoke. I was conflating it with the common case, which is that one of the deps for versionset's target was changed. I agree with your other comment, the first year or so I just cargo culted Brazil and versionset changes until I actually had to fix more complicated build issues (e.g. Packaging up a third party lib)

Not the OP but I am currently working at a company moving towards a Bazel monorepo and I previously worked at Amazon. For me the lightbulb of "this is a great system" didn't click until I actually left Amazon.

Compared to Gradle, Brazil is great because it lets you avoid thinking about a lot of smaller details. You declare a dependency on major version in the Config file and it'll grab the latest minor version from your versionset. A versionset is basically just a giant package-lock.json or cargo.lock file for all of your deps that is constantly updated in the CI/CD system. Since artifacts are deployed at a versionset level, you can always look up what specific version of a package is deployed anywhere. This means you don't have to think about minor versions 90% of the time, but when something breaks you can easily find it since the versionset interface is connected to the code browser. You can also easily ask "where does this package version exist" and see all the versionsets that are using a version of a package.

I think one of the other big things I miss about it is how well the tooling worked with multiple packages. Brazil had a concept of a "workspace". Normally if you're working with a package it pulls the deps from S3 or wherever. But if you wanted to work with multiple packages that depend on each other, you would run "brazil ws add package-foo-1.0" and it would clone that package to your workspace. Any other package in your workspace that depends on "package-foo-1.0" would now understand to use the local copy to build instead of pulling it remotely. This worked fairly seamlessly with the the Intellij Brazil plugin, making cross package refactoring pretty easy. Doing the same with gradle or npm requires manual work.

One of the biggest ways that brazil was misused was around handling of major versions. For context, only a single major version of a package is allowed to exist in a versionset at a time. If you tried to merge in a different major version of a package into your versionset, your pipeline would fail to build due to "Major version conflicts". One of the biggest sins was around bumping the major versions of the dependencies in a library without bumping the major version of that library at the same time. This would lead to many broken pipelines. Let's say you have a library Foo-1.0 with a bunch of users on other teams. You decide to bump up the Guava version from 25 to 29 and publish the new version of Foo-1.0. Anyone consuming Foo-1.0 would automatically pick up the new version of that lib because it's just a minor version change, however the merge would fail with a "major version conflict" because the major version of Guava they're using in their versionset is still 25. This means you would either have to pin that library back at a previous version, or bump your dependency on Guava in all of you packages to 29.

I think this last point really highlights the big difference between Bazel and Brazil. Bazel makes bumping versions a pain because you have to upgrade everything at the same time. However it also ensures that if there's a security issue with a lib, everyone is forced to upgrade at once. Brazil allows teams to adopt newer versions at their own paces, however you need a more complex CI/CD system, you have to deal with major version conflicts, and you have to deal with longer campaigns to upgrade libs with security issues. I think the two systems just have different tradeoffs, though the biggest advantage Bazel has is that it doesn't require the tight integration with a CI/CD system so it's easier to open source and operate.

I don't think that's true, internally there was a long term plan to bring cloudwatch up to par with internal metrics and alarming systems. At the time when I left Amazon, there were some shims to use internal dashboards and alarms with cloudwatch data. The eventual goal was for the whole company to move to cloudwatch instead of maintaining two systems.

Probably because that C code is the embedded Lua interpreter. See the lua.h file, there's the copyright notice at the bottom.

There's also a lightweight menu bar for X11 systems called bar. https://github.com/LemonBoy/bar

I agree, a lot of people tout how functional programming is easier to both read and write, but this really puts things in perspective. I especially like how he explains how the different standard functional functions work in very simple terms and immediately shows the advantages of using them.