147 karma · joined May 20, 2011
On an unrelated note, your work has inspired most of my career in Solaris/Illumos/Linux systems and honestly this project likely wouldn't have happened if it wasn't for all of your books/blogs/projects to help me along the way. Thank you!
The long answer is that the offsets are the byte alignment offsets for the go structs containing the pointers to the file descriptor and buffers. Fortunately we only have to calculate these for each version where the TLS structs within go actually change, so not even for every version. For instance, if a field is added, removed, or changes type then the location in memory where those pointers will be found changes. We can then calculate the actual offset at runtime where we know which architecture (amd64, arm64, etc) with a simple calculation. Within the eBPF probe, when the function is called, it uses pointer arithmetic to extract the location of the file descriptor and buffer directly.
You can customize config and/or integrate with existing observability pipelines, but initially you just need to turn it on for it to work. No app instrumentation required.
We do this by scanning every version of Go that is released to find offsets in the standard library that won't change. Then when we detect a new Go process, we use an ELF scanner to find some function offsets and hook into those with uprobes. Using both of these, we have all the information we need to see Go pre-encryption content as well as attribute it to connections and processes.
If you're reading this list of comparisons out of genuine interest, I would suggest also looking into Nanobox: https://nanobox.io
1- Explicitly define your app's environment with the Boxfile (https://docs.nanobox.io/getting-started/boxfile/)
2- Write your own engine that will actually setup the environment (https://desktop.nanobox.io/engine-dev/).
The erlang version has been running successfully for about 2 years and we haven't had any issues whatsoever (excepting a wrestling match with mnesia early on). We are erlang/elixir advocates and have used the erlang vm successfully on highly critical multi-million-concurrency services for over 6 years.
When we set out to build nanobox desktop (https://desktop.nanobox.io) our vision was to provide a single pre-compiled executable that could be run without any configuration. The design required a push layer and needed the same functionality that the erlang project was already providing. It wasn't feasible to package the erlang application into the nanobox binary, so we emulated the original project into a consumable golang package. As time went on we were porting more and more of the features into the golang port until all that was lacking was distribution and authentication. At that point we made the decision to consolidate our efforts into a single project, that could be a standalone service or composed within a golang binary.
I regret to inform you that there isn't a mass exodus within our company to ditch erlang for golang. While that certainly would make for a fun thread, in this case it was simply a matter of fit and effort consolidation.
Manatee didn't work for our use cases, which should not be interpreted as "manatee is bad". Here are a few of the reasons we forged a different solution:
- While node.js is a great language for many things, we didn't want the overhead of running the node.js/v8 runtime alongside postgres. 50M might seem negligible, but with thousands of postgres clusters for our clients every MB adds up.
- We didn't want the administrative overhead of managing a zookeeper cluster, and instead built the cluster management semantics directly into the yoke project.
- This may be different now, but at the time manatee was very heavily integrated into illumos/smartos. We really love smartos, but recognize that not all clients are able to use this tech.