These open-sourcings seem a bit like PR pieces with no guarantees of any support or evolution after being published.
Uber also uses M3DB extensively internally and the project is nowhere near being abandoned or on life support: https://github.com/m3db/m3/commits/master
A pretty common story these days:
1. I built an X to solve Unicorn U’s problem.
2. Open source it, give talks on it
3. Leave Unicorn U and start a company based on X
4. ...
5. Profit(???)
Also, debatable whether or not Unicorn U actually needed a freshly built X instead of using existing tools/tech.
What is the actual gap that is not addressed with one or all of the following?
- Prometheus / Grafana [1]
- Datadog [2]
- Cloudwatch [3]
1. https://docs.docker.com/config/thirdparty/prometheus/
2. https://www.datadoghq.com/blog/introducing-live-container-mo...
3. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...
Datadog gets very expensive beyond a certain point; much bigger than most startups, but much smaller than Uber.
Cloudwatch is usually used as a source of metrics rather than a destination. It doesn’t have as rich a data model as Prometheus for custom metrics, and has a lot of quite limiting restrictions (like only 10 tags per metric).
Unless Uber is actively blocking contributions, it's not Uber's fault if no community formed around something they opensourced.
As for this being a PR piece, they could have achieved the same with just a detailed blog post and no code. It looks like a expensive PR piece if they have to opensource work that took probably hundreds of development hours.
But without any certainty around the roadmap, support, and longterm commitment by Uber to maintain these projects, they're nothing more than interesting repos amongst a sea of interesting repos.
The way Uber brands them suggests that they're suitable for use in production environments, but so far that hasn't been the case with anything they open sourced outside a narrow envelope that resembles their own operating model. Maybe this project will set a new trend, but so far nothing they put out gained any traction or became suitable for general purpose production use. In that regard, H3 and their other projects have remained at the level of decent 'show HN' pieces rather than something you'd ever use professionally. In other words, marketing.
Based on previous news coming out e.g. https://news.ycombinator.com/item?id=20931644
It seems like Uber had too big of an engineering department with too little work to do, so they started reinventing wheels. Which is cool if they're willing to support them in the long term, but so far that hasn't proven to be the case.
Protects that don't do that are therefore unlikely to remain interesting for long.
Former Uber engineer here. I can assure you that while our engineering team was massive, there was anything but too little work. If anything most engineers were massively overtaxed. Whether or not the work we were undertaking was meritious and valuable is an entire branch of philosophy I'm pretty sure.
Part of the struggle at big companies is that a lot of existing solutions just don't work. Let me use an example with chat. A few years ago Slack was evaluated as a replacement for HipChat, since Atlassian's outages had finally started affecting us during our own outages.
Everybody wanted to go to Slack, but the cost of Slack was tremendously prohibitive and the state of the service then (as I was told) was such that it could not support a company of Uber's size. Tremendous effort would have been undertaken by Slack to support Uber and they didn't want to expend that effort for a single customer. This was late 2015 early 2016.
There were tons of options, but ultimately an in-house chat software was created. At the time it seemed required to make our own highly reliable chat, considering how distributed engineering teams were. I think if you talk to anybody without the background of how chat evolved at Uber they would think the in-house chat project would have been a boondoggle.
Not all over-scoped engineering projects are actually so noble. There was certainly a ton of "reinventing wheels" going on. There was significantly more "these problems are really hard and I only have bad solutions."
Though if the result is ultimately, "nothing more than interesting repos amongst a sea of interesting repos" sign me up.
Are you affiliated with Uber?
You drank too much kool-aid. uChat was just a reskinned Mattermost.
I have heard that the team that put it together actually tried to hide that fact from the company (for the glory, I guess). But that could be apocryphal.
Mattermost didn't work out of the box, and certainly not the way and at the scale Uber needed it to. I'm not overly familiar with the technical details, but one thing in particular stands out as an example. There was a Town Hall channel that every user had to be a member of. This unfortunately did not scale, and not enough ACLs were available to limit all the ways users could use this universal room. Eventually they really fixed the problem, but it was a tremendous pain point for a long time. There were a lot of fundamentally "less than great" things about Mattermost that had to get updated to work for Uber.
There was the amusing time employees found out anybody could change the topic in the room, even if our chat permissions had been disabled. It was absolute chaos for at least an hour, I can't remember if it actually negatively impacted the deployment though it sounds vaguely familiar.
It's pretty telling of employees that badmouth the uChat team. That team ultimately was trying to do what they thought was best for the company, even if at the time it seemed like they bit off more than they could chew. There was no other engineering team so directly visible and exposed to the entire company internally like they were. People dismissive of their efforts are generally not used to the difficulty of making so many very vocal customers happy all at the same time, and could be more sympathetic.
https://news.ycombinator.com/item?id=19101617
The fact that uChat is commonly mentioned by Uber employees as being the "custom chat solution we built in-house". It leads me to believe that comment that the team tried to hide that it was built on open source.
I don't doubt for a second that scaling Mattermost for a huge organization like Uber was a big undertaking. But it seems disingenuous for people to always mention that Uber built uChat when it should be more like "Uber put a lot of work into Mattermost to scale it up."
To be honest, I don't much like Slack because I feel the desktop app doesn't feel like a real macOS app. And I don't use all these features. So in the end, IRC would be fine for me. But it wouldn't for the rest of the company.
Not a contradiction. Many of these tools are suitable for use in production, almost by definition, since they are being used in production, at Uber. They might or might not work in your environment out of the box. But they are certainly often likely a better starting point than an empty editor, even when they do not. Most of the ones I am familiar with, are happy to get PRs generalizing them to more varied environments.
> they're nothing more than interesting repos amongst a sea of interesting repos.
As someone who has open-sourced on GitHub: research prototypes hacked together for a research paper deadline in grad school, class projects, for-fun hacks, and also production tooling I built as a paid engineer, I'd say there is a big difference! :) And there would still be a big difference even if the later were somehow never touched again after the first "we are open-sourcing this!" commit.
That said, we do try to maintain the things we open-source. Standards of support vary because individuals maintaining these projects, and their situations, vary. This is true for non-OSS internal tools too. In my experience, having gone through the Uber OSS process twice, and having started it a third time and decided against releasing (yet?), Uber does try to make reasonably sure that it's open-sourcing stuff that will be useful and is planned to be maintained. At the same time, they have to balance it with making it easy to open-source tools, otherwise too many useful things would remain internal only.
Also, note, some of these tools have exactly one developer internally as the maintainer, and not even as their full time job. For example, I am the sole internal maintainer[1] for https://github.com/uber/NullAway and also have 3-4 other projects internally on my plate, most of which are in earlier stages and need more frequent attention[2]. If and when said developer leaves, effort is made to find a new owner. This is not always successful, particularly if the tool has become non-critical internally. Sometimes, leaving owners retain admin rights on the repos and keep working on the tool (Manu, NullAway's original author, co-maintains it), but I don't think anyone is suggesting that that should be an obligation.
Finally, obviously, nothing here is the official Uber position on anything, just my own personal observations. This doesn't represent my employer, and so on. I am also pretty sure most of this is not even Uber specific :)
[1] Not the only internal contributor! Also, there is one external maintainer, as mentioned a few sentences later. But in terms of this being anyone's actual responsibility...
[2] Just to clarify, I think between Manu's interest, my own, and it being relatively critical tooling at Uber, NullAway is pretty well maintained. But I can understand why that isn't always a given for all projects.
First, "guarantee" is the wrong word to take too literally here. Depending on how you want to look at it, there are no guarantees, even with guarantees.
But looked at more loosely, answering that is really Uber's responsibility. Why did they release it? If it is just a PR release, fire-and-forget works fine for that.
If they want to see wider adoption outside of their firm, there are some fairly obvious things they should do to foster that. Sometimes you release just the right thing at just the right time and everyone else does your evangelism and support work for you, but it is much more normal for your next great thing to take a while to build a user base.
Based on your specific complaints, it sounds like your opinion doesn't matter in this case; you're not the audience. You want support and a predictable future: you're looking for a product, not for technology. This isn't a product, and it's not a platform.
If instead you represented another company looking into solving this same problem yourself, and are looking at starting points, then you're the perfect audience. In that case, you'd have time and motivation to contact the developers directly rather than gripe on HN. You'd be less interested in whether there was an organized community, and more interested in how to directly influence the roadmap. You'd care about what the code looks like, how they solved Problem X and Problem Y, that kind of thing.
It looks to me like maybe their engineers internally are fans of the idea of "open source", and the PR department is happy to try and get some good press out of it, but the company culture isn't really set up to develop in public or maintain these things they've nominally "released".
Sadly, this isn't unusual among tech companies, but it'd be more obvious what's happening if they just put up a bare-bones FTP with a README: "Here's some code under <LICENSE>. Use it at your own risk."
A company should first put their own system through hell and decide that "yeah, this is good, we are sticking with this", before luring people to use it.
But they are probably in the business of selling themselves as a software/ tech company. People value tech companies, so you'd better be one, even if you do taxi services, produce and distribute tv series, or rent office space.
Beringei, a TSDB, (https://github.com/facebookarchive/beringei) in particular with what you are saying was a PR piece (https://engineering.fb.com/core-data/beringei-a-high-perform...) since it was never really used by anyone outside of FB.
I really don't see the negative part of free code that you can learn from and/or incorporate at all.
When you rely on the systems themselves, and do so with expectation of support from the originating company, your expectations will almost certainly be broken.
I think that's the simplest takeaway - not to run away from any open-sourced project, but to take into proper consideration if/how they plan on supporting the tool, and how much you would be capable of adapting and owning yourself if the worst happened.
Chronosphere is the SaaS part for M3DB in this case. The negativity around someone open sourcing code for PR is nuts especially when all of the code is available. I love reading the code and getting ideas about how things work.
The test for open source is if it keeps getting maintained and supported for years. That only happens when the project is a core business effort, has some direct means of support (e.g. dual licensing or SaaS), or happens to be one of the few genuinely volunteer driven large scale open source projects.
We're also basically done with a new Python wrapper written in Cython. https://github.com/uber/h3-py/tree/cython
We could probably use some help with the last step of packaging, if anyone is interested!
Open sourcing something is naturally more expensive than not. It’s seldom the case that impact to the community triggers contributions that outweigh that cost.
The fallacy we hold is that companies will prop up software that is open source for everyone to use despite the lacking community contribution.
We as engineers should push ourselves to contribute when we find issues - rather than simply create tickets that represent work were want to have done for free. This is how open source software dies.
There is a minority that does this.
I get your frustration, but everyone should remember there are never any promises of support with open source software, regardless of how well supported it is at a particular time.