Hacker News Reader

Comment by warp_factor | Hacker News Reader

I think you are right.

I think that what is going there is also a bit political. They started to grow their Engineering department so fast that they need to justify the headcounts now. So each team is trying to invent new projects all the time. Anecdotally, this was partially confirmed to me by a friend working there.

I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.

View on HN

>I said this before, but I still cannot understand why a service like Uber need so many engineers in the backend (multiple thousands). It is a complex distributed application, but nowhere near the scale or complexity of a Facebook or Google.

Thank you so much, I thought I was going crazy. I understand the demands of running a service on the level Uber has, but well, for instance I can't imagine what kind of computational workload / infrastructure requirements would make developing your own resource scheduler a reasonable option - for a Taxi app? With non-essential (to the core product) machine learning?

Forgive me if I'm ignorant, but what exactly does Uber engineering team do?

edit: On their blog I was able to find that they namely "forecast rider demand", from a relatively small [0] article - that is, comapred to the article [1] about what essentially is "just" data visualization, which doesn't help my confusion much.

0 - https://eng.uber.com/neural-networks/ 1 - https://eng.uber.com/maze/

Makes it possible for me to get a ride, process payments and refunds even when the data centers are having issues or when there are temporary internet problems.

What does the facebook engineering team do?

Sure but that doesn't require thousands of backend engineers unless they are reinventing everything... and I'd be left wondering what they are rebuilding since, over the past year, every one of my trips with Uber has been a bar-lowering experience...

To be fair, operations becomes a much bigger deal as you get bigger. It’s not just the app, it’s having your infrastructure not fall over (because 0.1% failure rate means losing a lot of money)

Think of all the random bugs you’ve seen in your job and told yourself “eh, this would take someone 2 weeks to fix and is almost never hit by customers”

I periodically see the same content on my FB feed because well, it is NBD if I see the same update from my friend several times.

Let me assure you, it is a BFD if I get billed twice for the same trip. So I am pretty sure Uber needs quite a few engineers to make sure that their stuff works correctly every time in every market for every customer.

I think one of the bigger challenge is that when you become bigger you launch more projects to handle the scale and each of those projects introduce new bugs for which you need new team of engineers.

Basically, once you hit scaling I think you end up with a super small team that managed to keep simple (instagram for example) or you end up with a huge team that explodes in complexity and needs to grow exponentially to handle all the extra complexity. Uber is very obviously in the later bucket.

Those cars generate a lot of sensor data. (Tb per drive?). Id imagine that data needs to be made actionable and seperated into training and simulation sets pretty quickly. Mapping is a massive problem to automate.

Oh right, the self-driving cars. Well that's starting to make sense now.

I appreciate their efforts on open-source projects. Jaeger is wonderful and the effort they put into both making something great, and supporting the open standards (Opentracing and the legacy Zipkin propagation) is greatly appreciated. I recently had the need to write a service in Typescript (most everything else is Go), and I felt very at home using the Jaeger node bindings. It felt like I wasn't losing any features for using a less-popular language and everything just worked.

Sure, they just reinvented Dapper from Google... but unlike Dapper I can download and use Jaeger. That counts for a lot. Do I use their ride sharing service? Nope. But I do like their open source projects.

Maybe not at the same scale but imo their core system has a lot of tech challenges:

* Lots of realtime

* Resource scheduling problems

* Route optimization problems (especially with shared uber or shared lyft rides)

ok, but:

* Everything they do is low data (no video, image or anything high bandwith).

* Their whole model can be subdivided into smaller local problems (all users//drivers in the bay area have nothing to do with the users//drivers currently in NYC).

yes there is a couple of algorithms to develop for Uber Pool, and for the real time matching but everything else looks like a fairly simple app backend to me.

> Everything they do is low data (no video, image or anything high bandwith).*

So is everyone else? Storage and CDN isn't nearly as complex as ad serving on Facebook (Which is like Uber's matching - it's a realtime marketplace). Ad serving takes up relatively little bits.

I'm not a network engineer so I'm unfamiliar with how problems scale by bandwidth. I do know that solving NP hard problems is difficult, so I respect Uber engineers for that at least.

> need to justify the headcounts

Something similar happened in LinkedIn too I guess. Multiple teams building very similar tools that were on very related problems.

Reply on news.ycombinator.com