Hacker News Reader

Comment by scarejunba | Hacker News Reader

There's a famous 'rebuttal' post to this here https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...

So Uber's algorithm takes 58756 ns/op, and the fastest non-brutal algorithm took 471 ns/op. That's about 100 times faster.

They used 40 machines to serve New Year's Eve traffic. Perhaps with a better algorithm, they could have got away with one.

(Possibly not, because there's still a lot of HTTP and JSON munging work to be done, and the network card becomes a bottleneck at some point)

Another rebuttal which is less through due to the benchmark being very synthetic but still points out that Uber's numbers are nothing impressive.

https://www.cybertec-postgresql.com/en/beating-uber-with-a-p...

This is a crap article tbh They don't take any requierments and build a 5min poc with no constrains.

For instance their demo doesn't update anything, Uber updates location every seconds, I'd like to see how PG behaves when you rebuild index thousand time per seconds.

No, that was not a requirement. This is not the database for the locations of cars, but the database with human defined geofences which probably only update a couple of times per day. The database is used to check e.g. if a customer is currently at an airport or not. That is a very normal and simple GIS workload.

well basically ubers blog posts are really low quality and from the outside their engineering descisions are kinda vague.

like the switch from postgres to mysql.

I'm still clueless how you can have so much money and one of the biggest engineering team, but still can't correctly engineer your stuff. I mean, everybody makes wrong decisions or errors in production code.

From what I gathered off of several HN comments of Uber insiders, the size of their engineering team is actually more likely to be one of the causes for bad engineering decisions. They seem to have too many engineers running around for too little actual work to be done, which results in those engineers coming up with stuff to do to keep them busy (and of course to ensure they appear to be busy and worth their money to their superiors).

One of the best ways to create yourself some work is to not choose an already-proven path to a solution, but invent a new one just for the sake of inventing a new one. Of course that's not how this kind of doing is justified - the justification is usually "the proven path does not scale to our needs" or "by using a special approach adapted to our needs we can be more efficient" or "the proven path is too complex, we can get by with something simpler and easier to maintain". Which might actually all be proper justifications, it's just that you should have some hard proof for these statements, like benchmark results of a comparison of different approaches. That part often gets skipped, which is actually ironic, because doing extensive evaluation and benchmarking and implementing different approaches first before choosing one for production actually serves quite well to create even more work to do.

Also, it's resume-driven development. Five years later they want to be able to be the guy who can say, "Yeah, I was the original designer behind Kafka," but substituting in a brand-new Uber technology.

It might be sub-optimal for Uber but it's better for the industry as a whole to have many different implementations of the same type of solution.

And none of their blog posts about their own geo stuff ever come close to answer the why. What is wrong with e.g. PostGIS or Elasticsearch? Why do they need to reinvent geo databases several times?

Too much time and not enough things to do.

How does this happen?! They're so expensive

Give me twenty billion dollars to make a ride hailing app and I'll show you.

More seriously: they raised tens of billions of dollars to make a ride hailing app and research self-driving cars. Throwing more money at a problem doesn't solve it faster, but it does pay for hiring people, and headcount is seen as a proxy for doing stuff.

When a measure becomes a target, it ceases to be a good measure.

https://en.wikipedia.org/wiki/Goodhart%27s_law

Some big start-ups[0] solve this conundrum by investing in adjacent companies to find the solutions they're paid to find. Outsourcing! This still has limits because there's so much money flowing around right now. Tossing a few million at a company with hundreds still won't solve the problem faster.

[0] Start-up definition for this post: a company that has taken funding but hasn't yet found a sustainable business model. That's how Uber is still a start-up with more money than most companies make in decades.

Teams outside Infra aren't allowed persistent disk. Service authors get to pick from a small catalog of managed operational storage systems. Life would be a lot easier if a scalable Postgres were one of them, but it isn't. For the moment, the offerings with high scalability and good SLAs are all KV stores.

We do have ES but it's operated for log search, not production critical paths.

Thats a great write up. They should hire him on the spot

That'd be nice. For real though I did meet with them after, but it was basically the same as getting a referral so the same interview process which I chose to forgo.

Thank you for this :))

np ;)

Reply on news.ycombinator.com