I built Excel for Uber and they ditched it

This is so true. People in the US just don't understand the level of economic and industrial espionage that happens in China on a daily basis. I was responding to an unrelated breach at an unnamed tech company back in mid-2000s time frame and had a side bar conversation that went like the following:

Them: "Yeah, we just opened a tech center in Xinjiang and ... wow, we've had quite the rash of lost ID badges there recently"

Me: "Have you considered that they're not 'lost', but rather 'sold' for profit?"

... silence ...

I don't know if executives are aware but just don't care, or if they're simply incompetent, but China has productized industrial espionage on a massive scale. GE Aviation was a victim more recently: https://www.cincinnati.com/story/news/2022/11/16/accused-chi...

> Sometimes that’s just how it is. The devops saying “Cattle, not pets” is apt here: code (and by proxy, the products built with that code) is cattle. It does a job for you, and when that job is no longer useful, the code is ready to be retired. If you treat the code like a pet for sentimental reasons, you’re working in direct opposition to the interests of the business.

A lot of code is fun to write. A lot of problems are fun to solve. But a business, especially a startup, needs to stay razor focused. My entire career is effectively to sit in meetings and tell young, passionate engineers not to build things. It’s a bit depressing, but it’s also vital.

A good engineer can solve any problem with clever code. A great engineer knows what problems aren’t really problems and probably an XLS download link updated daily would have been fine.

My idea was to take this code and spruce it up for Uber’s use case."

"My first reaction was to publish the code on Github."

I’m very surprised by this, isn’t the code property of Box, or Uber? The author does not mention their authorisation before releasing it under MIT license.

I can't believe it either, and I don't mean this in a good way.

Apache POI lets you run headless Excel. You import and interact with sheets programmatically in Java. We used this in my old workplace for exactly the same reason (functions, cell references, the whole thing), it worked great.

You found the ‘circ’ problem with a bit of luck. What about all of the other hidden little quirks of Excel that you would ultimately run into down the road? Are you really going to build and maintain a full blown Excel clone in JS? Is this really the objective of the frontend team?

It seems to me like a bit of googling and >90% of the work here could have been avoided. As an added bonus it would have been done by the backend team instead.

They eventually built a homegrown "Excel" clone as the UI for their model because "city teams only know how to use Excel".

I would have done it the other way around - connected Excel to the data output by the model so the "city teams" could continue to use real excel. I think most finance teams do something like this.

> Unless you're familiar with iterative calculations, you probably won't want to keep any circular references intact. If you do, you can enable iterative calculations, but you need to determine how many times the formula should recalculate. When you turn on iterative calculations without changing the values for maximum iterations or maximum change, Excel stops calculating after 100 iterations, or after all values in the circular reference change by less than 0.001 between iterations, whichever comes first. However, you can control the maximum number of iterations and the amount of acceptable change.

I created a whole programming language as an intern for <defense megacorp>. It was lazily evaluated and garbage collected. Unquoted MAC addresses were valid syntax, among other application-specific oddities. No bytecode or JIT shenanigans - the interpreter just pushed and popped stuff from a stack as it traversed the parse tree, and that was fast enough for what we were doing with it. The interpreter was written in pure ANSI C, and Valgrind was very happy with it. Maybe it has been totally forgotten, or maybe it became critical to their technical infrastructure. That code never left the airgapped lab where I wrote it, so I have no way of knowing. 3 years ago, as a recent college grad, that was by far the coolest piece of "actually useful software" I had ever written. It's still high on the list. Sometimes I wonder whatever happened to it.

This spoke to me.

However, as anyone that has looked at my code can attest, I tend to also want my code (and its functionality) to be very pretty. I'm generally writing code that I will be maintaining, so it needs to be something that I can look at, in a year, and understand.

I'm currently in the final phases of a project that I will never announce here, and don't plan on taking much credit for, but it really is da schizz. It's that way, because no one is paying for it, and no one will make money from it.

Money both spoils everything, and also makes it all happen.

A director asking for an exact spreadsheet to be the UI would have been par for the course, especially during the Uber China days. Heck, I personally loaded FX prices into Vertica from a spreadsheet emailed every month to the team. That process remained for more than a year as there just wasn't enough bandwidth to invert the control as automated ingestion.

Thanks for digging up these memories, @bastawhiz. I'd love to see more. :)

I worked on all this at Uber back then, and this comment warmed my heart a bit. Thank you.

I will never do that again. We didn't build a whole spreadsheet engine, but we did build a web UI that simply doesn't do as much as Excel. Excel is powerful. Sometimes it's a fine tool for the job. Our team was laid off before we could roll it out, but I remember the growing sinking feeling of "I would hate using this if I already had a bunch of habits built up in Excel"

It would have been less fun but way, way less risky to wire a headless Excel up to a javascript front-end.

The question is, if you do some work, in your own time, on your own equipment, does your employer own it just because the employment contract says they do?

In California: if the work in any way relates to the employer's business, then yes, they own it. One way to guarantee that it relates to the employer's business is to bring it into the office and use it as part of your job. If your employer is Apple or Google or AWS or Microsoft, then probably anything you write would in some way relate to their business. Write spreadsheets by day, but games by night? All of those companies make games, or are in the games business.

I would love to hear a lawyer say, "Well, it doesn't matter what the employer does, it only matters what your job duties are, so writing games at night is fine if they don't pay you to work on games related things during they day." But I've never been told that by a lawyer, whether I paid them or otherwise.

Everywhere else in the USA: they probably own it. You could write software for washing machines, and write a video game, and if your contract says they own everything you write then they do. You signed it. There's no "but surely not!" defense.

> “Why can you see the formulas?”

> “You said to make it just like Excel.”

I can't keep up with the espionage story that followed but I had this conversation more than once in my professional career.

The first time I sat as a junior dev on a multi-month project replacing a excel spreadsheet for financial controlling of data centers that only one person who was going to retire understood with a web-based solution.

They were quite proud that they were going to get a "modern" solution.

Then they wanted me to make it like excel. What followed was evaluating every fricking excel JavaScript library out there at this time, going for one and started duct taping all the missing pieces.

They were pleased but the look was off. It wasn't excel. I slapped some styles on it coming quite close.

I was not prepared for what happened during the next presentation: They hated it because they wanted a modern web based solution (their words, not mine) and what they got was a poor excel knockoff running in their Internet Explorers. Tables are so 90s.

I remember the pain so vividly that I regard "just make it like excel" as some kind of forming meme for my career till today.

> "Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day."

...

> "Apparently that was a thing. I remember being only half-surprised at the time. I hadn’t considered that our threat model might include employees leaking the computations used to produce the numbers in question."

...

> "My first reaction was to publish the code on Github."

Perhaps the author feels "only half-surprised" due to their own disregard for corporate legal ownership of code. The hypocrisy is strong here.

The solution was not to recreate Excel in the browser, but ran the Excel file with its formulas and its data plus the input parameters from end users at the backend server. Apache POI was a nice Java library that could do everything on an Excel file. Once it finished the formula calculation, I just read the cells from the Excel file to extract the result data and generated the web pages to present the data and graphs.

One nice benefit was the analysts could update their work in the Excel file, uploaded it to the server, and got the new calculation reflected on the web pages right the way.

Like almost every spreadsheet before it: Lotus 1-2-3, Borland Quattro Pro, VP Planner ...

Spreadsheets iterating on circ references goes back to the 1980s.

The first spreadsheet application, VisiCalc, didn't track dependencies: it evaluated cells left to right, top to bottom, IIRC.

Microsoft had a product called Multiplan that competed with VisiCalc. Not sure if that did iteration.

I think it used to be a setting in some programs whether circular references are flagged as errors, or iterate. Maybe it's still that way in Excel?

You can't drive Excel from Python/Rust/etc (Microsoft just announced to great fanfare that Excel can call Python--which is the wrong way around). All the editable table widgets for the web seem to suck. Nobody seems to have a TUI which you can drive from an external program.

A poorly written spreadsheet with a driveable API seems like a component that has been built multiple times by lots of people yet seems to be unavailable.

Is there some solution that I'm missing?

This haphazard way of running compute jobs really stuck out to me. I can't imagine doing things this way (rather than having a central compute cluster running SLURM or similar) at a company bigger than, say, a dozen people - much less the scale of Uber. What's the rationale? Even if it's just a cluster of 3 or 4 machines in a rack shoved in the corner, isn't that better than ... laptops?

The lesson of what you build for a company as something that may be thrown away is a good one.

However, I believe the only way to actually have a sacred masterpiece is to own the company and make the company's mission to build that sacred masterpiece. For example, there are definitely business opportunity for converting excel sheets to online apps that are collaborative.

The core problem is alignment between the tech investment and the company's goal.

When I was 24, I was pretty much exactly this person. "Building a spreadsheet engine that runs in a browser. Sounds like so much fun!" And then I'd slog away at it for a month and get something working.

Now though, with age, I know I can't chug away coffee working late into the night. My back hurts when I have to sit too long. My wrists aren't being too kind to me these days either.

Now if someone asked me to build Excel I'd first laugh in their face. And if I don't see them smiling, I'd ask if they can afford any of these A,B,C...Z COTS products that are doing this that we can just buy. And if none of those work out, I'd look for how I can take something like ethercalc and repurpose it for our use case.

Looks like the older I get, rather than writing code, I seem to be getting more adept at how not to write it.

Box had a collaborative note-taking product called Box Notes (based on Hackpad).

Slight correction: Box Notes was forked off etherpad-lite. Hackpad was a parallel fork of Etherpad.

[context: I was lead engineer on Box Notes till around 2014/2015. I left before the events @bastawhiz described]

nice read. made me nostalgic for the wild days of Uber-China hacking.

2) You stole Box code, used it at Uber, and then stole Uber code and posted it on github. I understand no one's using the code or missing that code, but you really did steal that code. It belongs to the company, not to you. I would be careful not to do that in the future, because technically that's a trade secret and people have gone to jail over that, like that programmer at Goldman Sachs (wrongfully) and ironically Levandowski who took Google code and tried to use it at Uber.

This is the story of literally every single large project I've done under a tight deadline in my career:

- Some non-technical department (usually sales) promises a customer something very big very soon, with no regard to how much time might actually be needed to build it. Fulfilling this promise "on time" is now your problem, Engineer.

- Months are spent frantically building said Thing, working overtime, burning out engineers, and bastardizing the previously clean codebase in the rush.

- The deadline is met, and no customer uses Thing for at least several months. Or the deadline is not met and customer waits for Thing as long as it takes, because they're not going to blow up a big contract or integrate with your competitor instead of waiting another month. They have too much invested and the deadline wasn't that important anyway. The third possibility, as it was in OP's case, is that Thing gets thrown away in its entirety due to some fateful turning of the organizational wheels.

I swear, almost as if by some karmic law of the cosmos, literally every frantic deadline turns out to be irrelevant in the end, and you should have saved your sanity and gone home at 5 every day for the last 6 months.

If this is life or death, then the first reflex should be to find ways to make things simpler to ship. I understand that the requester in this case was a head of finance, but sh*t, if you want something, try to compromise once in a while.

This is a great story, but at some point as an engineer in a gig-selling company, I would have asked myself if my job was really to re-build excel.

Ha! I massively identify with this. 'The market-wide median is off by £3.07' is a much harder but to crack than the 'worse' report that 'the median is twice the max and the min is null'.

https://learn.microsoft.com/en-us/sharepoint/dev/general-dev...

W

T

F

> Uber Sheets

Hee hee, this name is funny on a number of levels :)

There are simply too many potential problems that could come up in this scenario and be the cause for catastrophic financial results in the end. And most of them wouldn't nessecarily be easily found via unit tests.

which of course leads to what some people are saying, did this have to be built? Sometimes I think our job is to optimize ourselves away.

Makes me wonder how many times this has been done haha.

This should be taught in classrooms.

[0]https://www.forbes.com/sites/ywang/2016/09/27/ghost-drivers-...

how many people there were at that point, @bastawhiz?

y’all seriously believe that the only valid reason for an engineer or a programmer to exist is to “create *bUsiNess~ value”? i’m shaking my head.

this blog post is full of ideological blindness.

Learning how the Excel model worked and then reimplementing it would have been a better example of 'getting better at using your skills to create business value'.

However, while the ExtJS table widget had been treated by product management as pretty immutable "this is what ExtJS gives us, we can customise the colours and wording and that's it", the idea that we could customise the table widget started something amongst one of the PMs. And so we would get a constant stream of feature requests for the table widget to add stuff and enhance it and soon we were significantly more featureful than the ExtJS widget. It's still, to this day, the most featureful table widget I've seen in a web app. Everything Excel had in terms of resizing tables, sticky columns, scrolling behaviours, sorting, filtering, searching, etc., all saved in your config so it was synced across all your devices, as well as all the performance goodies like recycled rendering etc. The constant stream of feature requests meant there were dev team years invested in this table widget.

As a more mature engineer looking back, it's clear that at some point this had stopped being about customer value and more about one PM's obsession with getting excel like functionality in a in-browser reporting tool, but at the time we just kept building those features.

Now at this time, our company had acquired a sort of competitor of ours. This competitor had what was effectively the same product, but in a different market. And so the first merger of the functionality was basically to reskin both applications so they would pretend to be tabs in a unified application, and change some terminology, etc. They actually did happen to have a pretty similar tech stack to us, so some newer components were available in both applications.

But it became clear that our users were not happy with two applications pretending to be one. They wanted to know why this other market was not accessible by, say, a dropdown in the configuration, and not an entire application which worked in different parts from subtly differently to entirely differently.

So the discussion became about building a ground up unified interface for both of them. Of course, this ignited the discussion of "which table component do we use?". On the one hand, the acquired team were looking at our table, with it's fifty billion options and single handedly accounting for half the page weight of the minified JS of our application and did not want something so bloated. On the other hand, our team were looking at their table widget which was effectively "for row in data, for column in row, print td" and dreading having to rebuild all these features for product management again.

Ultimately the conflict was resolved by choosing to use an open grid that had less features than ours but more than theirs and telling the PM in question that table features were going to be prioritised much less heavily from then on unless there was a real user need for it.