Comment by gruseom | Hacker News Reader

Comment by gruseom | Hacker News Reader

Parent Full thread

IMHO engineers find it difficult to identify and extract the common processes because every part is slightly different and complex and usually requires its own judgement calls to be made about how to deal with it. This leads naturally to everyone doing things their own way and therefore using the most flexible and cheap tool there is...Excel.

This touches on one of my pet theories. Spreadsheets were remarkably successful because they never require you to express a model abstractly (i.e. they never require you to write the formulas up front and plug in data later - which is the way that programming languages work). You're always dealing with concrete numbers. It's example-driven. I believe that this is one of the reasons why the spreadsheet is the computing tool of choice for non-programmers. Good programmers "get" abstraction; most non-programmers don't.

The trouble is that when you need to make many variations on the same theme ("every part is slightly different"), the only facility provided by the spreadsheet is copy/paste. Now you can modify Examples 1 thru N to your heart's content, but you've lost any link between the stuff that's common to them all. There's lots of research showing that this kind of spreadsheet development leads to billions of dollars in errors every year. Basically, imagine programming where you have no subroutines and nothing but copy/paste. We know what a mess that would be, the minute you tried to do anything complex.

I believe an improvement would be to allow a spreadsheet to clone ranges from another spreadsheet, thus inheriting their data and formulas, but preserving the linkage back to the original. You can override things in the clone to make the parts different that need to be different. But anything that you haven't overridden (or added yourself) is a live copy of the thing you cloned. That means if you change the master, the copies get updated. In fact, you haven't actually copied anything - you're just referring back. The system would be better able to trace the true dependencies of a sheet, and users would still be able to base new work on old examples.

Since you've struggled with these issues, I'd like to hear your thoughts.

This touches on one of my pet theories. Spreadsheets were remarkably successful because they never require you to express a model abstractly (i.e. they never require you to write the formulas up front and plug in data later - which is the way that programming languages work).

My pet theory is that people have a love of grids and graph paper -- I remember in grade school how graph paper was, for some reason, a rare thing, and it was treasured. The kid with a whole stack of graph paper and who DREW PICTURES OF TANKS AND DRAGONS ON IT was considered some kind of blasphemous rouge, but still respected because of access to graph paper. It brought some indication of order to the otherwise sloppy handwriting on looseleaf we had to do.

And that love of the grid translated into a love of spreadsheets. It's so neat and clean, ready to be filled up with MY organized madness! All those empty boxes, what fun! Look, I made a list of items! And it's arranged in neat rows and columns! And besides, who needs to be able to join these product ids with sales numbers? We can do that with a hundred clicks or so and create a pivot table!

Everything begins to look like a nail when your hammer of choice is Excel.

Now I am going to have to say that the spreadsheet UI is hugely valuable for three reasons: it fits the way non-programmers' minds work, it's familiar, and... it feels like graph paper!

Considering the kinds of things I've seen people put in spreadsheets, and the way they format them, I'm sure I could dig up a few examples of spreadsheets NOT fitting the way non-programmer's minds work, because they try to make it do things it wasn't meant to and it ends up being MORE work.

I don't doubt that such examples exist, but I'm adamant on this point: the spreadsheet is the invention that gave non-programmers the power to define their own computations. That's why it was so successful. It's the only "programming for non-programmers" invention I know of, of the zillions that have been tried, that doesn't suck. Can you name any others? The ones that come to my mind are the laughable boxes-and-lines "visual programming" gimmicks.

Oh, I agree with that. But I think the perception of (mock) organization it provides and its visual similarity to graph paper is what draws people to it. The power of spreadsheets are in the computations, and of course people use it for that, but I bet there are just as many spreadsheets that contain entirely static content as there exist spreadsheets that actually use spreadsheets as more than just a glorified grid formatting engine. I've seen people do calculations with desk calculators and then type the results into their spreadsheets. It does let non-programmers define their own computations, but I wouldn't call what non-programmers do with it to be programming, "computation for non-programmers", most definitely. I believe this may come down to how the both of us define programming, however.

Interesting contrast, though, is the relative avoidance of accounting ledgers, which spreadsheets are a closer cousin to than they are to graph paper.

My pet theory is that if people weren't able to to spend time setting up complex and impressive looking spreadsheets they would be bored out of their minds.

To address your idea specifically, I definitely think something like that is possible. If the formulas and data where separated such that one spreadsheet could inherit formulas from a master etc. There is so much variation in engineering though its just a matter of working it through and seeing where the limitations are.

In making things better in engineering the first goal is first try and stop people spending so much time managing and moving data around. This really takes up a large percentage of the work done, unnecessarily.

Second is to start getting engineers to see things more like programming where things aren't repeated and data is separated out from the method. If you receive new data (e.g. new aircraft loads and geometry) your goal is for there to be nothing whatsoever required for you to update your analysis. Its already setup, you just point at the new data and it's done. This is rarely the case I find.

Finally, I think is the issue you are talking about which is to tackle the problem of engineering by forming new abstractions and figuring out how to best represent those abstractions with existing tools or something new. This is definitely promising but honestly there's lots of easier stuff to do first that would make a big difference.

The thing that makes it so hard in engineering is that every job has its own set of "approved" methods and the amount of work required for each part varies for a number of reasons. There's a lot stuff you're never going to get the world to agree on but a spreadsheet type thing that separates the data and formulas completely is a good start.

Reply on news.ycombinator.com