Comment by nickstefan12 | Hacker News Reader

Comment by nickstefan12 | Hacker News Reader

Parent Full thread

> There are a number of tools that already help us do this - from Matlab to Quartz Composer - but Excel is unquestionably the king. Through Excel we can model any system that we can represent as numbers on a grid, which it turns out, is a lot of them. We have modeled everything from entire businesses to markets to family vacations. Millions of people are able to use spreadsheets to model aspects of their lives and it could be argued that, outside of the Internet, it's the single most important tool available to us on a computer. It gains this power by providing a simple and intuitive set of tools for shaping just one material: a grid of numbers. If we want to work with more than that, however, we have to code.

The idea that the entire average joe population needs more than excel has never sounded quite right to me. Anything closer to "real" programming and I think we're asking people to devote too much of their time to something that looks like software developing. I Just honesty think not everyone has the demeanor to enjoy programming. It's a very introverted, abstract, high focus, and high patience endevour.

Indeed, Excel is close to perfect for the purpose of building data abstractions in an intuitive way. The only part I find lacking is for building behavior abstractions which allowed users to build semi-automated workflows by changing views of data on the fly; to change presentation, Excel still requires you change languages and resort to imperative programming in Visual Basic, much like Hypercard.

A tool that combined Excel's functional data-flow programming with Hypercard's representation of state (as a stack of panels that change depending on the execution of small attached functions) would IMHO provide a general programming tool for end users.

Excel is easy to model in an intuitive way, but it end up being very poor models that come out of it, as it is basically a two dimensional grid. So the first two dimensions you are modelling will come out well, but once you need to start adding extra information in, then you have color coding, new fonts, and all sort of hacks to overcome the limitations of excel, when in reality you ought to be using a proper database by that stage.

That's right, and this is why a better tool is needed. In the end Excel can't support arbitrary levels of abstraction.

IMO this is not because the base model is a grid (after all, "proper databases" also follow that table model) but because you can't create new tables on the fly and treat them as new objects independent of their structure. Once you build a data type as fields laid out as columns, all the operations on that data (mainly filters and search) need to be operated by hand.

You can't properly express in Excel something like "there are three collections of employees, and I can create a '' function that works on any of the three". If you have separate tables with the same structure, you need to replicate the same operations on each independent table; the only way to build reusable code is with Visual Basic macros, which falls outside the spreadsheet declarative model.

But if you could create new functions with Excel formulas (like style handling or business processes) and apply them to any arbitrary table having the adequate structure, I think you could have robust abstractions within the spreadsheet context itself - from a theoretical level, the storage model of spreadsheets is no different from that of relational databases.

I cannot agree more.

Apple's stab at Excel with Calcs is the way Excel should always have been. A blank sheet to which you can add tables or charts, not a single grid. You only need to add an index by which you can query these tables (instead of VLOOKUPs), and you have something as simple and flexible as a spreadsheet and as powerful as a database.

Microsoft seem to be keen to observe and take inspiration in what is done outside, but somehow the office team seems to be an exception. Apart from re-shuffling around the buttons, I can hardly tell the difference between Excel 2000 and Excel 2013.

To be fair, Excel internals must be too brittle to allow for more extreme redesigns; they've been piling up layer upon layer of technological changes for three decades, and they need to keep it working without crashing too much.

I don't know Apple's Calcs, but Microsoft has built the PowerPivot extension that largely works as a modern, parallel version of Excel embedded within the same window. If you need to work on Excel with large datasets and/or complex relations, it may pay off to learn this extension.

The Visual Studio team overcame a similar problem with a complete rewrite. I think the problem is more about breaking the compatibility, many add-ins rely to a very old COM interface. But I think people wouldn't mind a breaking change if it comes with really new and powerful capabilities. In fact it would provide an incentive to upgrade. And it can run side by side with old-style spreadsheets that preserve compatibility.

The problem is not so much large datasets, for which there are better tools than Excel. It has more to do with doing more advanced thing conceptually.

For instance, what about adding a table to a sheet (Apple Calcs style), specifying a name and some inputs cells, and one output cell or range, and then being able to use that table as a function anywhere else in the workbook. I can already solve these types of problems with a UDF, but think about users who do not know how to code. Also, having the logic of a function in excel formula format makes it easily auditable by a non-programmer.

And also what about manipulating data in memory instead of having everything in big sheets and having to link every cell with each other, or having nested tables in a cell (or a reference to a table in memory or in another sheet).

And this is Excel only. Word and PowerPoint should really have a mark-up language side by side with the visual representation. That would make it so much easier to manage metadata (object names, linking a piece of text to an external source, working with templates, etc).

If you have time, could you take a look at this project I'm working on? https://vimeo.com/107069470

I would relish feedback.

I'll have to take a closer look at your video, but from a first glance I'd say you're firmly in the right direction.

Spreadsheets "declarative-reactive" execution model is a solid base for end-user development environments, and some features (like "attribute chaining", if it is what I think it is, and allowing users to build data structures as attribute collections on tables) match what I have in mind for my ideal "live modeling" programming environment.

Do you have a place where you document your design ideas (in addition to this video) and share the code of Kaya? On which language is it coded? I'd love to study how you've built the running environment.

Contact me: david 927 at gmail

Reply on news.ycombinator.com