Hacker News Reader

> ...and, in fact, that's also what you'd get if you taught everyone how to program Python in high school instead of Excel.

I disagree. Python leaves a readable instruction set, while Excel buries the logic in an opaque two dimensional grid of cells (plus or minus atrocious VBA). I think this gives a better ladder for people to improve their skills to the point of not producing unmaintainable garbage.

> Python leaves a readable instruction set

Not everyone structures programs in the same way, or puts functions in easily found locations, or names things usefully, or even approaches the problem in a way that makes sense to the majority of other people. For a concrete example, see this[1].

> Excel buries the logic in an opaque two dimensional grid of cells

It would be trivial to represent excel grid formulas as a set of predefined functions, or a special data structure that you register functions in, and then it could be in textual form. It wouldn't make it any easier to reason about, but then again control flow isn't always easy to reason about when a program is split into many different objects and functions, until you've internalized a fair amount of the program. Spreadsheets give people a very limited, but structured output and a very limited, but easily reasoned about way or processing. It's different than "normal" programming, but that difference allows for very intuitive use.

> I think this gives a better ladder for people to improve their skills to the point of not producing unmaintainable garbage.

How much work has been put into making spreadsheets easier to share with other people? I'm not sure the same culture has existed for spreadsheets as for programming, where people have needed to share and reason about complex algorithms and data structures. If it had, I'm sure we'd have much more in the way of how to structure your excel well so it can easily be consumed by others, or yourself in the future (if it doesn't already exist, which it may).

1: http://thedailywtf.com/articles/Python-Charmer

The grid of cells is your memory space. Coders like looking at a stack. Spreadsheet users like looking at the registers.

Which you prefer depends to some extent on which end of the problem you're coming from; if you're starting out with a lot of domain knowledge, a spreadsheet is attractive because you can look at the data, see whether the way it's being stored/processed is basically correct by spot checking it, and then automate. I'll still often reach for a spreadsheet if I don't know off the top of my head what the rules or relationships are and I need the equivalent of a whiteboard with logic.

If you have a top-down perspective and you want to automate a process whose fundamentals you know well, code is more comfortable. And you can go back and expand, modify, or adjust it later. But two things I don't like about code for handling large volumes of data are that I have to construct some sort of viewer to format the output (requiring more coding, or slower bug discovery if I press ahead without), and endless typing I like selecting things from menus and tool palettes because while I don't mind holding an instruction set in my head I don't like doing the same for every library I might want to use, especially remembering all the parameters. I hate hate hate writing glue code and want the computer to do that stuff for me.

Fortunately a lot of these problems seem to be receding thanks to tools like Jupyter and some of the promising Flow-based programming tools that let you easily visualize structure, code, and visualization without getting bogged down in details.

I should add that I prefer hacking to software engineering; that is, I'm only interested in programming as a means to an end of having a customizable tool for myself rather than building a product for other people, so I want spend as little of my time as possible on constructing the tool.

> Excel buries the logic in an opaque two dimensional grid of cells (plus or minus atrocious VBA)

Yes, mixing the good old Excel dependency tree together with VBA macros creates truly dreadful and opaque spreadsheets.

...and, in fact, that's also what you'd get if you taught everyone how to program Python in high school instead of Excel. You'd get tens of millions of non-interoperable messes of programs (which still get the job done!).

That's exactly what happened with Perl when everyone learned to use, modify, and develop CGI scripts in the early days of web-based application development. Dynamically-generated website content worked and evolved into the web we know today, but those scripts were awful and did a lot of harm to Perl's reputation.

I personally think that a lot of notoriety from Perl stemmed from insane overuse of regular expressions. I'm not talking simple and concise expressions doing ordinary useful things, but rather insanely long and intricate ones which one had to be at a god-expert-level on regex just to attempt to understand.

These kinds of expressions are what ultimately caused the derisive terming of Perl as "line noise". It was really unfair, to say the least, because there were plenty of ways to code in Perl and do so cleanly, with a minimal usage of regex.

Honestly, it seemed more like people were coding in regular expressions, and simply using Perl as a "run-time" wrapper for those expressions (and maybe those expressions existed first; I don't know). I'm not the greatest fan of regular expressions (to me - especially on more complex operations - it is almost like programming the tape of a Turing machine), but I do know they have their place in the world. However, because the "language" (if you will) of regex allows for an extreme number of operations and such, it almost succumbs to an "inner platform effect", allowing those with familiarity to roam free, to the detriment of future maintainers.

That said - none of this is new. If you go back to the beginning of microcomputing, you'll find the same thing happened with the various dialects of BASIC, as people self-taught themselves programming in that language, then spread the knowledge thru various means (magazines, books, and later bbs systems) - it became a horrible mess, even though it all pretty much "worked".

Regular expressions can get out of hand, but that's a developers (bad) choice. Perl lets you use multi-line regular expressions that ignore whitespace and allow end-of-line comments, which lets you properly document longer expressions if you need to use them.

Regular expressions are really powerful, and are so much better than the typical non-regex string operations most languages support. Combine that power with the ~= and qr operators, which make regexes as integral to the language syntax as == and "", and you get why Perl is so good at textual searching, parsing, and manipulation. It's easily what I miss most about Perl development since my career has moved to primarily C# and Javascript.

> I personally think that a lot of notoriety from Perl stemmed from insane overuse of regular expressions.

The problem was that Perl's performance on builtins(sorting and regular expressions in particular) was stunningly good while its performance on anything else was stunningly bad.

The idea that you would Decorate-Sort-Undecorate an enormous array because the overhead of passing a function to sort() was so much slower was ridiculous.

In the case of regexes, breaking apart the regex into readable chunks and processing the data multiple times was so much slower that everybody piled everything into the regex. Named captures didn't appear until 2007.

Passing a function into a sort() is going to be slow in any language, unless the language is able to inline the function. That depends on static analysis by the compiler, which is notoriously difficult in Perl.

The key, as you mentioned, is to use builtins, or more generally to use canonical expressions. There's always more than one way to write an expression, but Perl definitely has opinions about the correct way(s), and the Perl compiler does pattern-matching against the code to recognize many of the canonical expressions. They then get compiled to higher-performing IL code. This is why using things like the spaceship operator, <=>, will run much faster than equivalent verbose comparison code.

This is over-simplified, but I always thought about it this way: the Perl compiler generates "assembly code" for the "Perl CPU" to run. It can generate verbose assembly code using lots of low-level assembly commands, but it can also recognize commonly-used expressions and compile them to a single high-level assembly command, which can then run in a single "CPU cycle" instead of stepping through verbose code. Therefore, it was always worthwhile to learn how to write the commonly-used expressions in the common way, so that the compiler can recognize and optimize them.

I don't know if other languages have this kind of optimization. JIT compiling, as I understand it, does a quick and rough compile without optimizing on the first pass in order to start execution quickly, and then recompiles with more optimization on code blocks that are consuming a lot of runtime. I don't think any of the JIT compilers look for common expressions and apply pre-determined, hand-coded optimized function calls to them.

PHP always got heat for the same reasons (so much terrible, amateur advice replicating via StackOverflow et al.), and the only thing keeping Python from meeting the same fate is the fact that shoddy scripts written in Python generally don't get deployed to a public-facing platform for the world to abuse.

Craziest thing I ever found that someone did with excel was implementing a neural network using it. It was meant as a tutorial to teach how neural networks worked, and not meant for production systems, but I do wonder if anyone ever took it further.

Stands to reason that "everyone" is coding wrong. You wouldn't say people are programming just because natural language is turing complete, or because they are while physics is turing complete. One important aspect of information is structure, but programming classes at the low level merely teach syntax. So I'm thinking that therefore spreadsheets are unstructured. Sure that's elitist or at least academic - in comparison to amateurish.

Excel is a great way to code, but a horrible way to decode. Every try to pick apart someone else's spreadsheet.

You could make the same argument about most programming languages. I hate to sound like I'm parroting Spolsky, but, he's right that the reason that the reason every developer ever wants to re-write the code they inherit is because it's easier to write code than it is to read it.

It's obviously an exaggeration, but not by much.

the way i tell great developers from good developers is from how willing they are to dive into someone else's turd and make changes to it.

Oh god the years I spent on doing that with wireless drivers were a fucking nightmare.

Which I guess makes that test more about dedication and attitude than about raw skill.

have you tried to do scientific plots with Excel? I only tried libre calc, which admittedly might lack behind in a couple regards, but it's nigh impossible, no compare to scripting.