Array languages vs. the curse of the spreadsheet
blog.dhsdevelopments.com
blog.dhsdevelopments.com
when will array enthusiasts understand that having first letter A does not mean the language should be aimed at Aliens?
get over your "code golf" tendencies and start thinking about *readability* - give us long names, give us brackets, give us ASCII and stop pulling everything into a single line in examples - _THEN_ you'll have your first tiny chance of being heard and taken seriously
aim design at common people, not gurus
I've always assumed that people working on these languages come from a heavy maths background, so to them the sequence of symbols is more “naturally” readable. So saying “if array language people used words I'd understand better” is not unlike saying “if the Chinese would only speak English there would be less language barriers”. Not wrong, of course, but perhaps not a reasonable expectation.
In similar fashion, when engines were first invented and were being sold to miners, they didn't expect miners to understand the scientific terms around energy, they invented a term that could be easily understood: horsepower. Every miner could wrap their head around the work that twenty horses could do.
Here, x̄ or μ may mean average, but how do you pronounce either? Most are unfamiliar. Identifiers "avg()" or "average()" are more approachable, and already in use in spreadsheets for similar reasons.
I am not an English native speaker, but learned programming before even starting with learning English. In my head, the English keywords used in programming languages are thus just abstract (mathematical) symbols that are incidentally also English words.
As part of my work, I also work with Russian programmers, who, as long as they lived in Russia, hardly ever had to understand anything in English as part of their programming job ("Russia is so big that it is not necessary to learn English"). Just to be clear: they nevertheless attempted to learn English while living in Russia, but because of of intellectual interest, and not because it was necessary or helpful for their programming jobs. I heard rumors that a similar stance holds for China.
Thus: English may be the lingua franca of programming in North America and some European countries, but clearly not worldwide.
So, you don't have "for" or "while" loops where you live? Not even a "main" function?
> the English keywords used in programming languages are thus just abstract (mathematical) symbols that are incidentally also English words
Explanation: People learn python without understanding meaning of keywords used. One must understand what ‘for’ means in python (i.e. repeated execution), but there is absolutly no necessity to understand what it means in English ( https://www.merriam-webster.com/dictionary/for )
Summation (Capital Sigma), Multiplication (Capital Pi), integration, roots. Limit? "lim". Logarithm? "log", etc.
It is not like 50 glyphs of array languages. It is maybe 5 glyphs and 10 other conventions (like division over-under, root and power). Also, mathematics notation is 2D and multi-size (capital sigma is much larger than variable names and numbers, summation indices on under and over sigmas is much smaller) which helps immensely. Same formula written in one font in one line will be much less comprehensible.
UPDATE: Yes, I forogot about E-inverted (exists) and A-inverted (for all), and two or three common operators related to sets (includes, subset), but they are not needed for calculations even in math, only for theorems. And array languages are not theorem proofers.
Each area of math could have its own special notations too, but I don't expect that category theory specialist will understand notation of very specialized complex functions paper.
> Sorry, but mathematics notation is not THAT cryptic.
It can certainly be that information dense. Whether that is considered cryptic or not is in the eye of the beholder.
> It is maybe 5 glyphs and 10 other conventions
Heck, we've got more than 5 glyphs just for expressing (in)equality: =, ≡, <, >, ≤, ≥, ≠, ≢, ≈ (and those are just from memory of doing maths to A-level nearly three decades ago, a couple of years of CompSci after that, & programming since, and that isn't touching on symbol sets from subsets of mathematics like set theory).
> Each area of math could have its own special notations too, but I don't expect that category theory specialist will understand notation of very specialized complex functions paper.
While each area will have specialist notation that is not commonly understood by those unfamiliar with the area, most maths people will understand a chunk of the notation from a number of areas. I don't consider myself a maths person and I know the basic glyphs from a couple of areas when I see them.
> At this point, comments suggesting the code is “unreadable” are bound to be thrown around, but once you learn a handful of symbols, this is in fact significantly more readable than the kinds of formulas I see on a regular basis in Excel.
To your point: what would this look like using English words as identifiers rather than unicode soup whose names I have likely never heard of? It isn't like those characters are easy to find on my keyboard.
Don't let your dreams be dreams (I actually have one): https://www.pckeyboard.com/page/product/00UA41P4A
perhaps not to you but perhaps they are for a large number of people who are used to seeing words they recognize, surely being able to read a simple program before you have learned and guess what it means will be a great comfort to people and make them feel that yes, they can learn this.
A bunch of incomprehensible glyphs might interest someone like a puzzle solving challenge, but not everyone will want to solve a puzzle in order to first start to learn a language.
This argument was already used in the past to justify the very verbose syntax of COBOL.
My argument was that a language that looked comprehensible might invite learning, which is not exactly the same thing.
Looks like we found Goldilocks zone of programming languages in years after first version of COBOL.
BASIC is too little, COBOL is too much.
Array languages is less than BASIC.
And it is Perl which is humiliated for line noise all these years!
Considering the current state the Goldilocks zone of programming languages is like calling the verbose style in which mathematical proofs were written down in the early modern period the Goldilocks zone of writing down mathematical proofs. :-(
And it is good in both ways: some times ago there was idea that everybody should learn a little of programming (see, for example, BBC program in UK which brings us BBC Micro and, later ARM processor ;-) ), because now almost everything is computer (including my wahsing mashine). Now, it is not popular idea, now all corporations try to build non-programmable walled gardens, but it was not so always.
COBOL (and SQL) were created with managers, not programmers, in mind. Goal was to allow managers automate their business-processes without calling for IT consultant.
Spreadsheets are VERY near to this goal. I've seen A LOT of business and finance automation made in excel by people, who will punch you in the face, if you say them that now they are programmers.
If we want to have very specialized, PhD-requiring languages, it is perfectly Ok. But then don't try to sell them to Spreadsheet users.
The problem rather is that "fetching people where they are" regarding the programming language design means keeping these people stupid. Instead, you should market the programming language to spreadsheet users by explaining them how the new kind of thinking that the programming language enables lets their intellectual potential explode.
My experience shows, that programmers love their job and want to be better for sake of being better much more often, than other employees (and it is considering norm - like "show your github profile" on hiring, do you know cases when auto mechanic is required to show custom-build car to get job in the shop?). And even among programmers it is not universal rule, I know some colleagues (very competent ones, need to say) for whom it is simple job, not passion. They don't read bleeding-edge papers on CS, they don't have pet projects and, even, may not own computer at home - PlayStation/XBox and smartphone is enough for them, because computers are tool of trade, not home necessity.
Also, all these comparisons with mathematical notation left out that mathematical notation is not linear and one-font-size-fit-all, as our programs, even in array languages.
Of course they are! I can't type them in on my keyboard without doing a google search first.
It's a barrier according to all definitions of the word. It's unnecessary friction. It's slower to type in when learning. It's slower to read when learning.
Compare it to a language that isn't bragging about how smart you are if you know it - all those barriers are removed.
----
Yes because APL or array languages in general have absolutely no hidden gotchas ...
You mean like how you've got to find out how to type accented characters - á or ö for example?
The glyphs are available as a keyboard layer which you can easily find and install, or simply enable on Linux.
Yes there is an initial hurdle when you're learning, but the same applies with +-×÷, and any languague with a different alphabet/symbol set! Once learnt however, the ~50 odd symbols are all the primitive functions, and then have speed and ability like maths to write and play with expressions.
Thee difference is, with any other new programming language I already know those! IOW, it's not a barrier.
> Once learnt however, the ~50 odd symbols are all the primitive functions, and then have speed and ability like maths to write and play with expressions.
50? You'll forget them if you don't use them. Still a barrier, even when not learning anymore.
Look, I get that there's a speed upgrade, but it's a balance, and 50 is way too much for something that you'll forget easily due to lack of usage.
For example, why bother with 26 elements that compose to all the words in the English language? One can simply memorise the most frequent 30000 words and assign a different rune to each one.
Obviously, we don't do that. The languages that do do that get outcompeted by the ones that don't.
The simple reality is that languages, both human and programming, have to match what the average human is most productive in. It doesn't matter that genius developers are more productive with esoteric runes; their experience is irrelevant because they aren't doing the grunt work that 99.9999% of programming entails.
I don’t want any characters in my computer program I don’t see on my keyboard.
In the modern technical world, you are basically surrounded by mathematics; thus a lot of people (in particular nearly all programmers) have already seen many such symbols.
Why not make it straightforward first and then when people understand it, introduce the shortcuts? Not nerdy enough?
Sometimes I have the feeling that a certain type of peogrammer doesn't even want others to understand what's going on, what they want is bending them to their own will and conventions.
This is just a restricted form of functional and high order programming. It's slightly unusual but many people feel comfortable with it.
Chinese, theoretically, is more simple language than Vietnamese in terms of grammar structures and such. But Vietnamese uses Latin alphabet (augmented with diacritics) and for western person it is huge help when you start to learn language. Maybe, on B2+ level it is not relevant anymore, but when you try to study language from zero it is blessing (again, for western person).
So the only audience where this is not true, is non-English speakers who want to learn an array language as their first programming language.
IMHO, it is relevant to discussion about array languages.
The fact that nobody selling me on these arcane runes talks about something like that tells me it’s more about code golf for them.
Somehow the usual culprits of +-/*|^&% get a free pass on this, and are somehow less arcane than symbols like ⍋ or ⊃.
There's nothing inherent about this, it's about familiarity, and once familiar, everyone always prefers the symbols over spelling it out like plus, minus, divide, multiply...
Whereas APL fans will insist it’s not hard while listing no benefits besides dense code.
* except for the fact you're supposed to remember the idioms for everything : https://aplwiki.com/wiki/FinnAPL_idiom_library
And if you had a toggle-able mode, you could still have your code golf.
As it happens, I find terse languages, and "code golf", as more approachable and thus readbale. It is lower volume, more concise and compact. Generally, I can read all of it. I use an ASCII-based array language that uses brackets: https://en.wikipedia.org/wiki/K_(programming_language)
I am not sure why I prefer writing less code as opposed to writing more code. Maybe I just dislike typing. In any case, I am thankful these languages exist. For both reading and writing.
To insinuate that no one takes array languages seriously would of course be wrong, as one can see from the above Wikipedia page.
The question I have is why anyone who prefers popular verbose languages would care about the discussion of unpopular array languages on HN. Why even bother to comment and complain. If array languages are incomprehensible and unpopular, then why pay any attention to them. Appeals for verbosity over terseness do not seem to be working for the vast majority of Javascript I see in webpages from the larger so-called "tech" companies running the most popular websites.
Array languages are very easy to avoid. Imagine trying to avoid Python. Not easy (speaking from experience).
Telling array language authors to make their languages more verbose is like telling Bob Dylan to play what people want to hear and only then will he have a "tiny chance of being heard and taken seriously".
(Also wondering if commenter meant braces not brackets.)
Thank you, 1vuio0pswjnm7
in some sense it is a "thesaurus vs ancient egyptian" argument
I can totally understand that it would be infinitely nicer to build Stargate and go fight Goa'uld or inherit knowldge of the galaxy-spanning Asgard - but it's the events of "Yes, Minister" that actually do happen in our, real, world
> As it happens, I find terse languages, and "code golf", as more approachable and thus readbale. It is lower volume, more concise and compact. Generally, I can read all of it. I use an ASCII-based array language that uses brackets: https://en.wikipedia.org/wiki/K_(programming_language)
"A case of MUMPS" (https://thedailywtf.com/articles/A_Case_of_the_MUMPS) is on The Daily WTF. "WTF" is exactly the default behaviour of people encountering these codegolf-ed langs
> The question I have is why anyone who prefers popular verbose languages would care about the discussion of unpopular array languages on HN. Why even bother to comment and complain.
you're the seller, I'm the buyer
array enthusiasts advertise their product, listing advantages - we reply with "my problems with it are this, this and this. How can you fix it?". Yes I'm interested. No, I don't agree that my concerns are empty.
> (Also wondering if commenter meant braces not brackets.)
technically, I meant parenthesis. But nowadays, and especially in programming world, all 3 are synonyms and you just add [curly/round/square] in front
brackets are just easiest of the 3 for me to recall
https://pitchfork.com/thepitch/855-dylan-goes-electric-newpo...
"In 1965 Dylan was only 24 years old, but he had already gone through a lot of changes and been criticized and attacked for each of them. His first performances as a teenage rock'n'roller were met with laughter and jeers. When he turned to folk music, the high school girlfriend who had shared his early passions was troubled, asking why he'd given up "the hard blues stuff." When he turned from singing traditional folk songs to writing his own material, old supporters accused him of becoming "melodramatic and maudlin"-one, stumbling across a typescript of "The Times They Are a-Changin", demanded, "What is this shit?" When he turned from sharp topical lyrics to introspective poetic explorations, followers who had hailed him as the voice of a generation lamented that he was abandoning his true path and "wasting our precious time." When he added an electric band, he was booed around the world by anguished devotees, one famously shouting "Judas!""
https://www.mojo4music.com/articles/stories/robbie-robertson...
"We got booed all over North America, Australia, Europe, and people were saying this isn't working and we kept on and Bob didn't budge. We got to a place where we would listen to these tapes and say, "You know what? They're wrong. And we're right." Eight years later, we do a tour, the [1974] Dylan/Band tour, we play the same way, same intensity and everybody says, "Wow, that was amazing." The world came around - we didn't change a note.
The obvious thing we learned - that everybody learned - was there was a new way of songwriting. There was a much more colourful, descriptive, humorous, outrageous thrill ride of wordplay. We hadn't seen this before - this was breaking some big rules. I remember saying to Bob one time, "Maybe there's too many verses in this" (laughs), and he said, "There probably are, but that's what I was thinking about when I wrote it." His spirit was on fire, and he was knocking down the boundaries that had been built up around music. It excited me to be part of this revolution."
https://www.nytimes.com/1998/10/11/arts/music-when-dylan-s-g...
"BUT Dylan was too smart and too stubborn to let hidebound listeners prevent him from making the best music of his career.
The hecklers were wrong about the music, as most of them would probably admit now."
https://ultimateclassicrock.com/bob-dylan-goes-electric-newp...
"The New York Times' Robert Shelton, whose 1961 review of a Dylan concert is considered to have launched Dylan's career, wrote, "The young audience's displeasure was manifested at the end of most of the numbers, by booing and shouts of `we want the old Dylan.' The young star plowed valiantly on, with the sort of coolness he has rarely displayed on stage."
Calling the crowd "rude," "immature" and "noisy young boors," [Dylan] couldn't help note the positive reaction given to "Like a Rolling Stone," which by now was a hit. "Evidently the hostility extends only toward things with which they aren't familiar," he sneered.
[Hostility toward things with which they arent't familiar. Like HN commenters' hostility toward array languages.]
This song-and-dance with his audience continued over the course of the next year."
http://www.goldminemag.com:80/article/bob-dylan-gets-religio...
"The show opened with as excitedly noisy a crowd as Dylan had ever faced, 2,300 souls jammed in, and all with their own loudly exclaimed wish list of favorite Dylan songs and expectations. Fourteen years earlier, you could imagine the 1965 Newport Folk Festival buzzing with similar anticipation and excitement ... and reacting with similar bemusement as the show took shape. Only it was not the sacrilegious explosion of electricity that left the audience reeling in its seats.
As he led the band through a 19-song set, Dylan did not play a single song that predated his religious conversion. Even the encore was uncharted territory. "Blessed Is The Name Of The Lord" and "Pressing On" were both new songs, and though the former rocked and the latter soothed, the crowd was in no mood to be impressed.
Afterwards, Dylan congratulated himself on not playing a single song he'd ever performed onstage in the past, and when the subject of his fans' expectations was broached, he simply shrugged. "Oh, yeah, that's right. They want the old stuff."
Pittsburgh's KDKA TV station even filmed the fans' exodus, with one disgruntled ticket-holder snapping to the cameraman, "He stinks. He's the worst. He's bad. It's a waste."
Dylan's audience was furious. Shocked. Cheated. It was Newport all over again, or maybe the Manchester Free Trade Hall, later on in that so-controversial 1965 tour, when the bootlegger's tape was suddenly interrupted by a bellow of "Judas!""
there is demand for readable replacement of regex
write-only languages stop giving "better thought" right after you stop writing that part of the code
Had a guy that tried to push LaTeX as documentation writing tooling for the whole company.
Sales guys went with Confluence. They were more concerned with actually closing million dollar contracts than learning fancy typesetting system.
It's not necessary to make everything toddler-safe; we already have Python.
Or, you could use C++ which, on one hand is hard and obtuse to read, and on the other hand, has 10x as many footguns as C.
Here is a short read on this topic,
https://www.amazon.ca/Embracing-Modern-Safely-John-Lakos/dp/...
It's only 2661 pages. Yes, that is almost 3x longer than The Rust Programming Language, 2nd Ed.
After explaining how spreadsheets enabled ordinary users to solve real-world problems better than traditional programming, the author proposes to take it to the next level with APL-style expressions, and gives a code example for the following task:
You're given a dictionary and asked to find the longest word which contains no more than 2 vowels.
I don’t believe there is a single Excel user in 40 years who ever needed to perform this operation. It’s the kind of abstracted “l33tcode” exercise that only appears in computer science classes and a certain type of programming job interview.
A practical example of how an array language IDE works better than a spreadsheet for some useful task would go a long way here.
I pretty rapidly realized that the App Architecture group knew even less than I did about macros. At least, I had talked to a handful of macro developers and some Excel old-timers to get a grip on what people actually did with Excel macros: things like recalculating a spreadsheet every day, or rearranging some data according to a certain pattern. But the App Architecture group had merely *thought* about macros as an academic exercise, and they couldn’t actually come up with any examples of the kind of macros people would want to write. Pressured, one of them came up with the idea that since Excel already had underlining and double-underlining, perhaps someone would want to write a macro to triple underline. Yep. REAL common. So I proceeded to ignore them as diplomatically as possible.
I think a good language for replacing spreadsheets would be Scheme, or something similar. You can teach it to someone very quickly as the syntax is quite minimal. Provided functions are given human readable names, I think it could be very intuitive. You would also need to offer some kind of graphical editor for the code. Lisp has wins here as well since you could likely translate the code into a graph without much issue. However it would also be good to allow the inverse of such transformations.
Hard disagree. Which do you find more readable?
-b/(2a) + 1/(2a) √(b²-4ac)
(plus (minus (div b (times 2 a)) (times (div 1 (times 2 a)) (sqrt (minus (pow b 2) (times 4 a c))))))
plus(minus(div(b, times(2,a)), times(div(1, times(2,a)), sqrt(minus(pow(b,2), times(4, a, c))))))
APL is actually simpler in that regard in practice, and the gains you get in readability from the symbols are just like above. Better yet, though, which of the above do you immediately think, "that 1/(2a) can be factored out"? Oh, and here's APL for the above:
(-b÷2×a) + (1÷2×a)×((b*2)-4×a×c)*1÷2
fn quadratic_roots (a: real, b: real, c: real) -> (cmpx, cmpx) {
let root1 = (-b + sqrt(b^2 - 4 * a * c)) / (2 * a);
let root2 = (-b - sqrt(b^2 - 4 * a * c)) / (2 * a);
return (root1, root2)
}
The thing that array languages get wrong is that code is read more often than it is written, and repeating yourself is worthwhile in order to make it easier to understand for the next person who looks at it. Compilers are extremely good at recognizing patterns and optimizing it for numerical code. Humans are really bad at recognizing patterns from unique sequences of characters.
It seems to me that array language designers understand this, but they make a different trade-off by prioritizing density.
A screenful of an APL-family language can contain a program that might take thousands of lines in a Java-style language. There’s power in being able to see the whole thing at once.
Would a passenger jet airplane be easier to use if the cockpit only had an iPad and the pilots would have to navigate through UI trees to find available actions? It would certainly be more discoverable to the ordinary person, but the pilots would probably be deeply unhappy with this design. In this analogy, an APL-style program can be like a “cockpit” full of instruments that you designed yourself for that exact job.
Except you sneak in alien symbols: +, - (used both as a unary and a binary operator!), *, ^, and /. See how readable they make things, though? :)
Your quadratic_roots is, indeed, nice in isolation. I'd even go as far to say that it's a good pedagogical piece. However, in production code pedagogy is not what I want to optimize for. Production code often repeats patterns with slight variations on a theme: What if you only want real roots? What about complex a and b? What about just grabbing the discriminant?
It's easy enough to modify quadratic_roots, but then you either get a combinatorial explosion of function definitions, or you end up with a function with extra parameters to select the different variations, and you often end up with a deeply nested function call graph, e.g. replacing sqrt(b^2 - 4*a*c) with discriminant(a, b, c), which makes quadratic_roots more annoying to read.
In practice, defining a function or some abstraction barrier is making an architectural decision. Ideally that decision would correspond perfectly to some fundamental feature of the problem you're trying to solve, but in practice we rarely are coding with perfect problem domain knowledge, right? In practice, our functions/classes/abstractions end up accumulating cruft, right? Why is that?
Where we traditionally handle complexity by setting up abstractions to let us hide parts of that complexity, APL is good at using a different tactic, called "subordination of detail." Done well, this looks like writing very simple, direct code that empowers your basic language tools to take on domain-specific meaning without introducing any abstractions.
Here's an example that I came across recently: t[p]=E. The primitive operations are simply equality comparison (x=y) and array indexing (x[y]). However, in the specific problem, t[p]=E effectively selects parts of an AST that correspond to expressions. It's just a friggin' index operation and equality comparison! Normally this kind of operation would hinge on a reasonably large hierarchy of datatype definitions, traversal patterns, and hard-to-read performance hacks.
Instead, the APL (t[p]=E) is crazy short, obvious in meaning, and you can literally just read the computational complexity right from the expression! What other language does that? Granted, you gotta learn a little APL first.
> The thing that array languages get wrong is that code is read more often than it is written
Personally, I find APL quite pleasant to read. You really should give it an honest try.
Yes and that's what programming languages like APL do. Whole algorithms for manipulating equations represented as matrices can be groked with just a few symbols. At least to my mind, it's a very efficient way to write code, at least for domains that fit this model.
I learned APL in the mid '70s in high school. The math department offered it as an option to those of us learning linear algebra. One course reinforced the lessons from the other. Those of us in the APL course found ways to solve other problems not directly related to linear algebra, and we marveled at how compact the code was (especially compared to Basic, which a friend and I had learned the year before).
∇ quadraticRoots (a;b;c) {
root1 ← ((-b)+√(b⋆2)-(4×a×c)) ÷ (2×a)
root2 ← ((-b)-√(b⋆2)-(4×a×c)) ÷ (2×a)
root1 root2
}
As for the language, the longer version looks like an imperative solution please it is an imperative solution. Kap allows you to write code that looks mostly like your average scripting language:
a ← 0
sum ← 0
while (a < 10) {
sum ← sum + a
a ← a + 1
}
sum
Of course, an actual Kap programmer wouldn't write the code above like that. They'd write it like this instead: +/⍳10
There is a perfectly valid argument that the ⍳ in the example above could be written in a different way, and sure, Rob Pike chose to use actual words in his version of APL, or you could use J where the example would be written as +/i.10
But none of that is really important, and most people in the array programming community doesn't care whether you write "plus reduce iota 10", or +/⍳10. What they really care about it how the idea of using array operations completely eliminates loops in most cases. That's what is interesting, not the choice of symbols.
> Scheme actually has a ginormous dictionary by comparison
Oh, and also, I should point out that Scheme has no symbols to learn. Quite small compared to the 50 odd that APL has. The reason for this is that users already know all of the symbols in Scheme. "log" exists as a concept in people's brains already because it is a word and also the name for the function it describes. You know the word "log". You can start typing "log" and it will show up in auto-complete. ⍟ (circle with small star in it) doesn't exist in people's head and you can barely make out what it is without a special font and you cannot type it without a special input method. People do not know the word "⍟". It is an entirely new concept for something people understand perfectly well and all it does is reduce the length of a line by 2 characters.
(+ (- (/ b (* 2 a))) (* (/ 1 (* 2 a)) (sqrt (- (^ b 2) (* 4 a c)))))
;; or just
(/ (+ (- b) (sqrt (- (^ b 2) (* 4 a c))))
(* 2 a))
;; With Scheme's SRFI-105 – Curly Infix Expressions
{{-(b) / {2 * a}} + {{1 / {2 * a}} * sqrt{{b ^ 2} - (* 4 a c)}}}
;; With a macro
(math - b / (2 * a) + 1 / (2 * a) * (sqrt (b ^ 2 - 4 * a * c)))
;; Reader macros
#math(-b/(2*a) + 1/(2*a) * (b^2 - 4*a*c))
(formula (+ (- (/ b (* 2 a))) (* (/ 1 (* 2 a)) (sqrt (- (^ b 2) (* 4 a c))))))
(formula -b/(2a) + 1/(2a) √(b²-4ac))
That logic is broken. Even accepting it as true, "X is easier to read than Y once you learn a specialized symbolic language" is not "proof that one is not more difficult to learn than the other". It is, at best, proof that there is value that may make it worth learning the one even if it is harder to learn, which is very different from proof that it is not harder to learn.
The Windows version of Improv wasn't nearly as good, possibly because it seemed too advanced for PC users familiar with 1-2-3, or because it might disrupt an already very profitable revenue stream.
I believe there's a spiritual successor in Quantrix.
I think Improv could only satisfy a subset of Excel use cases. In particular, any non-tabular sheet was hard to do.
If you have one tool that does it all, even if imperfectly, why bother learning a second, special tool just for a subset?
Note also that it was easier for Excel to superset Improv (e.g., by adding Pivot Tables) than the reverse.
We're supposed to rejoice and hail the spreadsheets gods because a spreadsheet can take unstructured data and create a map of (key,value) where key is a value appearing in one column and where value is the sum (for example) of the values appearing in another column for that key.
We literally have a simple mapping like that, call it a "pivot table", and it's supposed to be some achievement of mankind.
I beg to differ.
From what I've seen I'm pretty sure the world runs on broken spreadsheets, with bogus numbers being presented during meetings, used for decisions (including political decisions), etc.
At some point management realizes the company (or institution) has been taking nonsensical decisions since years and it's not only a minefield of bugs and near impossible to debug "code" (that'd be the "I've certainly spent more time than I want to admit trying to untangle one of those bowls of spaghetti trying to find out why a certain number is computed the way it is." from TFA, which any of us ever having been tasked to port a spreadhseet to an app know all too well)... But it's also a minefield politically: you cannot ask too many questions or point out too many bugs because it exposes the brokenness everything has been running on since years.
Even though somewhere, at some level, people knew everything was FUBARed (otherwise the decision to rewrite the spreadsheet to a dedicated app would not have been taken), it's a minefield. You have to walk on eggs to not piss anyone.
My epitath may be:
"The world runs on broken spreadsheets".
People create spreadsheet messes not because they are stupid or lazy. They do what everyone else itt would deem impossible: create and evolve a mostly working system as needed while doing their regular job.
I’ve helped to migrate both excel-based and software-based enterprises. Few factories on foxpro/clipper, a couple of realties and one leasing on excel. There’s no real difference in how much gray hair you get in the process.
Pretty sure if some company came to ours with what we see in TFA and asked us to help with migration, we’d laugh for a week straight.
> Even though somewhere, at some level, people knew everything was FUBARed (otherwise the decision to rewrite the spreadsheet to a dedicated app would not have been taken), it's a minefield. You have to walk on eggs to not piss anyone.
Your argumentation implicitly makes use of the assumption that the respective person scores high on agreeableness:
> https://en.wikipedia.org/wiki/Big_Five_personality_traits
There exist people who do not mind pissing other people off ... ;-)
[0] https://support.microsoft.com/en-us/office/guidelines-and-ex...
https://support.microsoft.com/en-us/office/dynamic-array-for...
He is probably referring to the new dynamic array handling which enables “array spill” for almost any built-in function and can be much more intuitive than the previous (still supported, of course) variant. This was introduced in ~2019 I think.
> If you have a set of numbers, and you want to add some constant to them, the way you'd do that is to create a formula that adds the constant to a cell and put that into a different location, usually on the same sheet, and then paste that formula to all the cells in an area with the same size as the original data, and hope that you got the dimensions right.
>users of spreadsheets are already thinking in terms of matrices of values
I am pretty sure most users of Excel are not thinking about matrices. They are thinking of rows and columns. Programmers and mathematicians might generalize this to matrices and values, but the users themselves are not thinking about it. This is akin to viewing Tinder as a "human interaction app" and lumping it in with Slack under that abstraction, and not realizing that those are two very different use cases and mindsets of the user.
> The spreadsheet paradigm hides what's important (the formulas) in favor of the superficial (the data)
To most users, the data is what is important in a spreadsheet and the formulas are implementation details.
In my experience of erm... hardcore-spreadsheeting (engineering, finance)... the cells whose value matters and become a final report that motivates design/decision-making are a tiny percentage of the total cells. Most of the cells in a spreadsheet are either raw data or intermediate values. Their values have no real meaning to the author, just steps in a long calculation. They might look at them for sanity checking but if they need to verify their values then that is better as another calculation not by repeated manual inspection.
Of course there are many types of spreadsheet user. Every cell value matters for your Vinyl collection or whatever :)
So, if the goal is to make it easier for someone to take the leap from a spreadsheet to a programming language, array languages are likely an easier hill to climb (people already familiar with, say, Python, might struggle, but they are not likely to be interested in learning a new programming paradigm anyway).
The macro was the first thing that came to mind. Your formula is cleaner.
When needed, a visual indication can easily be created with conditional formatting, or with the Show Formulas command (Ctrl+`), as noted by the sibling. Know your tools is the answer here.
I don't agree with that. It's like saying in JavaScript the "const" -keyword is distracting because it creates a distinction between values that are constant and values that change during running the program.
The spreadsheet-app might have a "publish-mode" where the distinction between inputs and formulas is hidden but in general it helps that user can easily distinguish between between inputs and procedures.
Distinction does not imply distraction.
What is often distracting is the need to spend mental effort to make a distinction, to try to understand a spreadsheet.
You might as well say that gaslighting in general reduces distraction, because there are fewer distinctions you can make in the gas-light.
Though like you can optionally use const (for varying degres of optional depending on your local coding style rules), you can optionally highlight formulas using conditional formatting based on the ISFORMULA() function.
It does seem like something that makes sense to be a built-in though.
Big companies have all the money to modernize their systems yet they know the execution is difficult if not impossible to achieve in time. The classical story about running an older operating system and/or application that works instead on venturing on modernizing it.
But I do like the idea of different kinds of cells having a different style entirely. You can style them yourself but a default theme would be a cool feature.
A 2nd improvement I would go for would be to allow the cell-formula to be written in JavaScript.
https://developers.google.com/apps-script/guides/sheets/func...
solution =: >@:{.@:(\: #&>)@:(((+/@:(e.&'aeiou') <: 2:) # ])&.>)@:;:
Example
solution 'yes, today you are reading something that is not so easy to grasp'
the result is: today
Ruby : frase.split.select{|x| x.count("aeiou")<3}.sort_by(&:length).last => "today"
Edited: Added a comparison with Ruby. It seems Ruby here is easier to read and to compose.
The enumerable module of Ruby provides many methods that can be easily implemented in J. Just to show one of them
max_by =: 1 : '{~ (i. >./)@:(u&>)'
F =: >@{.@(\: #@>)@(#~ (2 >: +/@e.&'aeoiu')@>)
which I think maybe composes a little better
This doesn't include the splitting. I would imagine I would be feeding this function from a boxed list from a word list, like: F 'b' fread 'dict.txt'
Also `;:` is unreliable on hyphenated words, etc. Better to use `cut`.
maximumBy (on compare length) . filter ((< 3) . length . filter (`elem` "aeiou")) $ words phrase
`sort(split(p)[count.(r"[aeiou]", split(p)) .< 3]; by=r -> length(r))[end]`
which also returns "grasp".
But this one appeals to me more, because it doesn’t split twice:
`sort(filter(w -> count(r"[aeiou]", w) < 3, split(p)); by=r -> length(r))[end]`
maxBy(on cmp (#)).(((<3).(#).(el"aeiou"|=))|=)$words phrase(1) look for changes in formulas, which could indicate an error. Why is the formula for cell A99 different from that in A100?
(2) write formulas in Python/NumPy or another language that replicates the formulas.
For #1, there has been research on linting and automated semantic error or smell detection in Excel (e.g., https://github.com/ExceLint/ExceLint). Some features are live in Excel today!
For #2, Excel now supports Python.
I don't think you need to do any conversions, all of the necessary information exists in the spreadsheet.
How do you define a reusable function?
---
(I know the answer....it's a proper programming language.)
Abstractly.
Listing all possible inputs with their outputs, and then doing a lookup....that's only sometimes useful.
If we're talking Excel 2021, Excel 2019, etc. then no.
https://support.microsoft.com/en-us/office/create-custom-fun...
https://github.com/bradbase/xlcalculator
https://github.com/dgorissen/pycel
https://github.com/vinci1it2000/formulas
Funfact: it's more complicated than you think to implement these engines. There's no spec for the formulae, so you have to reverse-engineer based on observed behaviour. You run into a lot of strange edge cases in seemingly mundane arithmetic formulae, and type conversions are a constant source of surprise.
Emacs org-mode has tables, which you can use as spreadsheets [1]. The formulas are not written in the cells, but after the table, with one formula per line. You have to update the table every time you update the data or the formulas.
I once tried to use it, but it was way too difficult, and I went back to Libreoffice.
"Show formulas" is fine if it's "=A1-B1" but for real work, formulas get long with lots of filters and look-ups etc. I break them across multiple lines just to read them. If I toggle visible formulas, I only see the first few characters of such formulas that fit in the data cells. Even if it did work, reading every cell when you have 1000s of rows just sucks for spotting errors caused by a fill down error or a cat on keyboard or something.
The only way I can be confident with a spreadsheet is to have a validation sheet that performs a bunch of error-checking. For serious stuff, I think you almost need to keep an entire duplicate of the data and recreate the results with a different process because it's just so hard to catch everything e.g. heartbreaking to open an old important spreadsheet and see a line of data got duplicated somehow. Spreadsheets are just so damn mutable. A keyboard shortcut with the wrong application in focus and who knows what damage you might have just done to a spreadsheet.
If you think they will, I also ask, what language that looks like this has ever been used mainstream? Or even in a hacker community? Brainfuck and golf languages are used by a very, very small niche. Regex is probably the most similar, but you only need to learn like 4 symbols to be useful (. * + \). And that hasn't caught on in the average excel user's repertoire.
I really wish something like this would exist and be useful because I share the exact same gripes as the author.
There is also J, which is used in real businesses. And K and Q are array languages that is popular in finance.
Kap takes the good ideas of APL, as well as its basic syntax, and attempts to combine that with good ideas from other languages, mainly Lisp. I personally use it as the go-to language to solve problems that most people would use Excel to solve, so clearly it's working for one person.
While Kap can be used as an Excel alternative for some subset of of problems, I have no expectation that the average Excel user will switch to it. Instead, I'm trying to adapt some of the nice user interface ideas of a spreadsheet (editing and formatting) while keeping the rest as a regular language. I also want to ensure it's easy to integrate with spreadsheets (copy&paste between Kap and Excel for example, there is also initial work done to have a realtime link with a running spreadsheet).
We can add validations and type to sheet so that the data consistency is not an issue. A good UI of scratch sheet/worksheet will allow for adhoc work on it. Also, we can label formulas in a worksheet (aka variable) that should simplify the code.
See https://en.wikipedia.org/wiki/APL_(programming_language)#/me...
As an outsider it looks like wasted novelty budget. There's already a lot of new things to learn and get used to in an APL school array language, and then on top of that you have to learn to read them right to left.
A lot of APL programmers continue this tradition, and it works for them. Not everybody does it though. I'm one of those people that adds spaces and newlines to try to make the code more clear for myself.
As an example of the kind of code I tend to write, here's the code for the Kap output formatter:
https://codeberg.org/loke/array/src/branch/master/array/stan...
Of course, if you don't know what the symbols mean, a lot of it is probably still impenetrable, but I think this code is less dense than your typical APL code.
max((w for w in words if len(set('aioue').intersection(w)) <= 2), key=len)
Spreadsheets are reactive and stateless and allow you to move seamlessly from data storage to computation on that data. This is really just a calculator REPL that can ingest tabular data easily. I’d be more interested if they’d innovated more with the interface — maybe something node-based?
{ ∪ ⍵ ∪ }Point taken about the tutorials. It's still a work in progress.
> At this point, comments suggesting the code is “unreadable” are bound to be thrown around
I was actually thinking it was unpronounceable, un-searchable, and almost impossible to type.
I would love to see more work on array languages that map onto human languages, larynxes, and keyboards better.
(Obviously most programming languages are rather English-centric. But they also use a character set that is, for better or for worse, accessible to most of the world. More objectively, it’s also a small character set as such things go, and it’s a simple script in the sense that the characters look the same regardless of context, especially in the fonts used for programming. [0] Character sets like CJK are simple but not small. Arabic and Hebrew are small but not simple (Arabic more so than Hebrew). Even French is mildly complex.
I admit that “{ ⊃ ⍵ ⊇⍨ ↑⍒≢¨ ⍵ /⍨ 2≥ (⊂"aeiou") (+/∊⍨)¨ ⍵ }” is at least approximately equally foreign to everyone. But it’s hard for me to ask a coworker “hey, remind me what ⍒ does again?”
[0] I have never seen anyone suggest programming in English cursive. And ligatures for programming are exotic and rarely affect anyone who doesn’t opt in to them.
Would writing argsort(name_of_array, reverse=True) really be so much worse? Or argsort[reverse] (made up syntax) if you want a point-free function form?
You can read more about it here if you're interested: https://www.aplwiki.com/wiki/Grade
And here's a silly example where a random array is created, then an array lookup is performed on the sorted indexes:
https://kapdemo.dhsdevelopments.com/clientweb2/#a%20%E2%86%9...
https://numpy.org/doc/stable/reference/generated/numpy.argso...
I assume this name derives from argmin and argmax, which go back at least to MATLAB. But MATLAB has sort returning indices too.
Some people prefer terseness, think of this as "advanced next-level Perl one-liners" if that helps ...
On the other hand, conventions for reading formulae out loud are difficult and prone to ambiguity.
One more idea. Even in ASCII, there are programming language notations that are hard to say. What is Go's "<-" symbol called? "Arrow"? "Send or receive"? What about C++'s "::"? "Colon colon"?
It is a neat linguistic puzzle to assign sounds to “{ ⊃ ⍵ ⊇⍨ ↑⍒≢¨ ⍵ /⍨ 2≥ (⊂"aeiou") (+/∊⍨)¨ ⍵ }”.
https://stackoverflow.com/questions/592322/php-expects-t-paa...
T_PAAMAYIM_NEKUDOTAYIM which of course is Hebrew for "Double Colon"
At least in Physics you can say “the gradient of f,” and I’ve rarely heard people say “nabla” when talking about physics. And you can look up “gradient” just fine.
We don’t need more jargon just for jargon’s sake.
{ ⊃ ⍵ ⊇⍨ ↑⍒≢¨ ⍵ /⍨ 2≥ (⊂"aeiou") (+/∊⍨)¨ ⍵ } io:read "dict.txt"
This is particularly true when the alternative is Excel, which was widely accessible before it was taught in schools, and is now taught in schools too.
(This language comes from a history where it used to be necessary to by special typewriter balls to print the text. https://www.computerhistory.org/collections/catalog/10269647... )
Classical Math notation is full of weird Greek symbols and absolutely incomprehensible without any training. APL is way more regular and easier to grok as well as to type on computers.
With it's small set of special symbols, regular syntax and human-friendly infix notation APL is designed be very easy to learn.
It looks utterly alien to programmers that are used to the C-style syntax based languages, yes. but your JS or C# code looks as alien if not more alien to an non-programmer as does APL to you.
I am not convinced that APL is harder to learn that JS for non-programmers. I think existing programmer struggle way more with learning APL because it requires them to unlearn certain ideas.
As you might guess from my original post, I'm a Lisp apologist. I like the syntax and I can make an argument for Lisp [1] that sounds a lot like the argument you make here for APL. From what you say and from what I've read, I suspect APL is another language like Lisp (and Smalltalk, Forth, and Tcl). These are all languages derive a great deal of elegance and coherence from a pure approach to syntax and design, and all are seductive in their way to people that have taken on the effort to learn how they work.
The problem is that this doesn't matter enough to overcome the incumbency bias in favor of more traditional languages.
> but your JS or C# code looks as alien if not more alien to an non-programmer as does APL to you.
It would be convenient for APL if this were true, but the reality is that it's not.
I learned Logo and Basic when I was eight. My kids learned JS/Python around the same age. This is a common experience. It's just not that hard to learn basic imperative programming in an infix language.
When I was learning programming back then, those keywords could just as well have been single letters. It wouldn’t have made a difference.
Also, I went up to my printer and there is no “WTF” there. Surely, this program doesn’t work! More seriously, this just illustrates that convention and idioms are much more powerful carriers of meaning than dictionary words. Array languages rely heavily on conventions/idioms.
This is the experience of learning to read music without speaking Italian. "Allegro ma non troppo" doesn't mean much in English.
As for expecting "print" to go to printer, I doubt that's the default expectation of anyone under the age of ~35 at this point.
Array languages rely heavily on idiosyncratic conventions/idioms that are utterly alien to all outsiders. Hence why they don't see uptake, and that is not going to change until their proponents are going to take this seriously. This all reminds me how Lisp adherents keep saying that syntax doesn't matter, except, looking at the actual adoption of Lisp, it evidently does. You are not going to convince everyone else to learn to "hold it the right way".
It demonstrates that your criticism of the syntax of array languages boils down to “it’s different than the things I already know” and that not using words in a programming language isn’t a meaningful barrier to most people, given that most programmers in the world overcame this obstacle with no sweat. There is a meaningful criticism to be had in the area of syntax, but this is not it IMO. One valid criticism I know of is that most array languages have a context-sensitive syntax: you need to maintain a symbol table in your head to know how to parenthesise a line of code. Fortunately, BQN fixes this issue by using a context-free grammar.
Lisp is defined by much more than syntax. I certainly see many flaws in it (starting with the cons cell), which are total deal-breakers for me and have nothing to do with syntax. I think your view that syntax is the biggest obstacle to Lisp’s popularity isn’t as evident as you claim. There are too many other, IMO bigger obstacles.
There's a lot of value in leveraging existing knowledge and experience. If something is different, it needs to justify the expense of overcoming that difference in terms of time and utility.
This is not just a 'tech' issue, it predates that. To pick the example of syntax, infix languages date back to at least John Backus' FORTRAN compiler in the 1950's. That team selected an infix syntax (at the cost of taking on implementation complexity) for the express purpose of aligning with what its expected user base was familiar with. In this case, it was an approximation of the infix notation universal to mathematics for centuries. Even now, infix is taught to kids as young as five or six, if not younger.
Any alternative syntax has to overcome heavily ingrained training in legacy syntax. It also has to overcome the fact that the existing legacy syntax is a more universal means of communication. The fact that something else is new, does not necessarily make it better or worth the investment. This is particularly the case when when there are many competing ways to spend time/energy the point is communicating ideas. (It strikes me as a little odd the tendency of our industry to so often pursue inventing novel languages to help improve communication.)
> Lisp is defined by much more than syntax. I certainly see many flaws in it (starting with the cons cell), which are total deal-breakers for me
This is a weird complaint, given that 1) linked data structures are almost universal in languages with pointers/references 2) all serious Lisps dating back to before Common Lisp offer alternative data structures 3) some Lisp's (Clojure comes to mind) essentially deprecate the cons cell in favor of these alternatives.
But let’s now take something that is actually widespread: angular degrees. Before high school, I used only degrees at school and everywhere else. In high school I was taught about radians. Radians are a lot less used by random people than degrees. But since I learned them, as I later went through uni, then job etc, I’ve completely stopped thinking in degrees and use only radians, both professionally and in everyday life. That’s because their benefits justify the switch. So even when something is actually widespread, we can still find good reasons to switch to something that provides material benefits. I claim that the notation of array languages is beneficial enough (at least in the case of BQN).
As for Lisp:
1) Most languages offer a hash table as the default associative data structure. In Lisp culture alists are the default. Books about lisp also focus pn data structures built out of cons cells disproportionally more than any other data structures.
2) They do. But then you read actual programs available on the web and they still default to using alists. There is an impedance mismatch, if you’re more array/stack/hashtable-minded, which isn’t there in most mainstream languages. FTR I’m a big fan of SML too, but even in SML using the best data structures is very much going against the grain, when the language tries to steer you so much towards linked lists and away from arrays. I swear my SML programs look nothing like the things I find on GitHub.
I was wondering how true this was, so I checked Planet Lisp. The first article is an admonition to use structures and classes _instead_ of alists, and it's followed by a bunch of other articles that use instances, structures, and hash tables. No a-lists to be found.
I've used Lisp (mainly Clojure, but others also) and I don't even remember the last time I used an a-list, outside of maybe Emacs Lisp. As you point out, they're a mostly dumb idea in an age where there are better data structures and memory hierarchies are multi-leveled and often packet switched.
I'll agree with you that it has been part of the Lisp culture, but that's the Lisp culture of the 70's and 80's. The language has moved on.