Notation as a Tool of Thought
jsoftware.com
jsoftware.com
Tricky.
[0] https://www.numberphile.com/videos/juggling-by-numbers
[1] As far as we know. Without the notation we don't actually know what had been done, but when I took the new patterns to juggling conventions, no one knew them[2].
[2] Actually it's stronger than that. I showed people some of the new patterns at the British Juggling Convention in 1985 and no one knew them. Then at the European Juggling Convention just 4 months later, people from the USA were proclaiming them as the latest patterns that they had just learned, and were perplexed at how I not only knew them, but knew many, many more.
[3] And actually Paul Klimek had beaten us to it, but hadn't been able to get others interested in the notation. As far as we can tell, Paul was the first to get the notation.
Another example is juggling notation, that allowed not only the sharing of patterns but the discovery of new ones [2].
A key takeaway for me is that computers make it possible to have unambiguous notation. Quoting from Iverson's paper: "The thesis of the present paper is that the advantages of executability and universality found in programming languages can be effectively combined, in a single coherent language, with the advantages offered by mathematical notation."
Ambiguity is a real problem with much of conventional mathematical notation, which has evolved in fits and starts throughout history. Mathematical symbols are used, reused, and overloaded with different meanings again and again, in so many ways, that context is often necessary for understanding. Ambiguity hampers the use of mathematical notation as a tool for thought.
The other big takeaway from this paper, for me, is that succinctness makes it easier to reason. That is, programming languages that enable us to express more with less code make it easier for us to reason about -- and with -- code.
Its modern descendent are https://en.wikipedia.org/wiki/J_(programming_language) & https://en.wikipedia.org/wiki/K_(programming_language).
There's also bqn[0], among others.
The opaque one-liner:
using IterTools,ImageInTerminal,Colors;for g in iterated(a->let n=sum(map(t->circshift(a,t),product(-1:1,-1:1)));(a.&(n.==4)).|(n.==3);end,rand(Bool,(99,99)));imshow(map(Gray,g));print("\n\n");end
The legible version where we give everything descriptive names so it's not cryptic and mysterious:
using ImageInTerminal,Colors #the APL demo also uses a library for pretty display
using IterTools #okay *technically* this is a minor cheat
function nextgen(grid)
neighborcount = sum(map((t)->circshift(grid,t), product(-1:1,-1:1)))
return (grid .& (neighborcount .== 4)) .| (neighborcount .== 3)
end
function animate(grid)
for gen in iterated(nextgen, grid)
imshow(map(Gray, gen))
print("\n\n")
sleep(0.05)
end
end
animate(rand(Bool,(100,100)))I still think learning mathematical symbols is better than spelling out mathematical formulas and likewise APL and J to me allow the same power of abstraction; it just takes some effort to learn them. A lot of friction is learning something new.
But the power of abstraction of APL is available to any other language, with the right functions. Most scientific languages come with those functions out of the box, as demonstrated by my 1<->1 translation of 'Life in APL' into Julia above. And APL doesn't give you a bunch of other really useful general-purpose stuff; that's why I term it a 'DSL'. It's a one-trick pony. It's a great trick, but it's ultimately not quite enough. That's why NumPy hasn't replaced pure Python - you still need to get your hands dirty outside the array paradigm from time to time, and APL is very primitive at that. In fact there's nothing to stop anyone from aliasing array-functions to their APL equivalents in any Unicode-aware language, like Julia (oddly, nobody does). What you're left with is a rather basic parser, some odd syntax quirks like arrow assignment, and some ugly imperative flow control.
Are the fancy symbols really worth it?
Engineers love it for prototyping, though, so maybe I just haven't worked with Matlab enough.
Makes perfect sense. Matlab is for engineers, not for mathematicians. They use computer algebra systems, proof assistants, etc. Difference is that engineers (and physicists) want answers and don’t care about how they are obtained, while its the reverse for mathematicians.
I think APL, although it, too, is a language for computing numbers, spiritually is a bit closer to mathematics than Matlab.
Since the computer is just a glorified calculator with memory (sorry Apple), we can fit the whole thing into a formal mathematical framework.
But that overhead is constant factor, more or less anything you can express well in matlab can be expressed straightforwardly in APL, too, if you have the right numerical routines. That's not true in the other direction though: there's a lot of stuff in APL you cannot express adequately in matlab at all. For example J (and these days Dyalog as well IIRC) have an operation called under which basically does this: u(f,g) = x => f^-1(g(f(x)). So you can write geometric_mean = u(mean, log).
It is completely impossible to implement something like "under" in matlab. Admittedly the J implementation at least of deriving a generalized inverse for an arbitrary function f is a somewhat ill-defined hack, but this is still something that is both conceptually and practically quite powerful. Also, whilst Matlab is really clunky for anything that is not a 2D array and hardcodes matrix multiplication as the one inner-product, APL has more powerful abstractions for manipulating arbitrary rank arrays and a more general concept of inner products.
Also, APL has some really dumb but cherished-by-the-community ideas that make the language less expressive and much more awkward to learn, e.g. the idea of replicating the terrible defect of normal mathematical notation where - is overloaded for negation and subtraction to every other function.
Have you seen the version used by dzaima/apl[1]? The equivalent of '(-&.:{:) i.5' works and results in 0 1 2 3 _4.
> APL has some really dumb but cherished-by-the-community ideas that make the language less expressive and much more awkward to learn, e.g. the idea of replicating the terrible defect of normal mathematical notation where - is overloaded for negation and subtraction to every other function
Klong[2] is a partial attempt to resolve this. I won't repeat the arguments in favour of ambivalent functions, as I guess you've heard them a dozen times before
> u(f,g) = x => f^-1(g(f(x)).
Other way round; it's g^-1(f(g(x)))
I hadn't seen Dzaima's APL, thanks! I like that he made a processing binding; APL always seemed like it would be such an obvious choice for doing dweet style graphics code golfing that I wondered why no one seemed to be doing it. A web-based APL would be a better choice though.
In that case you'll be wanting ngn/apl[1], which runs in a browser and compiles to js.
> ambivalent
The arguments are mostly linguistic. Natural language is also context-sensitive, so we are well-equipped to parse such formations; and they allow us to reuse information. The monadic and dyadic forms of '-' are related, so it's less cognitive overhead to recognize its meaning.
dzaima/APL being written in Java means getting it to run in a browser would be a bit hard, and ngn has given up on ngn/apl, but BQN[0] could definitely get a web canvas based graphics interface.
Somewhat interesting to add to the conversation about Under is that, in my impl, calling a function, calling its inverse, or doing something under it (i.e. structural under) are all equally valid ways to "use" a function, it's just a "coincidence" that there's direct syntax for invoking only one. (Dyalog does not yet have under, but it definitely is planned.)
I’m a little curious about this. Does J have a notion of the relationship between certain functions and their inverse? What is it that enables “under” in J which makes it impossible in Matlab?
Yes. Many built-in words have inverses assigned, and you can assign inverse functions to your own words with :. https://code.jsoftware.com/wiki/Vocabulary/codot
EDIT: and here's a table with predefined inverses: https://code.jsoftware.com/wiki/Vocabulary/Inverses
But interesting nonetheless.
>> syms x
>> f = @(x) log(sqrt(x)).^2
f = function_handle with value:
@(x)log(sqrt(x)).^2
>> f(x)
ans = log(x^(1/2))^2
>> finverse(f(x))
ans = exp(2*x^(1/2))
function u = under(f, g)
syms x
g_inv = matlabFunction(finverse(g(x)));
u = @(x) g_inv(f(g(x)));
end function y=odddouble(a,b)
y=2*x+1
endfunction
>>> h = under(@(x) 1/x, odddouble)
>>> h(3)
It is definitely not as elegant as the built-in facility in J, but definitely doable and usable in Matlab. In fact, I think any language with flexible enough function overloading should be able to implement such a feature.
f=:(1+*&2)
f 1 2 3 4
3 5 7 9
(f^:_1)f 1 2 3 4
1 2 3 4
(f^:_1) 1 2 3 4
0 0.5 1 1.5
People always say that such a thing will never fly in teams due to syntactic issues, but APL really is a productivity secret weapon for loners and small teams!
Following the same principle, R dplyr allows you to seamlessly interact with the DB by using a translation layer. However, every time I open R I find myself having to write tens if not hundreds of loc to shape the data where I'd do that in a few lines of J. For single researchers, it's actually much easier to read your one-page of J code 6 months later than it is for your 500-loc R script (again IMO).
Although I imagine it could be possible to really make a specialized APL geared towards data analysis as a strict DSL (APL is not a DSL). Meaning for example, making it more static and therefore statically compileable at the expense of losing things such as first-class environments (namespaces) or the "execute" primitive. One could also specialize the notation further towards statistics. There really is a whole realm of possibilities, here!
In a word, there is a market for lone scientists. It would be nice to have tools for that market ;-)
Commercial k variants come with a columnar data store (see Kx's q/kdb+, Shakti's k9).
I was thinking of more radical changes that would certainly disappoint APL purists by exchanging some well-known and highly-regarded APL capabilities for a more specialized language. Maybe I'll put my thoughts in code someday, if life doesn't get in the way.
The strength of APLs relies on having a fixed set of versatile primitives. From there, the terseness of the language allows one to write small but expressive and useful programs. This strongly limits the need for external libs and this is where the real value lies. Whereas in Python, you'd reach for multiple libs made in another language and often by other people, in APL you learn the prmitives and you're off to the races! Therefore, the fixed fraction of the programs you write (primitives vs functions) is much smaller (no libs with poor documentation) and you don't risk the carpet being swept from under you by changing lib APIs.
BTW J is stellar for data wrangling, and I encourage everyone to endure the multiple weeks of effort required to learn the basics of the language. Spend the time, it will be really rewarding!
J for C programmers is a good book - https://www.jsoftware.com/docs/help807/jforc/contents.htm - and it could take significantly less than many weeks to "get" some important ideas. Specifically, make sure you understand ranks at chapters 5 and 6.
After you understand how +/ with different ranks can sum along different axis, you're well on the way.
I mean, here is a cube of numbers:
i. 2 3 4
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
16 17 18 19
20 21 22 23
+/ i. 2 3 4
12 14 16 18
20 22 24 26
28 30 32 34
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
16 17 18 19
20 21 22 23
If you specify +/"0 - this sets the rank of the verb (function) to 0 - then +/ will be applied to each number separately and results will be combined. Adding a single number (not with itself - just as it is, without the other argument for summation) makes the same number, so +/"0 doesn't change the result - it's the same cube as in i. 2 3 4
Trying with +/"1 gives
+/"1 i. 2 3 4
6 22 38
54 70 86
Finally,
+/"2 i. 2 3 4
12 15 18 21
48 51 54 57
The book tells it better, of course.
What if you could invent a more efficient "notation" for chess? For example, FEN is a chess notation that is very efficient for computers. So maybe something similar exists for humans? Perhaps a three-dimensional notation, or perhaps a rearrangement of the board using knight moves and octagons instead of squares. Knights can jump in eight directions at most so a board using octagons would make it easy to see where they land.
E( E(X|Y) ) = E(X)
This is known as "the law of total expectation", and as a programmer this notation is so weakly typed it makes no sense. The more correct notation is
E_Y(E_X(X|Y))
If you can see that the outer E is summing over Y and the inner one over X, then the theorem is immediately clearer and very intuitive.
In the outer expectation, there’s only one choice: X no longer exists as a potential random variable (as if it were a local variable in the inner expectation) so the outer expectation must be over Y.
I’m not saying that you’re wrong that those subscripts could be used (they often are) but the meaning of the expression is clear after a little while working with expectations.
> Definition: The probability mass function (pmf) of a discrete random variable is the function p(a)=P(X=a).
This only makes sense if you already know what the definition is.
Even the abstractions we build into programming languages invites a certain way of thinking which is why there’s so many different paradigms like functional, imperative, logic, declarative, etc.