There's also this; a fascinating study that investigated the ways people intuitively try to articulate algorithms when they haven't been biased by experience with existing programming paradigms: https://dl.acm.org/citation.cfm?id=373422
Of course, this means nothing pertaining to a preferred language, formal or natural, for encoding it such notions.
>a fascinating study that investigated the ways people intuitively try to articulate algorithms
Try to articulate algorithms. And generally fail, because they don't have any experience expressing complex systems in their shoddy, careless notation.
Naturally, when the physicist sits down to express her algorithm using Lie theory and tensor calculus, she's really a moron because one should obviously grab an inexperience lay person and have them dictate your notation for you. The natural consequence of this is that you'll see things you couldn't have seen before and discover many novel and interesting structures in the language, through a process of lay magic.
Just as using a more computer-compatible language (assembly, at the extreme) more optimally utilizes the computer's available resources, using languages with cognition-compatible paradigms more optimally utilize the brain's available resources, reducing cognitive load and enabling anyone - intelligent engineer or layperson - to express and mentally parse greater and more complex concepts than they otherwise would have been able to.
The power of the Lisp syntax comes from the fact that everything looks the same, but this also makes it harder for a human to parse. For example, using different symbols to wrap function arguments than you do to wrap procedural blocks gives the brain something to latch on to, reducing the cognition necessary to differentiate between the two. This is also why code editors make the same types of tokens the same color.
Two big takeaways from the linked study were that event-based programming and set-based operations are intuitive. These aren't layperson schlock, these are real paradigms. Every notation, every language, is made-up. There's nothing fundamentally more true about one or the other; each offers different metaphors for expressing complex concepts. The question is what those metaphors should be, and in my experience many programmers don't think about general human cognition when deciding which to prefer.
If you stick to intuitive operations, you'll suffer a lack of perspective. You never expand your intuition. Better metaphors are those that more closely encapsulate the facts of the problem at hand, not those that more nicely fit human mental models. The human mental model is wrong by default. Aristotle's intuition about gravity was wrong, and a significant amount of damage would be done to force discussion of gravity to match what he found intuitive.
People can learn new notations. They do it every day. People can adapt to complex metaphors. But complex problems will never simplify themselves to fit human preconceptions and intuition.
If you're dealing with a set-like problem, then a set-based language is good. If you're dealing with an event-like problem, then an event-based language is good. Human intuition is not an important factor, save for our bias toward seeing a given problem as set-like or event-like. An arbitrary problem may be better modeled using another frame of mind. But you won't see that if you're requiring that problems be solved using 'intuitive' language.
To be doubly clear: I'm not saying that unintuitive, highly-formalized syntaxes are useless, just like assembly isn't useless. Each is crucial for certain uses. What I am saying is, they shouldn't be necessary for (and often aren't even well-suited to) solving the average programming problem. We don't write software in assembly any more, but we haven't evolved as far beyond its paradigms as we like to think.
There is a huge number of problems that software engineers work on every day, whose domain (what's being modeled) is fully understood by many laypersons (or nontechnical subject-matter experts). And yet those laypeople lack the ability to express their ideas to a computer, and those engineers waste huge amounts of effort translating those simple ideas into needlessly esoteric code. This is a fundamental failure in language design, and it needs to be addressed.
Lol, like what? Go have a layperson check if a file exists. Or pass a substring of UTF-8 to C function. Or check equality of two floating point calculations. Or multiply two signed integers together. Or fix anything that doesn't work due to performance problems.
And when they do all these wrong, ask them to show you how to debug and correct a useful but 'simple' program in production.
Lay people can't express their ideas to a computer because their ideas don't work in a computer. The code is esoteric because logic is esoteric, and the human brain is just not good at reasoning about edge cases without years of practice and experience.
>I'm not saying that unintuitive, highly-formalized syntaxes are useless, just like assembly isn't useless.
Ok, good. So why did you object to the existence of LISP? Why would you say it is a mistake?
In any case, my overall point in all of this is that there's a difference between simplifying a language and simplifying the learning experience. We should always be looking to make things easier to learn, but that doesn't actually require changes to the language. You can do that by finding better ways of teaching, better documentation, and better explanations for how and why things work.
I've yet to see a better approach than Ruby's.
I don't necessarily care for the data as code, most of the time; but I do like that most everything has a sensible way to serialize to a file. Even better, unless I am wanting to get optimized, the round trip can take care of itself.
Curious what the syntax is for a basic list of key value pairs? Well, it is the same as any list of pairs. Which is the same as any list, unless you are trying to get fancy.
This freedom really is liberating, at times. Yes, some folks have made macros so that they have a syntax to have a different literal for different styles of maps/tables/etc. By and large, life is easier if you don't use those and just stick to the minimal syntax necessary to do this.
Which really helps with your concern. When I can see what the transformation is, I can literally substitute into my mental model what the macro is doing, so that I don't have to juggle a bunch of crap around.
And all of this, oddly enough, lands you into not caring if you are calling to a function or a macro. I have more trust in calling things in most lisps I've worked with, because I can understand both ways in terms of themselves easily enough. With ruby, I'm often wary of using a new method in a source file, because I really can't trust all of the interactions that are assumed for it to work out.
I find that the language is flexible enough that I never really reach for meta programming when tackling problems.
Whenever I pull in a new gem, I usually budget a few minutes to wrap my head around the semantics of how the gem is expected to be used. Occasionally the documentation isn't good enough and I need to source dive.
What I never have to deal with is new syntax. No matter how Ruby is being used, the syntax is always the same comforting, pleasing-to-read near-prose. What changes when you use metaprogramming in Ruby is semantics. You can't add new syntax to the language unless you actually change the language.
Like you, I find people misusing metaprogramming. For me usually this is fairly easy to detect. If I look for the source of a method and it's dynamically generated, then showing the source will take me to where it was generated.
I won't pretend to have chosen Ruby over Lisp because I liked the language better. I chose it because I figured it would help me get a job better. But as time went on I found myself at home with Ruby.
I don't think it would take me all that long to get used to any given Lisp. But I don't believe it actually gives anyone superpowers, at least, not superpowers you can't have with Ruby. I have yet to hear a lisper actually articulate a solid advantage for code as data.
When there isn't any functional reason to choose between alternatives, all there's left is the aesthetic. Which is where all those parenthesis finally become important.
But again, in lisp, you probably make use of more metaprogramming than you'd realize, precisely because it is indistinguishable from normal programming.
(I'm tempted to contrast this with some of the DSLs I've seen in the likes of scala and ruby... I have a hypothesis that all of the hoops you have to jump through to make a macro like construct cause people to go all in when they do so.)
Now, I'm also not claiming lisp gives superpowers. Again, I'd almost go the other route. Most lisp is typically quite simple, because you can accomplish quite a lot that way.
Which does bring it back to aesthetics some. The parens just flat out don't bother me. They are a bemusing joke, that never actually made sense. Too often, the difference is minimal compared to most C programs. Foo(bar) becomes (Foo bar). Which isn't that big of a deal. The basic math changes from infix to prefix is a bit to get used to. However, I find it is much more pleasant to deal with generics (+ matrix1 matrix2) as opposed to matrix1.plus(matrix2) of java and the like. Of course, I was also a huge fan of my HP calculator in college. RPN for the win! :)
So, yes, I do think it comes down to preference. I don't think you're preference is wrong/bad. I do think your view of lisp sounds colored more by bemusing jokes than actual problems in lisp code. But, I fully grant my experience with ruby may be colored by an overly ambitious project written in it.
I don't think we have any points of contention left to argue over, but I will say that many Rubyists just aren't familiar enough with Ruby's object model and scoping to be able to do DSLs cleanly. It's certainly possible, but there's a learning curve.
Actually I can articulate how to do DSLs in a paragraph. Create a DSL class where all instance methods are DSL calls that set instance variables. The instantiator takes a block, the DSL input, and simply runs instance_exec on the block so it's executed in instance scope. Hand the DSL object with all the ivars to something else to actually do things with. Nothing in DSL behavior should do anything other than set ivars. The only metaprogramming is instance_exec.
To that end, thanks for keeping the discussion going! This is a social site, so it is fun to flat out be social. :)
The format function can take a function as an argument in place of the format string.
The formatter macro can compile a format string into such a function.
(format nil (formatter "(~a-~a)") x y)
Why don't you consider macros like DEFUN, DEFCLASS, COND, etc to be "syntax breaking"?
Cond might. But really, if I said to prepare a list of test to action pairs, it has the somewhat obvious syntax to execute the first one with a true test.
Pcond is a bit crazier. But you can go a long way without ever touching that one.
Defclass is powerful, though again, you can go a long way without it. And if you are in a codebase using it, it is probably a touch on the ubiquitous side.
Loop is just famous for having a ridiculously powerful language inside it. Much more so than the others, which, quite frankly look like normal function calls most of the time.
defun could be like this, though its ANSI CL specification is quite screwed up.
The sytax is given as:
defun function-name lambda-list [[declaration* | documentation]] form*
"For example, the expression
(x [[A | B* | C]] y)
(x y)
(x B A C y)
(x A B B B B B C y)
(x C B A B B B y)
(x B B A A C C y)
(x C B C y)
(defun name (args ...) decl doc decl decl decl body...)
(defun name (args ...) doc decl decl decl decl body...)
(defun name (args ...) decl decl decl decl doc body...)
Note that if we interpret A|B*|C as a simple regex notation, it wants all the B's clumped together; someone really had to work hard to concoct this counterintuitive BNF extension.
The general shape of a LOOP expression is just a list of symbols and lists. A complicated LOOP expression that uses many features at once doesn't end up looking very Lispy, but a shorter one like (loop :repeat 10 :do (print 'hello)) shouldn't look at all foreign, especially if you write the keywords as keywords, which many people do.
I guess when people talk about the regularity of Lisp syntax, I think they usually mean that (op ...args) applies the operator OP to the list ARGS. LOOP obeys this very well: the only LOOP syntax I can think of that breaks it is `using (hash-key k)` or with HASH-VALUE. Universally accepted macros break this rule all the time, though. By shoving everything into lists, they totally abandon trying to keep with the (op ...args) form. If you see that regularity as a great advantage of Lisp's, then it seems like you should be much more bothered by the more "mundane" macros than you should be by LOOP.
(loop :for i :below 10 :do (print i))
(dotimes (i 10) (print i)) when <condition> do
for <var> in <list>
for <var0> = <init0> then <step0> and
for <var1> = <init1> then <step1>
collecting <expr> into <var>
edit: Oh, right, destructuring, too! I guess that also counts.
Out of curiosity, how would you feel about a macro like this (ignoring that it doesn't enforce any keyword order)?
(defmacro for (var &key = (then =) while do)
`(do ((,var ,= ,then))
((not ,while))
,do))