Lisp is Abstract Syntax

Macros are good because they let you refactor repetitive bits of code that functions can’t refactor. With the extra refactoring, your program is simpler and easier to modify, because its abstractions are more apt.

The macro system can be extremely powerful and when abused, is a nightmare to decipher. But. In the real world, I've rarely ever encountered someone who has actually programmed in such a bad style. A macro to me is almost no different than a function declaration (ie. (func arg1 arg2)), I just have to bear in mind whether it's being executed at compile or run time. Frequently I find a macro unnecessary and replace it with a straight function definition.

My golden rule is to avoid nesting macros in my own code (libraries are a different matter - wrap 'em in error handlers). If I stick to that, it becomes very simple and very powerful. It's not a very strict rule, but it is a helpful one as it reduces the general level of abstraction to a manageable level, while still being useful. Due to Lisp's flexibility, I think you have to maintain stricter coding standards than you perhaps would otherwise (but also know when to break them). If you create the right macros, you can save hundreds of lines of code and make the codebase far more maintainable.

A good example I had recently was for a website; I had a template macro for all the <doctype>/<head>/css/js and had the navbar, footer, general layout in the template as well. Then I had each webpage defined as a function that called the template macro and then inserted the rest of the <body> into the page. (This saved a ton of code in terms of templating; it could probably be done in other languages too, but you'd have to squeeze it into OO format or something)

This was all well and good, I had about 20 pages done before I ran into a bug. In some other languages, you would then go round adding a debug line/condition handler to every function, but instead I added it to the main template macro just once and it handled all the functions. Even better, the macro system and condition handler allowed me to deal with the code both before and after it had been compiled. I then passed the error up the stack from the function to the macro's handler, had the handler choose a fix (recompile the page function with different params), then send it back to the function (without ever exiting the stack) and carry on as if nothing happened. The users see nothing.

I don't use a lot of the things Lisp can do (eg. I can't remember the last time I made a function generate another function and return it), but it works exceptionally well for what I do use (currying, composing, macros and more).

http://en.wikipedia.org/wiki/Homoiconicity#Examples

...does homoiconicity really mean anything more than a language has a "read" procedure available?

As far as Lisp's syntax itself, the lack of special cases means there's less to learn. Myself, I like some of the sugar you get in Clojure for hashes, vectors, sets, etc.

As for macros themselves, ostensibly you don't need to know the implementation details of every macro any more than you need to understand what executing a Java method does in terms of JVM bytecode. In practice sometimes you'll goof up and try to map a macro over a list. Depending on your Lisp implementation, the mistake should be obvious. And depending on your editor, it may even highlight some or all macros differently.

Also, Racket also has a different approach than some of the Lisp reader manipulation you see in Common Lisp. We're getting outside my familiarity here though.

For example a LET binding is following a (LET ((var1 binding1) (var2 binding2)) body) pattern.

Knowing a few C-style languages, a few lisp style languages, and a few ML style languages, I have found unsurprisingly that the language I'm using most becomes the easiest to read.

But since all we have is anecdotia and talks about our feelings, cracks knuckles, I'll give my two cents.

I think ML style is the easiest to _learn_ to read. It's sparse, simple, and almost always obvious. It's probably my favorite to read, and I usually find myself able to read it quickly.

My favorite to write and edit is a lisp style. With paredit and vim, I feel like I can edit code effortlessly. Every time I'm back in a C or ML descendant, I find I'm always wishing I could easily grab this expression or that parameter, and whoops, don't forget that comma there, nope this line it is a semicolon, nope this line it's a period. The irregular shape of the code with a different character for each concept (period for chained method call, comma for chained list or argument, semicolon for end of line, etc) makes it very hard to edit quickly. With so many different representations it is hard to add keyboard macros to help you structure code.

Specifically, in C# and JS, I always feel like I'm just pounding the text like a blacksmith, with sparks of commas, periods, semicolons, colons, brackets, square brackets, and angle brackets flying around. With lisps, it's usually just parens, and with paredit and Vim's in/a parens commands, they mostly work for me, not against me.

That same simplicity of representation is why macros are so fluid and simple in lisps. Can macros exist in other styles? Theoretically, yes, but all the issues I mentioned would be barriers in the way.

Basically, I think it's a marketing problem.

So this isn't ringing true for me.

I also recall Gerry Sussman speaking quite highly of Algol 60.

No, I think you're quite wrong: the reasons for the popularity of Lisp at the AI Lab have to do with its actual advantages over other languages for the kinds of problems they were working on.

And there is an example of a language in the MIT tradition with infix syntax: Dylan. Great language, but wasn't in the right place at the right time to catch on.

Perhaps lisp programmers like the idea of composition, and decades later we still have no useful method for composing formal grammars? When we instead use a uniform tree structure and replace syntax with vocabulary, the problem disappears, and we only need to be concerned with composing the semantics of different "languages".

Or maybe they also like being able to quote some tree structure and treat it as code, or quote some code an treat it as data too - without the need for some genius to create an "API" to serialize and deserialize the data, and re-implement the compiler/interpreter.

Oh, if you need a feature, you could always ask the developers of your formal syntax to add it to the language, then wait a few years for the next revision to come around.

How much flexibility does LISP really give you here? For example, say I wanted my DSL to use Python-style significant whitespace instead of brackets. Does LISP make that easy, or is my DSL restricted to using stuff that looks like S-expressions?