Hacker News Reader

Comment by big_spammer | Hacker News Reader

> Smalltalk doesn’t need macros because it has classes instead.

I'm not sure this is true. Surely any programming language that lacks macros would be more powerful with them.

It might be more accurate to say:

> Smalltalk doesn’t need macros because it has classes, powerful introspection capabilities, and simple expressive syntax (especially blocks) instead.

There's a debate to be had if compile-time macros are superior to passing blocks as arguments. It also is easy to make your language parser extensible or easy to modify without having traditional lisp macros. Metalua does something like this.

Yea, that's a weird statement, however a better one is that Smalltalk doesn't need macros because it has a clean syntax for lambas that remove a common use for macros, hiding boilerplate use of Lisp's lambda. Beyond that, Smalltalk isn't file based, you don't edit some version of the code that gets compiled (and macro expanded) into some runtime version of the code you can only see through introspection; rather in Smalltalk you're actually editing the runtime version in a running image. Macros simply wouldn't fit into Smalltalk in any meaningful way and Smalltalk's syntax is pretty much already ideal for building DSL's without the need to clean it up with macros.

What makes both Lisp and Smalltalk interesting is that there's no difference between language and library; the constructs you create yourself are on equal footing with the ones most consider built in. Macros let you build special forms to control evaluation semantics, Smalltalk simply uses blocks [ ] to delay evaluation, and both languages are their libraries. Lisp has functions/macros, if/cond, etc; Smalltalk has objects used in a way you simply do not see in other object oriented langauges. Smalltalk has no if statement, no while statement, no reserved words beyond true, false, nil, self, super, and thisContext, everything else is library including all control flow constructs which are implemented with objects/classes/inhertance, and polymorphism.

They are both "pure" languages in a sense, and that pleases some people greatly. If you haven't programmed in Smalltalk, you really have no idea what object oriented actually means at a deep level. All of the popular so called OO languages are actually just procedural languages with hundreds of special keywords that have classes, but the languages themselves aren't build from classes and objects, they're procedural and defined by the compiler writer as special forms you cannot create yourself. Having objects, and being truly object oriented all the way down, are drastically different things.

> Smalltalk isn't file based, you don't edit some version of the code that gets compiled (and macro expanded) […] Macros simply wouldn't fit into Smalltalk in any meaningful way […]

You speak as if “macros” means “macros as implemented in the C preprocessor”. Lisp macros, as I understand it, operate on the parsed syntax tree, not on the file text level, and are expanded at runtime, not compile time.

The key to understanding macros is to be quite clear about the distinction between the code that generates code (macros) and the code that eventually makes up the program (everything else). When you write macros, you're writing programs that will be used by the compiler to generate the code that will then be compiled. Only after all the macros have been fully expanded and the resulting code compiled can the program actually be run. The time when macros run is called macro expansion time; this is distinct from runtime, when regular code, including the code generated by macros, runs.

http://www.gigamonkeys.com/book/macros-defining-your-own.htm...

This is the view of a compiled execution. Lisp can also be interpreted from source as data. Then macro-expansion time is interleaved with execution time. Each use of a macro form may trigger a macro expansion.

Remember, a Lisp interpreter interprets Lisp source as data. Not byte code. Unlike Python, Java, Smalltalk, ... which all have popular implementations which compile to byte code and execute that byte code in a byte code interpreter, aka virtual machine.

Let's use a Lisp interpreter, here from LispWorks:

We define a primitive MY-IF macro. It expands into a simple IF use. But the macro will also count the number of macro expansions.

    CL-USER 46 > (defparameter *myif-counter* 0)
    *MYIF-COUNTER*

    CL-USER 47 > (defmacro my-if (c a b)
                   (incf *myif-counter*)
                   `(if ,c ,a ,b))
    MY-IF

LispWorks can trace macros, too.

    CL-USER 48 > (trace my-if)
    (MY-IF)

Now a simple function which uses our macro:

    CL-USER 49 > (defun fac (n)
                   (my-if (= 1 n)
                          1
                          (* n (fac (1- n)))))
    FAC

Now we use it and we will see the trace information for the macro use: you see the incoming form and the result form.

    CL-USER 50 > (fac 2)
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv NIL fenv ((#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv (#<Venv 275415194600  N>) fenv ((#:SOURCE-LEVEL-ENVIRONMENT-MARKER . #<COMPILER::FLET-INFO (NIL . #)>) (#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv NIL fenv ((#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    0 MY-IF > ...
      >> COMPILER::FORM        : (MY-IF (= 1 N) 1 (* N (FAC (1- N))))
      >> COMPILER::ENVIRONMENT : #<Augmented Environment venv (#<Venv 275416002360  N>) fenv ((#:SOURCE-LEVEL-ENVIRONMENT-MARKER . #<COMPILER::FLET-INFO (NIL . #)>) (#:FUNCTOR-MARKER . #<COMPILER::FLET-INFO (# # # #)>)) benv NIL tenv NIL decl NIL>
    0 MY-IF < ...
      << VALUE-0 : (IF (= 1 N) 1 (* N (FAC (1- N))))
    2

Let's see how often our macro function has been used to expand code:

    CL-USER 51 > *myif-counter*
    4

Is it not the fact that macros can be defined and, more importantly, redefined at runtime? If so, would that not mean that macros must also be expanded at runtime?

If you have compiled code, you have to recompile the code when you change a macro. It's not done automagically.

If you have interpreted Lisp code, then the code can use the new macro automagically.

My understanding is that typically, Lisp macros are expanded at 'compile' time. The compiler expands macros recursively until there's nothing left to expand and then compilation proceeds normally.

A Lisp macro operates on data structures. Macros transform one data structure into another data structure. Lisp code is a data structure (typically a list) and ultimately all macros get resolved by expansion to ordinary Lisp code and the expanded code then is compiled/interpreted.

Unlike C, Lisp macros have access to all of Lisp (including other macros) during expansion. So a macro's expansion can be determined programmatically by executing Lisp code...this part is where it gets harder to understand macros intuitively because the code called during the macro expansion phase does not have access to the code that runs at the runtime and vice versa.

That's why it is handy to think about macros as manipulating data structures...we don't expect data structures to return values or generate values in the environment by execution.

> Beyond that, Smalltalk isn't file based, you don't edit some version of the code that gets compiled (and macro expanded) into some runtime version of the code you can only see through introspection; rather in Smalltalk you're actually editing the runtime version in a running image.

That's not really true. Smalltalk is text-based, too, but hides it behind an integrated source management system. When you edit a method in a Smalltalk IDE, then you edit TEXT. The text then gets compiled to typically some byte code which gets interpreted by the Smalltalk virtual machine (which also might have some way to convert it to machine code).

If the text of the source code is not available, then Smalltalk needs to disassemble the byte code. But the disassembled byte code is not equal to the original source.

The sources are EXTERNALLY kept as text, outside the running system.

Just download your favorite Squeak and check out the contents. There is a huge sources file and there is a changes file. Those are text files with the sources and its changes.

This is actually different from some Lisp system, where the source actually is data inside the running Lisp and the Lisp interpreter runs this data. If you edit this code, Lisp then presents you a structure editor, which works on this data - not on text. It's not what a typical Lisp system does today, but it is still a possibility. Xerox' Interlisp used to use a structure editor for Lisp source code as data and a source code management system based on that.

This is different from Smalltalk, where the 'Interpreter' runs compiled byte-code and the byte code is generated from source code, which is actually text and stored outside the Smalltalk image. The Smalltalk image has then source code management data, like an index in each method which points to its external source.

Typical Lisp systems are doing the same. They record the source code location for functions and other things. If you edit the source for a method in a typical Smalltalk environment, it will retrieve the text for the method and in a text editor you can edit the text then. In a typical Lisp environment, the Lisp system will present you the whole text file and just jump to the definition using the editor...

GNU-Smalltalk is not just text based, but file based. http://smalltalk.gnu.org/

I'm well aware, I've programmed in Pharo daily for a decade; I Smalltalk for a living.

What I said it true practically speaking, you're getting into implementation details that don't make a practical difference. Lisp and Smalltalk are both image based systems, but Smalltalk'ers actually work on the running live image all of the time, they aren't ever booting from the sources file nor are they ever editing it manually (excluding the one file based GNU Smalltalk), it's pared with a changes file to enable the IDE to expose the current source of a method but those are hidden implementation details whereas I'm talking about the abstraction presented to the programmer. Lispers "can" do this as well, and sometimes do, but Lisp has a much more standard general workflow of editing files that are then read and launched to create/patch a runtime image it's not the normal worflow to work entirely in the REPL which is essentially what Smalltalk'ers do.

Consider the difference, what Smalltalker's see in their code browsers is a text version of what's actually running; if they had macros, which are nothing more than code generators, they'd be seeing is the macro expansion rather than the original source call to the macro. Certainly Lispers "can" do this as well, they can macro expand something to see what's actually running but as their primary mode is editing the original source they're accustomed to seeing the macro unexpanded, they might define an accessor like

    (name :accessor person-name
         :initform 'bill
         :initarg :name)

And from experience know that an accessor is a getter, setter, and backing instance variable but they don't see those things in their editors, they just know they'll exist at runtime; whereas a Smalltalker is accustomed to looking at the runtime where he actually sees the getter, setter, and and backing instance variable.

Macros just don't fit into Smalltalk; they've been added before, people keep trying it, and it just doesn't fit and so doesn't catch on.

> editing it manually

You edit the source code via the IDE. Just like in Lisp systems. It's just that the IDE works differently.

> Smalltalk'ers actually work on the running live image all of the time

That's the dominant way to work in Lisp, too. My Lisp Machine even runs it as an OS. I use LispWorks for development on my Mac - the IDE is the running Lisp system.

> it's not the normal worflow to work entirely in the REPL

Actually that's the default mode. If you develop Lisp code with SLIME / GNU Emacs, it talks to a live Lisp system.

> if they had macros, which are nothing more than code generators, they'd be seeing is the macro expansion rather than the original source call to the macro.

That's not how Lisp works. Macros are not simple code generators, where the workflow would be write macro code, expand that, and use the expanded code. The Lisp developer is not generating code and working with that generated code. The generated code is usually hidden and generated by the Lisp system on demand/incrementally for internal use.

> And from experience know that an accessor is a getter, setter, and backing instance variable but they don't see those things in their editors, they just know they'll exist at runtime; whereas a Smalltalker is accustomed to looking at the runtime where he actually sees the getter, setter, and and backing instance variable.

In this case the Lisp developer sees the generated objects: the getter, setter slot definitions. What the Lisp developer usually does not look at is the code generated by the macros. It might be useful to have source code for the macro forms and being able to edit them, but it is usually not done in Lisp. There are many macros, where there are no generated objects and the resulting code is extremely complex and large. Thus it would not make sense at all to present this code or edit this code...

> don't see those things in their editors, they just know they'll exist at runtime

I'll see it in the runtime. The introspective capability tells me that it is there. If I want to edit them, I would then go back to the source. For example when I call (ed 'foobar) to edit the accessor foobar, it would open up the defclass form in the editor. Some Lisp systems (like LispWorks)also offer the link back from the objects to the macro form which was responsible for creating them. What I usually don't want to see is the generated code, since that's not fit for human consumption.

> Macros just don't fit into Smalltalk; they've been added before, people keep trying it, and it just doesn't fit and so doesn't catch on.

Because they difficult to integrate into Smalltalk and its idea of an IDE. Macros add a lot complexity to the system and how the developers are using it. They offer code manipulation and the price is losing a direct connection of the object code / the objects from the source code.

> Actually that's the default mode. If you develop Lisp code with SLIME / GNU Emacs, it talks to a live Lisp system.

So do you dump your image at the end? My understanding is that most people work with files, and execute stuff occasionally while they're editing, then save the files. Next time, you reload from source - you don't start from an image with state. And if you fail to run something, your source and your repl become out of sync.

I've never seen anyone actually doing image-based development in a lisp, despite some lisps supporting it - even in livecoding (music), the standard is to edit a file, then occasionally send parts of the file to a repl.

> So do you dump your image at the end?

For example if you deliver an application, that's what one usually does.

> My understanding is that most people work with files, and execute stuff occasionally while they're editing, then save the files

One works with files, but together with the running Lisp system. I would work with a mix of compiled (fast load) and source files. Let's say you work on a new version of your graphics editor. You start the Lisp image and load the current version into it - mostly from fasl files. Some years ago people would more often have started a dumped image, because it took some time to load the fasl files. Even today, if the program is really large, you might want to use a dumped image of some version of it.

But independent of how you reach there, you start Lisp and then recreate a state of the working image (either from a dumped image or from loading files), where you continue to work from.

The image then will have a the development information, debug information, application state, compiler info, ...

In Smalltalk you would typically load a saved image more often. In Lisp you would recreate the state from a base image and then fast-load the stuff you need.

> And if you fail to run something, your source and your repl become out of sync.

Yes that happens and current Lisp systems actually don't have a goal to keep that in sync. You have to check manually that the source or the compiled system actually loads and runs - means recreate the correct state.

> I've never seen anyone actually doing image-based development in a lisp, despite some lisps supporting it - even in livecoding (music), the standard is to edit a file, then occasionally send parts of the file to a repl.

In the sense of Smalltalk (image + managed source code) there are only few people doing that in Lisp - nowadays. That debate (managed code images vs. a mixed file/image model) was lost in the early 80s with some later attempts to create a manage source model with code in data bases.

In the sense of saving images as pre-loaded code, that's seen and especially for application delivery. The LispWorks IDE I'm using has a lot of stuff pre-loaded (with some on-demand loading) as a single image. If I develop something, it is loaded into it and the application then is a newly dumped image. LispWorks also supports some exotic stuff like regularly saved sessions. It also keeps track which sessions have been saved based on which other sessions.

http://www.lispworks.com/documentation/lw70/IDE-M/html/ide-m...

> That's not how Lisp works. Macros are not simple code generators, where the workflow would be write macro code, expand that, and use the expanded code.

You're putting words in my mouth, and yes macros are just code generators. I didn't imply or say they're expanded and then devs use the expanded code. The whole point of them is you're not supposed to think about or see the generated code, the macro becomes the abstraction the dev works with.

> In this case the Lisp developer sees the generated objects: the getter, setter slot definitions. What the Lisp developer usually does not look at is the code generated by the macros. There are many macros, where there are no generated objects and the resulting code is extremely complex and large. Thus it would not make sense at all to present this code or edit this code...

Exactly the point I was making. You're actually agreeing with me though the tone doesn't come off that way.

> Actually that's the default mode. If you develop Lisp code with SLIME / GNU Emacs, it talks to a live Lisp system.

I said multiple times Lisp "can" work that way, stop being so defensive. But the point in fact is most Lisp developers don't work that they, they don't live in the REPL full time like Smalltalk devs do, you load from source quite often in comparison to the 0 times Smalltalk'ers do.

Look this isn't a competition; Lisp is technically superior to Smalltalk in features both due to macros and due to CLOS and its multiple dispatch OO system. Lisp was the inspiration for Smalltalk, Alan Kay goes so far as to call eval/apply the Maxwell's equations of computer science. However, I any many others having tried both Lisp and Smalltalk, prefer Smalltalk largely due to the syntax and dev environment. Emacs and Slime just don't cut it for us. Generic functions don't feel like object orientation, the way Smalltalk does OO just feels better and the dev environment feels better than anything else out there.

On living in the REPL full time and having a better environment, is there something you wish Smalltalk could do to become a better environment that it doesn't?

What would feel better?

Yes, several, it's slightly too dependent on the mouse for focus; I'd rather never touch a mouse. I wish it used native windows (this is specific to Squeak/Pharo) so my editor felt like any other editor rather than a window within a window that it currently is (some other Smalltalk's do this but they're windows based and I'm on Linux). I wish the code browser had a back button so I could go dig into the source of something and then come back rather than popping a new browser to investigate that thing. Even better I'd like bookmarks like you get from more modern IDE's where you can mark a few key spots you want to constantly be jumping between rather than having a different browser open on each one.

> I said multiple times Lisp "can" work that way, stop being so defensive. But the point in fact is most Lisp developers don't work that they, they don't live in the REPL full time like Smalltalk devs do, you load from source quite often in comparison to the 0 times Smalltalk'ers do.

That Lisp developers load source does not mean that the image gets restarted. Lisp usually is designed such that you can incrementally update/manipulate the running image. As I said that's the dominant development style for Lisp, AFAIK.

> the dev environment feels better than anything else out there.

That's what my friends who still use Symbolics Open Genera tell me all the time. ;-)

Reply on news.ycombinator.com