Common Lisp implementation of the Forth 2012 Standard
github.com
github.com
Forth is absolutely worth learning, if for no other reason than understanding how stack-based interpreters work. The CPython VM, as an example, behaves similar in some respects to the average Forth program.
There's definitely better examples out there, but here[1] is an implementation of Mage the Awakening 2e's (tabletop game) spellcasting system in Forth that I wrote to help with playing awile ago.
The old saying is "if you've seen one forth, then you've seen one forth". It isn't a language like Python, but more like a language philosophy.
The creator of forth (chuck Moore) wasn't a fan of standards and felt they led to bloat. He was uniquely lucky and gifted to where he would figure out the minimum implementation of both hardware + software to solve a problem and by hardware...I mean he designed his own chips. So at the end of the day, he had a holistic tool where the hardware and software were designed together to solve a specific problem. He wouldn't implement anything he didn't need. It's a fascinating process that I would guess wouldn't work for 99.999% of people as most of us aren't at that level of knowledge/efficiency.
Forth and Lisp are both super freaking cool.
I wrote a FORTH in 6809 assembly for the TRS-80 Color Computer running this OS
https://en.wikipedia.org/wiki/OS-9
which was very much UNIX influenced and could support three users logged in simultaneously. I wrote to the Forth Interest Group and got back a reference card for the standard and used that for a guide. The big thing I did differently was file I/O; at that point in time FORTHs often accessed individual blocks on the disk, your code editor would edit one block at a time, etc. If you already have an OS, text editor and such it makes more sense to provide an API similar to the C API for the filesystem so that's what I did and the standard caught up.
Might I interest you in the opposite? Sort of a Lisp implemented in Forth that can compile Lisp functions to native code, all of this in less than 500 lines of Dusk code.
https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/doc/co... http://duskos.org/
Thanks for sharing, I'll be diving into this for a while
I hadn't looked at comp/lisp before. It definitely looks like a real Lisp to me. In https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/mem/co... I see that you have dynamic typing, implemented in assembly, and from https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/mem/co... to line 70 you have a garbage collector. From https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/comp/l... I see you have closures, but I'm not sure how memory allocation for their captured variables works; from https://git.sr.ht/~vdupras/duskos/tree/master/item/fs/comp/l... I infer that you are just using dynamic scoping, thus ruling out both upward funargs and tail-call elimination?
What's the most interesting program you've written in comp/lisp?
None. I wrote comp/lisp because my lack of Lisp knowledge seemed to me like a liability as I invest more and more time in Forth. I had nothing I felt like writing in Lisp, so I thought it would be fun to implement one.
While I can't call myself a Lisp programmer, I think I can say now I have an idea of what it's about. It didn't grow on me, so I leave it aside for now. Maybe I'll come back to it later.
> I infer that you are just using dynamic scoping, thus ruling out both upward funargs and tail-call elimination?
Tail-call elimination is possible with my implementation within the same function, but it's not "natural". I'm not sure of what it would take to have it like we see them in other lisps, or if it's possible at all. It's been a while since I swam in that pond, details aren't fresh in my mind.
As for scoping, I'm unfamiliar with the theory. I don't think it's what we call "lexical scoping" because variable names have very little meaning in the compilation process. It's eliminated very early and only "compile time Parameter Stack offset" is kept in the AST.
I don't know what you mean by "upward funargs", but if it's to access arguments from outer scope, yes it's possible. That's exactly the point of the last link you pointed to: lambda generation. A call to a closure generates a lambda which has its outer scope "injected" to PS.
scheme@(guile-user)> (define (make-adder x) (lambda (y) (+ x y)))
scheme@(guile-user)> (define x+3 (make-adder 3))
scheme@(guile-user)> (define x+4 (make-adder 4))
scheme@(guile-user)> (x+3 7)
$1 = 10
scheme@(guile-user)> (x+4 8)
$2 = 12
In Scheme those bindings are even mutable: scheme@(guile-user)> (define (counter n) (lambda () (set! n (1+ n)) n))
scheme@(guile-user)> (define booga (counter 53))
scheme@(guile-user)> (booga)
$3 = 54
scheme@(guile-user)> (booga)
$4 = 55
This makes Scheme closures a fully general substitute for Smalltalk-like objects.Upward funargs generally require a non-stack-like allocation discipline for the local-variable bindings.
While implicit variable capture like Scheme is the most common way to do closures, it isn't the only way; C++ and old versions of Python (before nested_scopes) have a form of closures where you instead explicitly list the variables you want to capture. Old dynamically-scoped Lisps like Lisp 1.5 had a different solution which I don't understand; see AI memo AIM-199, "The function of FUNCTION in LISP".
Usually dynamic scoping is incompatible with tail-call elimination (except in the special case of tail recursion) because you can't eliminate a tail call if which you have more work to do after it returns; usually popping some variable bindings off a context stack amounts to "more work to do". Maybe your parameter stack has a way for that to happen implicitly, for example by implicitly restoring a parameter-stack offset on function return.
I hope this helps. Keep being awesome!
$ make run
stty -icanon -echo min 0; ./dusk ; stty icanon echo
Dusk OS
79KB used 31MB free ok
needs comp/lisp
ok
lisp (defun make-adder (x) (lambda (y) (+ x y)))
ok
lisp (defun x+3 (x) ((make-adder 3) x))
ok
lisp (defun x+7 (x) ((make-adder 7) x))
ok
lisp. (x+3 7)
10 ok
lisp. (x+7 8)
15 okhttps://jxself.org/git/?p=spiritwrak.git;a=summary
You can get inform6 from the same page of the repos, and informlib by cloning spiritwrak with
git clone --recursive https://jxself.org/git/?p=spiritwrak.git;a=summary"Open" here was a popular term back in the day and bunch of companies used it .. sorta like AI, LLM, and blockchain.
Genera did a few cool things but in the grand scheme of things it is essentially not much (the Dynamic Listener probably being the most interesting thing, the rest of the system was essentially the same).
The MIT version of the Lisp Machine is free software without any legal conundrums. So there you go …
not true
System 99 has support for Common Lisp. System 301 will hopefully have better support than Genera (seeing that it doesn't really support CL very well anyway) at that.
The Document Examiner is a separate system (and LMI did it differently, with Gateway), and not part of the base System. X11, and NFS likewise. So listing a bunch of orthogonal systems isn't really what was being discussed.
I don't see much of that in System 99 from MIT.
Stuff like X11 and NFS is also not random stuff. Open Genera uses it for its console and its files.
> seeing that it doesn't really support CL very well anyway
Well enough that it is able to run a lot of its software written in Common Lisp. It's able to run stuff like PCL, ASDF, ITERATE, ...
System 99 is from 1984 (?), Symbolics started in 81/82. Open Genera appeared 1992/93. You are ignoring a full decade of development.
Earlier:
> Genera did a few cool things but in the grand scheme of things it is essentially not much (the Dynamic Listener probably being the most interesting thing, the rest of the system was essentially the same).
I would guess that less than 20% of the core SYSTEM is essentially the same (but often quite a bit enhanced). For the rest of the software, I would think it's even less. Which is not surprising, since System 99 did not support the 3600 systems, the G-Machines, the I-machines (XL400, XL1200, XL1201, UX400, UX1200, NXP1000, MacIvory 1/2/3, ...), Open Genera, ..., because all that was developed later by Symbolics.
> Well enough that it is able to run a lot of its software written in Common Lisp. It's able to run stuff like PCL, ASDF, ITERATE, ...
No it is not. You are thinking of Portable Genera.
> System 99 is from 1984 (?), Symbolics started in 81/82. Open Genera appeared 1992/93. You are ignoring a full decade of development.
System 99.32 is from 1987, which continued via LMI into System 130. Genera, or before when it got forked, is from 1978 when it got forked from System 78 or there about being re-branded into System 200. All three (four if you count TI -- which did quite more in renaming things making it hard to follow) with parallel development going on from there into the early 1990s.
Genera is a Lisp Machine system based on the work done at MIT, where most of the guts still the same. To the point that how you rebootstrap the system is the same, how the core areas look work, how the scheduler works, how the windowing system works, how flavours works, etc.
Open Genera did very little on top of Genera at that, mostly targeting and making it work on the Ivory. You are purposefully conflating Portable Genera, Open Genera and Genera.
> You mention a bunch of things that are also _not_ part of Genera
Of course they are. I have the thing right in front of me, running. I'm typing to it. I'm looking at the system definition of SYSTEM 501. On a live running Genera 9 on a VLM. If I would look at my MacIvory running Genera 8 it also would not look much too different.
Stuff like EH has been long superseded. The Scheduler has been redesigned & rewritten. TV has mostly been superseded by Dynamic Windows. The Garbage Collector has been extended, by new GC variants.
> Open Genera did very little on top of Genera at that, mostly targeting and making it work on the Ivory.
Open Genera does not work on the Ivory processor. It's a Virtual Lisp Machine.
> Genera is a Lisp Machine system based on the work done at MIT, where most of the guts still the same. To the point that how you rebootstrap the system is the same, how the core areas look work, how the scheduler works, how the windowing system works, how flavours works, etc.
The Scheduler works different (the old scheduler polled, the new own is event driven), the window system is now Dynamic Windows, Flavors has been updated to New Flavors & the new CLOS (the Common Lisp Object System), ...
ams's hyperbolic perhaps point is that genera is significantly SYSTEM, that symbolics contribution is a kind of obvious extension of the grand vision that was already there in its totality and potential in the MIT's work. I think it's a valid argument, which I don't think can be resolved just by listing names of subsystems.
for example you can't just say "oh they replaced tv and window with dynamic windows", because dw uses both tv and legacy, for lack of better term, window. if you look at the flavor definition of basic dynamic-window it uses tv:stream-mixin tv:graphics-mixin and tv:minimum-window. and tv minimum-window is a venerable SYSTEM flavor. not to mention that other systems (like zwei) still use tv window directly. how thick a layer dynamic-window is on top of tv? answering that question require systems level knowledge and investigation.
other symbolics extensions are of similar nature.
I'm not sure what you are talking about. It's not related to what I wrote.
> how thick a layer dynamic-window is on top of tv? answering that question require systems level knowledge and investigation.
Dynamic Windows introduced a new UI look and feel, different from the old "static windows". DW has new APIs even for reimplementations of old UI features from TV , like the new drawing interface in the graphics package, which replaces the old TV flavor messages, with generic functions. It has also a lot of new features, like the presentation system. Later applications typically will use the new DW interfaces and new features. Both DW and TC are documented in "Programming the User Interface", with DW providing much of the high-level application window features, described in the first chapters.
There is another version of it, in another implementation, which is CLIM, which is then based on CLOS. Even later applications were thought to use the CLIM interfaces, to be able to write portable user interfaces, able to run on various other Lisp systems. Both DW and CLIM are substantially different from TV. Neither DW, nor CLIM are in the MIT software.
Other symbolics extensions are of similar nature.
Sure, and those extensions can run on the MIT Lisp Machine.
But we are taking about the base system. Not extensions. And why one can take those systems and for the most part just run them in another system is cause they are so similar!
The reason is as I said before, Genera is layers on top of an existing system that hasn't changed (much).
E.g, DW is of flavours on top of TV (with lots and lots of extra additions .. additions are easy to handle -- after all the DW code is like 1M of Lisp from 1989 or whatever).
The Dynamic Listener was more complicated to rip out, but mostly work. The hard part, at which I gave up, was adding all the presentations for objects and other such.
DW/CLIM treated TV as one possible driver, if not sidestepped it, the problem was that some software (iirc mainly related to some S-Graphics products) had some hardcoded TV dependencies from earlier versions - it's mentioned as part of the porting plans for OpenGenera, because OpenGenera has no working TV subsystem at all, because TV didn't work over X11.
But that was the case on other machines, too. The embedded UX400/1200 machines and the headless NXP1000 use an X11 console, too.
Lots of TV software does not run on these machines, because they don't have a framebuffer.
Ivory support was implemented in Genera 7.3. Open Genera added the virtual machine to run on DEC Alpha hardware, it was Genera 8.3 + VLM, upgraded to Genera 8.5 with Open Genera 2.0.
ZWEI, TV and SI makes up the majority of the base system which is far more than "20%" percent, which is ignoring EH, WINDOW, and anything else that is essentially processor agnostic.
The target processor is not that important from the users point of view, to the point that Lambda and CADR shared the exact same source code, and microcode. The Explorer I was essentially a Lambda. Much was "feature" protected with #+/#- to the point where many of the systems worked on Genera as well.
So that seems at par with the CPU architectures that Symbolics made, which is "just" a change to the compiler .. and not the overall system .. which is what was being discussed. But yes, Symbolics did design hardware done by Symbolics and made sure it worked on their system .. not sure what your point is other than giving an incorrect view of history and what Symbolics actually did to the Lisp Machine system parts (Symbolics work was much more interesting when it came to other systems like the 3D stuff and what not).
> The target processor is not that important from the users point of view
It's important for the operating system and the processor was used for very different machine hardware: standalone workstations, headless systems, embedded in a Mac, embedded in a SUN, embedded in a network switch, ...
> Lambda and CADR shared the exact same source code, and microcode.
But not the 3600, which was a 36 bit architecture and not the Ivory, which was a 40 bit microprocessor. A lot of the core routines were different on a 3600 and an Ivory machine. The VLM was even different, running on a 64bit microprocessor, implementing the emulator in C and assembler. There is low-level functionality which has two or three different implementations.
The Ivory machines could be very different from a LAMBDA and a CADR, for example as an embedded CPU in a very different host machine (Mac & SUN). That had a lot of consequences for the operating system.
Nobody will use Genera without Zmacs or the error handler (aka Debugger)…
System 200 ran in the CADR and 3600. Using the same code base, with different microcode. With just bunch of #+ and #-.
And no, the target is not that important that it makes up “80% of the system”.
Your going into tangents and constantly raising neg things not being discussed, we aren’t talking about the host architecture but about the Lisp Machine system which is mostly agnostic of the target CPU. And between those two, Genera and the MIT system, are essentially the same when it comes to the base layer. To the point you can take the Dynamic listener or even DW and have it running on a MIT system without much work.
Genera 9 (which started development just a few years ago going of a system that hadn’t been touched for 30 years and under dubious legal situation) is essentially Genera 8.5 with several fixes.
Genera was carefully designed to keep compatibility with the MIT system. And that is one of the reasons why feature and code are so similar.
Nobody will run Genera without TCP/IP and a lot of other stuff.
The error handler is not the same anymore.
> you can take the Dynamic listener or even DW and have it running on a MIT system without much work
I think that's pure fantasy.
> Genera was carefully designed to keep compatibility with the MIT system.
I don't think that was a goal, given that the MIT system was not relevant anymore and the vendors all had their own forks. TI converted much of the old code to Common Lisp with a translator. Symbolics wrote most new code in Common Lisp.
Without an extensive implementation of Common Lisp incl. CLOS, basically nothing runs on the old system.
The times of a few conditional reader macros was soon over.
Does Portable Common LOOPS (PCL) run on the MIT Lisp Machine?
https://github.com/binghe/PCL/tree/master/impl
It lacks a port, but there are ports to TI and Symbolics. That should be simple.
Plenty of people still do, e.g. on a Ivory or 36xx machines.
> I think that's pure fantasy.
I ported the Dynamic Listener (with large parts of DW) a bunch of years back to run on MIT System 78.
> ... MIT system was not relevant anymore ...
I realise you like to raise Symbolics to the skies, but to say that the MIT system, which then was continued by LMI was "not relevant" is just making up history. Both systems where still heavily used until the demise of Symbolics, and LMI and portability was absolutely one goal since breaking peoples code was not considered nice.
> Does Portable Common LOOPS (PCL) run on the MIT Lisp Machine?
No idea, LOOP on the MIT Lisp Machine is the standard Loop, feel free to port it though -- patches welcome as they say.
However, he only actually owns it if Andrew Topping actually paid the Symbolics bankruptcy executor for it in full. (Not doing so would entirely fit Topping's character, based on his criminal rap sheet.) If Topping didn't fully pay for it, then it still belongs to whoever or whatever the successor of the Symbolics bankruptcy executor is; I think that was ABN AMRO.
Good luck finding any of that out!
It even appears there's ongoing development, but in some secretive fashion: https://hachyderm.io/@gmpalter/109553491209727096 (by the same dude who wrote TFA).
Could anyone more knowledgeable provide firther direction?
nah. what you are describing is Open in the present day, like OpenAI.
back in the day it meant "open standards, allowing cooperation between secretive competitors". IBM and Digital could both implement to an open standard instead of their traditional proprietary so their devices would interoperate.
This type of openness is what Microsoft liked to "embrace, extend, extinguish", meaning "we'll get kudos joining the interoperating consortium, we'll look cutting edge by adding features which would make us not actually interoperate but are useful so people will adopt them, then we will have killed off this threat to our monopoly"
in that same period of time, open source started to be called that because what was open was the source.
Open Genera in the early 90s then ran on OSF/1 UNIX (https://en.wikipedia.org/wiki/OSF/1 , renamed to Digital UNIX) on DEC Alpha. OSF meant Open Software Foundation ( https://en.wikipedia.org/wiki/Open_Software_Foundation ), a group propagating "open standards" for UNIX.
Instead of having slow-ish boot times in a Lisp machine, GNU's plan was to put several memory-protected concurrent interpreters under Unix, giving the user far more power by default with GNU/Hurd against a common Unix users. WIth namespaces, for instance, you could forget about suid and/or needing group perms to mount a device or access different media and soundcards.
Not free. This is useless for preservation. Just compare it against Medley/Interlisp.https://github.com/codr7/cl-scrl
It is so great tool, even just for stretching your brain cells..
http://www.drdobbs.com/blog/archives/2010/09/forth_love_if_h...
but the Wayback Machine is down so I can't check.
I have been designing some sort of "persistent" Forth OS, that boots into an interpreter and persists your defined words across sessions and reboots, so you can incrementally build your own personal computer from scratch.
How exactly?
This sounds like the opposite of having the more moldable environment.
F83 implements an assembler, a memory hex dumper, an interpreter, a sort of bytecode compiler, a decompiler, a single-stepping source-level debugger, a screen editor (which could jump to the definition of any function and jump back and forth between source and documentation, and which tagged each screen of source code with the last edit date and author's initials), cooperative multitasking with thread-local data, a print spooler, and virtual memory. It can also compile programs into standalone executables, and in fact it's written in itself, so it can cross-compile itself for other architectures and operating systems; it supported CP/M-86, MS-DOS, and CP/M for the 68000.
The Forth programming language it implements is low-level but relatively powerful; among other things, it includes structured control flow, Turing-complete compile-time macros, closures, pointers, and multiple return values.
This is nothing unusual for a Lisp system, of course. What is different is that F83 is only about 2100 lines of code by my count. That's what makes it "more moldable" than a Lisp system with the same features, which would require around an order of magnitude more code.
There are some major compromises made in pursuit of this level of minimality, though: no type-safety (not even to the extent of calling functions with the right number of arguments), no built-in structs (though you can build them with its macro system and closures), no bounds-checking on arrays, error-prone manual access to the virtual-memory system, no syntax, no stack-allocated named variables, etc.
Probably the most influential use of XLISP is as AutoCAD's built-in scripting language, AutoLISP, but it's also the basis of Nyquist.
They had a good run, and they're still fun to play around with and useful for teaching and stealing ideas from. But the world has moved on in too many ways.
There's still a lot of ground to explore between Forth and Lisp though, and plenty of modern ideas to steal from scripting languages.
(not its standard sure)
I prefer Lisp for tricky problem solving, but it's unfortunately a no go professionally outside of prototyping ideas.
https://github.com/azzamsa/awesome-lisp-companies/
for some, prototyping goes to production: https://blog.funcall.org/lisp%20psychoacoustics/2024/05/01/w... (this year example)
Thriving as recognized by the general public, no, but do many implementations improve and are libraries released for practical matters? Yes.
wouldn’t this normally preclude doing any actual work with it?
Most people playing around with this will be using sbcl though. It's overwhelmingly the most popular free common lisp system.