Plan9 from?
ultratechnology.com
ultratechnology.com
http://yosefk.com/blog/my-history-with-forth-stack-machines....
I adore Forth and feel that it's in the small handful of languages / approaches to language that will permanently change a developer's understanding of the field. Perhaps if GreenArrays had succeeded well enough to make a few more generations of processor the landscape would be very different. Perhaps not.
Given today's architecture, the impedance mismatch between the stack paradigm and the register architecture costs Forth most of its advantage. Still, if there's another language that allows one to write an interactive assembler for a system with 8k of RAM, I've yet to hear of it.
The minimum investment in Forth, which I recommend to all thoughtful developers, is reading Programming a Problem Oriented Language by Chuck Moore, Forth's inventor.
I'd also recommend looking at Thinking Forth if you want to get some more insight into Forth's philosophy, and really program structuring in general. It's also a good read even for people who don't have much practical use for forth in itself, many of the ideas are language independent.
Alternatively, if you're not really interested, just always use "whoever" as it's acceptable in all situations.
What an absolutely fascinating article. I wonder how useful Forth would be for implementing another language atop a VM (e.g. the JVM, CLR or Lua's VM — or WebAssembly).
A way to think about Forth is not as a solution, but a strategy: build the smallest abstraction that works, then build a second abstraction that refines things slightly closer to the domain problem, but explicitly compiles to the semantics of the previous one. If you do this several times you can get a very powerful environment quickly. But it takes time to reinvent the world from scratch, and shipping a project still usually leads to taking on dependencies. "Pure" Forth is just a matter of applying the approach directly to the hardware, resulting in a barebones, loose-cannon system that is clearly very powerful, but too dangerous to use out of the box.
Traditional compiler technology, in contrast, exchanges a big set of dependencies(a compiler, a build system, a module system, etc.) for a predefined semantics, type system, error checking, etc. These things are really useful for applications, but such languages can never be quite as direct since they are general-purpose. The Forth approach is still useful always.
But as a programming language to work in? Well...
- Forth has no (well, very little) runtime error checking. If your stack underflows, and you're lucky, it'll halt and tell you. But get anything else wrong, and it won't notice, and it'll just silently and invisibly poison its internal state and then crash later in a way that's really hard to debug.
- The language is strongly typed. It just doesn't have any type checking, and if you get the types wrong, it'll silently and invisibly poison its internal state and the crash later etc. You have to remember what type of object you push onto the stack, because you need to use the right kind of word to consume the object again. Does this word push a float? You need to know, because you need to use FDROP instead of DROP to consume it.
- The main culprit here is the compiler, which passes internal state around is by pushing objects onto the compiler stack. e.g. the BEGIN word, which starts a loop, pushes a 'dest' object onto the compiler stack. You have to consume this with UNTIL, WHILE or AGAIN. If you use the wrong word, or exit the definition without consuming it, it'll silently and invisibly poison its internal state etc. So you're now mentally juggling three different stacks: the data stack, the return stack, and the compiler stack, and if you make a mistake anywhere you're going to have a bad day.
- There's a lot of trivia to remember, which you need to remember. EXIT is allowed inside a BEGIN..AGAIN loop. But it's not allowed inside a DO..LOOP loop, because DO..LOOP loops are permitted to store a state object on the return stack. I will retrieve the index of the innermost DO..LOOP loop, and J will retrieve the index of the next DO..LOOP loop... but not if you've pushed something onto the return stack yourself inside the outer loop. If you don't get it right, it will silently and invisibly etc.
- Forth's only really intended to work on bare metal, and this shows. e.g. data space is assumed to increase from HERE up to the top of memory, and there's no facility for disjoint data spaces. The way you define data is to take a pointer to HERE and then just write stuff. All that stuff is defined to be contiguous. There's no scope for, e.g., running out of heap and allocating another block from the operating system. Oh, and try not to run out of memory, because it will etc.
- Stack juggling gets old very, very quickly. (Although some Forths, not mine, support local variables.)
Where Forth really excels is as an assembler substitute, where you're will to focus down and concentrate on every byte of source and you're going to put up with a complete absence of error checking because you need the performance. It's certainly great there. Being able to fit a complete interactive environment into a tiny, tiny space is astonishingly useful for debugging. But I'd hate to have to write anything of any great complexity in it.
I have heard of type-checked Forths; here's one, although I notice it takes a lot of liberties with the language (like not really having a return stack): http://www.arestlessmind.org/2009/02/03/intro.html
But I've never used one. I should, one day; I'm willing to believe that it's a much more pleasant experience than raw Forth. In particular, an embedded Forth with strict type checking but no runtime overhead sounds amazing...
(Aside: the Jupiter Ace was a Z80-based home computer from the 1980s, about the same grade as the ZX81. It came with Forth. Javascript emulator! http://jupiler.retrolandia.net/ It's got games! In Forth! Load one, do SHIFT+SPACE to exit, then 'vlist' will dump the dictionary and 'list word' will decode a word.)
I don't think the standard prevents you from implementing that. If you want it, do it.
> The language is [WEAKLY] typed. It just doesn't have any type checking
True. However, a linter that does some type checking seems possible, provided some annotation syntax for the hard cases where it's not possible to infer types. The thing is, despite its long existence one doesn't see many "Lint for Forth" around, so we can guess people managed to solve this problem in a different way.
> So you're now mentally juggling three different stacks: the data stack, the return stack, and the compiler stack, and if you make a mistake anywhere you're going to have a bad day.
The "compiler stack" is never an issue unless you overuse the "metaprogramming" capabilities of Forth like you'd abuse macros in Lisp. I understand you might have had that impression if you spent more time implementing the interpreter than implementing programs.
> There's a lot of trivia to remember, which you need to remember. EXIT is allowed inside a BEGIN..AGAIN loop. But it's not allowed inside a DO..LOOP loop
Well of course you're supposed to know that the loop index is stored on the return stack. For a Forth programmer, it's as trivial and obvious as taking care of freeing malloc'ed memory is for a C programmer.
> There's no scope for, e.g., running out of heap and allocating another block from the operating system. Oh, and try not to run out of memory, because it will etc.
That's because it's typically not needed for various reasons. However if you really need it you can: - make all of your addresses relative so that your system can reallocate the dictionary space; in other words, make your VM run Place Independent Code (PIC). - create a library that allows you to allocate memory from the OS or the heap.
> Stack juggling gets old very, very quickly. (Although some Forths, not mine, support local variables.)
It's like saying FP is a hoax because you use mutability all the time. The problem is not in the language, it's how you use the language.
> While back I wrote a Forth interpreter. I recommend this to anybody interested in programming languages; you don't have to finish it
You should really finish and actually use it because you don't understand Forth yet.
Oh, and don't be afraid to discard standard compliance. It's a bad trade for you: you get all the constrains and will probably never benefit from the advantages.
It's interesting that you assume I never finished it.
There was DO LOOP there was FOR NEXT and there was BEGIN UNTIL
DO LOOP was from FORTRAN, FOR NEXT was from BASIC, BEGIN UNTIL was from ALGOL.
What one do we pick for Forth? This (DO LOOP) has two loop control parameters and it is just too complicated. This (FOR NEXT) has one loop control parameter and is good with a hardware implementation and is simple enough to have a hardware implementation. And this one (BEGIN) has variable number of parameters.
I've got a new looping construct that I am using in Color Forth and that I find superior to all the others. That is that if I have a WORD I can have in here some kind of a conditional with a reference to WORD. And this is my loop.
WORD ~~~ IF ~~~ WORD ;
THEN ~~~ ;
It would of course not be convenient to nest loops but nested loops are a very dicey concept anyway. You may as well have nested definitions. We've talked over the last fifteen years about such things. Should you have conditional execution of a word or should you have something like IF THEN? Here is an example where I think it pays well in clarity, the only loop you have to repeat the current word.
WORD ~~~ IF ~~~ WORD ;
THEN ~~~ ;
(Jeff) You might point out that your semicolon after WORD results in tail recursion and converting the call in WORD to a jump and that is how it functions.
(Chuck) So there is no reason to make that a call since you are never going to go anywhere afterwards so you just make that jump. In fact in all my latest Forths semicolon kind of meant either return or jump depending on the context and it's optimized in the compiler to do that. It's a very simple look back optimization that actually saves a very important resource, the return stack.
Seems to me that this replacement of the tail recursion with a jump is basically TCO.
(Which, I should add, mine doesn't support. My compiler is written in Forth, but it isn't compiler isn't compiled in Forth --- the Forth subset it's written in is precompiled ahead of time. With awk.)
Daniel Ingalls famously wrote in 1981: "An operating system is a collection of things that don't fit into a language. There shouldn't be one." [1] A few years ago a paper argued that Plan 9 was basically trying to achieve that Smalltalk vision. [2]
And if you think about it, that view is extremely fashinable today, with the rise of unikernels [3]. Granted, they are not used for Desktop systems as of now, but who says they won't be in a few years.
[1] https://www.cs.virginia.edu/~evans/cs655/readings/smalltalk....
[2] https://www.cl.cam.ac.uk/~srk31/research/papers/kell13operat...
People who grew up in the 8-bit era might remember the reverse of this mental shift when they first encountered the IBM PC, and were told they needed an operating system for it. Why is the OS an independent idea from the hardware I bought? Why do I need it? Why isn't everything I need built into the computer itself?
"Maybe I'm missing something?"
It flowed for me, I'll try to explain my reaction.
What we think of as an operating system or a language are specialisations of a common idea: they are mechanism for driving a computer.
Mainstream development practice is all about facading existing technology. You write a web gui that comes from a web server that is set up with a DSL. The webserver calls a perl script that shells out and calls perl libraries and makes system calls. And so on. And then we build complex tools to herd all of the complexity: nixos and node.js and IDEs and LLVM and systemd.
We have created human language to refer to these abstractions: operating system, programming language, compiler, package manager, IDE.
Forth is different. It's a technique for solving problems by eliminating cruft (rather than facading it). It doesn't have a whole lot of respect for facader language ideas.
There are similar projects working in the same spirit, but which have let the bedrock sit further out from the hardware.
There's a crowd who want to bedrock on an array language shell, and they build it in a bizarre dialect of C with a means-to-an-end attitude: that's the operating system project the kparc.com team are working on.
There's a crowd who want to bedrock on an easily-networked file-system+bitmap-display abstraction over the hardware, and who are fond of algol-like programming languages: that's plan 9.
There's a crowd who want to bedrock on linux's excellent suite of hardware drivers: that's suckless.org.
A dev who only lives and breathes IntelliJ might mark a vi-using suckless user as a minimalism extremist, be unable to understand their motive. Whereas a kparc developer will flag the same suckless fan a pragmatist but recognise the shared purpose.
The culture war over systemd is an example of the tension between facaders and minimalists. As the facade-the-world attitude has ramped up, Unix has gradually become less and less effective as a host system. If you accept the facader worldview, systemd is the appropriate response. But systemd alienated the minimalists. Before systemd, linux was effective for people who wanted an expedient shell into their computer with adequate hardware and browser support. Systemd added complexity to their bedrock.
Also now I've read a bit more, I can also see how Forth systems can be a bit like operating systems since they normally allow multitasking. So my initial comment was a bit rash (using a different account because I can't remember my password and not at home).
I must say that the diagram does remind me a little bit of TIS-100: http://i.imgur.com/pxA3H4N.png
Admittedly its not that novel of a structure, but still it looks like fun piece of kit to play with.
Otherwise, in general I think Forth has bit similar culture/community than LISP, even if they come from very different backgrounds. Both feel smugly superior compared to common C programmers and feel slightly bitter from being unfairly pushed into niche. Both highlight the minimalism of their idol and celebrate it by creating bazillion tiny implementations.
Seconded. Additionally, implementing Lisp in Forth (on a Forth CPU for bonus points) sounds like something one should try for the sheer coolness of it.
Amen.
I would like an APL using ASCII characters written in Forth.
The true beauty of Forth to me is the relative ease with which one can boot into a Forth REPL and start controlling hardware, without an "OS".
The idea is a computer with APL as its "OS". The project at kparc.com is closer to this idea than the one at cosy.com. What is evident in both cases that this idea has survived. And that's good.
In short, there's no money in it, which is the real takeaway from this article.
We tried the simple approach. It's a nightmare.
(Yes, on single application embedded CPUs simple is great. Those are disappearing faster than glaciers. Everything has different applications running in parallel nowadays.)
[1] The author of the article, Jeff Fox, worked for them. He died in 2011.
But if you just want it to be preemptive, you'll need a small supervisor, a heavier memory manager, and some context switches. You'll get about the same performance green threads get on modern languages.
Linux is millions of lines of C code already, so using Forth in that context feels a bit ridiculous.
But yeah, I guess for single units people will just shell out 10 euro more and use a Linux capable unit.
However there are options to use better languages than C or C++ for bare metal embedded development, but none of them tends to be Forth, rather Basic, Pascal, Ada, Java, .NET, Oberon variants.
Quote:
Forth does it differently. There is no syntax, no redundancy, no typing. There are no errors that can be detected. Forth uses postfix, there are no parentheses. No indentation. Comments are deferred to the documentation. No hooks, no compatibility. Words are never hyphenated. There's no hierarchy. No files. No operating system.
Forth shoves down your throat its ancient stack model and lack of higher order types.