Show HN: A Lispy, embedded Forth | Hacker News Reader

Show HN: A Lispy, embedded Forth

github.com

61 pointscodr4life49 comments

I strongly believe concatenative languages – particularly statically typed ones – are one paradigm of programming that have not been explored nearly enough. It's good to see more activity in this area :)

Here's another interesting concatenative language in development that I'm following: http://kittenlang.org/

I dunno, it feels like Factor kind of exhausted it several years ago? I think the problem is that concatenative programming is super elegant for some things, and then falls on its face for others.

In particular the object/class system of Factor was baffling to me. The paradigms just don't fit together well.

I also think Lisp has an awkward object system(s). If you want that paradigm, I think a dedicated language is better.

Somewhat OT, but I'm curious what you think is awkward about CLOS?

Simple member access like

    object.foo.bar

is apt to read more like this:

    (class2-bar-of (class1-foo-of object))

Why not use simpler names?

    (bar (foo object))

I mean, I understand that "a.foo" (class A) does not represent the same name as "b.foo" (class B) due to scoping rules, and so logically you would have to use (a-foo a) and (b-foo b) if you wanted to replicate the same in CLOS. But on the other hand, FOO can be generic and applied to different types of objects. You might even have a mixin providing the FOO slot, which classes A and B inherit from.

I find the notation fine, it looks like regular function application (some people prefer threading macros, I personally don't care).

Note also that CLOS allows you to add code "between the dots" by specializing accessors on qualifiers (:before, :after, :around), which can be useful to add orthogonal things like logging, persistency, ...

That could work within a single, well coordinated project. Beyond that, the Common Lisp package system doesn't cooperate the way you'd like it to, so you're back at

    (libraryA:bar (libraryB:foo object))

Since the package system is not hierarchical and doesn't allow local aliases, if you're unlucky you get fully qualified Java-convention package names and end up writing:

    (com.myfavoritecorp.dept1a.project7.subprojectX:bar
       (com.biggercorp.dept32.project2.subprojectE:foo
          object)

> Beyond that, [it] doesn't cooperate the way you'd like it to

I know what I would like it to do. There are some additional features that people would like to use (https://github.com/fare/package-renaming), but even though the package system could be improved, I don't think it is so bad right know. It can be frustrating to try to use it in a way it wasn't designed.

I seldom use qualified symbols, so it is generally "(bar (foo x))" anyway. The case when fully qualifiable symbols might be the best solution is when you need symbols with the same name from different packages at the same time (without shadowing), which is quite rare IMO. Also, defining short packages as a facade for other packages is possible. When you develop an application, not a library, you can also choose to mess things up by renaming packages and adding nicknames.

By using unqualified packages, I can change the meaning of the code by editing only the package definitions (and recompiling): you code says `(foo x)` and originally meant `(a:foo x)`, but then you change a package and now it means `(b:foo x)`. For example `b:foo` could be a wrapper around `a:foo`.

Packages prefix help a little when trying to understand where a symbol comes from. For example, "json:decode" is a little easier to read than "decode". However, with the right editor (or just the REPL) you point to the symbol and have the information.

> Beyond that, the Common Lisp package system doesn't cooperate the way you'd like it to

I'm pretty sure that it is, and that generally in my package I would import bar from libraryA & foo from libraryB and not have to qualify the names.

It's only in the case that both packages export a symbol with the same name that one is forced to disambiguate.

I'd say that Lisp's package system is very close to perfect — it really does work well.

The CL package system allows local aliases. How do you think you can be in the default cl-user package yet refer to cl:cons as just cons?

Firstly, the exported symbols in one package can be made visible in another package through package use. Secondly, a symbol from package A can actually be made present in package B via import: http://clhs.lisp.se/Body/f_import.htm

Regarding packages, sometimes I wonder whether package names could have been symbols: that would allow for gensymed packages, as well as hierarchical packages.

I implemented a package system influenced by CL's. I thought of the idea of package names themselves being symbols (themselves interned in some root package that has no name, or perhaps has one of those symbols as its name). I couldn't come up with a good set of requirements that would make things better, so I didn't bother with it. It just seemed like self-refererence for the sake of self-reference, like some new appendix to D. Hofstadter's GEB. :)

I prototyped a reader-macro for CL that does this, specifically because it gives you package-local nicknames:

https://github.com/jasom/spm-reader

I once made a kind of "package Swiss Army Knife" reader macro for Common Lisp which lets you do cool things. It's reasonably well documented in a header comment.

It might be up your alley, since you're working on a package-related reader macro:

http://www.kylheku.com/cgit/lisp-snippets/tree/pkg.lisp

The context was: "FOO can be generic and applied to different types of objects." That only works if everybody uses the same symbol FOO. Library:FOO is a different symbol and won't be combined automatically, so you must package-qualify.

(Regarding local aliases, I was talking about local package name aliases, which could make it somewhat less awkward.)

Of course. Why would LOTTERY:DRAW and GRAPHICS:DRAW be smashed together?

but now you're back to (lottery:draw x) which isn't any shorter than (lottery-draw x), so the point about names not being namespaced to the object they are applied to remains.

If I have inherited both, I cannot use the symbol DRAW to refer to both; at most one of them can be just DRAW.

Simpler names is certainly less idiomatic (sibling comments have already mentioned why), so it's a fair complaint.

Note that the lisp equivalent of foo.bar(x).baz(y) would be something like:

(send (send foo :bar x) :baz y)) which can be made even more terse with a macro, doing something like (@ foo (:bar x) (:baz y)). This was how most pre-CLOS object systems in lisp worked, and CLOS supplanted this.

Implementing SEND in CLOS is trivial.

There are many people who experienced SEND style object systems and CLOS but I haven't found any of them to be detractors of CLOS, which is what prompted my original question; lisp had something much closer to smalltalk before switching to CLOS and if there is a group of lispers who want to go back to that, they are in a small minority.

One thing one could do is thinking of

    object.foo.bar

similar as

    1+2+3

The later would be

     (+ 1 2 3)

in Lisp.

Thus slot access could be written as:

     (? object foo bar)

All we need would be a macro ?, which is easy.

Alternatively we can write a reader macro such that we can write:

    #?object.foo.bar

The problem with short names and possible package prefixes remains:

    (? w1 ws:panes ws:background-color color:name)

vs.

    #?w1.ws:panes.ws:background-color.color:name

vs.

    (color:name (ws:back-ground-color (ws:panes w1)))

See my other comment; keywords allow for short names and there is no concern about GF collisions if methods are namespaced to the class anyways.

Yes, there is a concern about collisions when methods are namespaced to classes using non-packaged identifiers.

Collisions occur under inheritance. Example: you would like to add a member foo to a base class. The class is widely derived in lots of code, not all of which you even have access to, which might be injecting its own foo. C++ partially deals with this with hacks, like derived identifiers shadowing base class ones (which is totally non-OOP and breaks for virtual functions, which cannot be non-OOP by their nature). C++ partialy deals with the virtual function issue by treating the type signature as part of the name, so that foo(int) and foo(widget&) are different virtuals.

Collisions could be a problem even if there is just one class, but its contents are controlled by several parties. Every group has to watch out that they aren't reusing any other group's symbols in that class.

Any sort of automatic mechanism which can inject symbols into a class (e.g. in support of aspect-oriented programming or whatever) will run into a clash, unless it uses gensyms, which is inconvenient if the intent is to inject names that are to be publicly known and referenced.

Even though Factor is based on Forth, that's about where the similarities end. Factor was designed from the start as a stand-alone general purpose language, which is not what I'm looking for here; I'm fine with Lisp as a foundation.

Object systems crammed into Forth will always be baffling, the fact that it's doable doesn't make it a good idea.

I don't really buy into OOP at all; but i strongly prefer CLOS to alternatives, including Smalltalk; as it let's me reuse pieces of functionality without dealing with the rest.

Using a fundamentalist language as a foundation will always come back to bite you from my experience. I prefer my foundations as general purpose as possible, which usually means C or Common Lisp.

That's cool! I actually tried something like this myself, but ended up with a less conventional syntax.

I'm not really familiar with Forth, but my impression from writing a little bit of postfix code was that some pattern matching would make e.g. writing a large conditional much easier.

Forth is trivial to learn but takes serious effort to master, like Lisp and C. I'm a humble beginner myself when it comes to Forth. A match operator is a possibility, but too much of that will ruin all that's good about Forth. I already added word scoped variables, and I already feel like I'm balancing near the point of negative return.

That's something I'm working on, actually. I think matching the top stack item(s) fits well into a postfix stack-oriented language. For example:

  TYPE: Tree
    | Branch(Tree,Tree)
    | Leaf
    ;

  :: depth Tree -> Int ;
  : Branch($,$) depth {depth} dip depth + ;
  : Leaf depth 1 ;

$ is extract, so the pattern Branch($,$) pops the top of the stack and pushes the 2 members. Types and type constructors have to start with a capital letter, but I like the

  : Pattern word definition ;

syntax a lot.

> I've been craving for a trivial, embedded scripting language that feels just right for a long, long time; something I can quickly drop into any project that needs scripting without too much ceremony.

How are you quickly and casually going to drop in a scripting language into any project, if it requires Common Lisp?

Maybe this means "into any CL project". But why would you need a scripting language in a CL project.

The idea doesn't really require Common Lisp; the same thing could be implemented in C with a bit more effort. But for me, any project mostly means Common Lisp as that's the first tool I reach for. I need a scripting language because exposing the full host language is not really scripting, pretending so is a cop out that always comes with unintended consequences. Given the right definitions, Forth reads like the kind of pseudo code most non-programmers would write; and the lack of syntactic fluff makes it a viable alternative for scenarios involving sequential processing, templating etc.

So, serious question: if the property of a concatenative language is the ability to pass outputs to inputs of a function without specifying parameters, could Iolang or Smalltalk also be described as concatenative? Ie:

  Transcript show: 'foo'; cr.

In this case the output of "Transcript show:" is the Transcript object, which is used as the sender for cr.

The property of concatenative languages is not the ability to compose functions without specifying parameters, but that this in the only way of building programs.

Your Smalltalk example has a lot more special syntax going on, in a stack-based concatenative syntax it might be written as

    Transcript ' foo' show cr

assuming appropriate primitive programs "Transcript", "'" (for literal quoting), "show" and "cr"

That is a bit too exciting this early in the morning...

Sorry about that :) Time zones...I strongly prefer Lisping and Forthing after midnight, too much noise on all levels during days.

Yep, I live in the EU but I am in Cambodia now and I love three languages the most; Lisp, Forth and APL. So you know what to do to make me cry and have eternal enlightenment...

Edit; I use Lisp in the form of Clojure daily as well as Forth for embedded. When writing embedded code, instead of assembly (check my profile, the stuff we use definitely has no room for Node but often not even C) I usually port a Forth if one is not available, at least for testing but usually it is good for production too.

What resources would you recommend for getting started with Forth?

This is a more interesting question that it might seem. For a Sunday afternoon read, please take a look to Jonesforth [1] for an implementation of a basic Forth for Linux/32 bits. It's written in literate style and it reads like a nice book.

If you are not that interested in the implementation of the compiler (and if so you might be missing the ultimate point of Forth), Starting Forth [2] is a classic resource.

Edit: formatting

[1] https://github.com/AlexandreAbreu/jonesforth

[2] https://www.forth.com/starting-forth/

Charles Moore Thinking Forth is excellent if you want to work with resource limited systems.

The author of Thinking Forth is Leo Brodie.

From Chuck Moore, the father of Forth, "1X Forth" ( http://www.ultratechnology.com/1xforth.htm ) is worth reading before and after Thinking Forth.

You are absolutely right. Got my authors mixed up. Both are great reads.

I have a reasonably portable pure-ANSI-C Forth here in about 2.5kloc:

https://github.com/EtchedPixels/FUZIX/blob/master/Applicatio...

It's not well commented, but it should be easy enough to read. It passes the basic ANS Forth test suite.

The C file is also an exciting self-modifying shell script with embedded awk and Forth parts.

Heh. I like you. One of my many language projects is an array-oriented Forth-like language. I've iterated on the idea several times, I think it's really great. Unfortunately, I have yet to capture the beautiful simplicity and elegance of Forth or APL in any of my projects.

Prolog is another one of my favorites, you should check that out.

Second Prolog, I have a feeling the same kind of fusing can be accomplished as with Forth. I've seen a couple of attempts, the most recent being in Clojure; but none of them hit the sweet spot for me. Logic by itself doesn't really help any one; but fused to a swiss army chain saw, it's very useful.

I did my degree at UvA which is the home of SWI Prolog and I studied AI. But now you really upset my sore brain ;) I am going to say Mercury and then step away.

Back in the day (early 90s) there was something called ASYST which was indeed a array-based Forth on steroids. It had a nice library and could make plots interactively. It was not elegant ;)

You must be my long lost twin ;) I went from Basic, Assembly and then C to Lisp, Forth and then APL, well J [0], but then some APl (Dyalog). I have decided to do all of my work in either Lisp, Forth or J.

I sometimes have to code in Python (there's always Hylang!), but all of my personal interest stuff is in the other three. I am currently going through the 4th edition of "Fractals, Visualizations and J", which is a great introduction to the J language and graphics.

I am trying to make a livecoding (toplap.org) environment in J similar to oK, but more musical and with webgl shaders [1,2].

I'm convinced the proliferation of using GPUs, distributed processing, and multi-core CPUs are all a good fit for a true array or vector-based language like APL, J, and K. I get more of a functional high on J than in Haskell, and feel like I am writing Math equations after memorizing the 'words' of J, more like operators (nouns, adverbs, conjunctions, verbs, etc...)

Shen is a great mix of Lisp and a typed (optional), lazy (optional) language with great pattern matching that has been implemented on a lot major programming platforms (Python, Ruby, JavaScript, Scheme, Haskell, Common Lisp, Clojure, JVM, and Emacs Lisp) because it runs atop a small set of 46 primitives called KLambda similar to the original Lisp 1.5. If you implement KLambda, and pass the tests, in a given PL, you can then run any Shen program on that platform [3].

  [0]  jsoftware.com
  [1]  https://github.com/JohnEarnest/ok
  [2]  http://johnearnest.github.io/ok/index.html
  [3]  http://www.shenlanguage.org/

I love Shen. Check the mailinglist (Google) you can see my name there months ago. I have my own implementation of KLambda and Shen which is faster than any of those on the site, I sponsored the open source license and bought and read both his books. However when Mark decided to make it closed again kind of for professionals and when I found his mindset is too different from OSS I stopped working on it. I am still sorry because I really like Shen. I might revisit that when I have more time because it is very nicely done. Not OSS for a programming language really does not work for me. I mailed and talked how to make money with OSS (I do...) and that a language bar Mathematica, Matlab, K/Q and Labview cannot be closed and paid. And to be one of those you need a niche and first mover clients. I had a first mover client to pay for OSS dev and libs but it never happened. So Clojure. Pun intended.

Yes, I bought the books too, and I am not using Shen Professional, but Shen on Emacs Lisp by Deech brought me back into the fold. I just can't stand the stack traces in Clojure, although I like the language. Shen is like Haskell, Lisp, and Prolog all rolled into one, and satisfies my need for a truly universal language to program in. Now it's just more libraries for me!

I've been wondering if they called the hp48g's forth ui lisp to avoid lawsuits.

actually I'd like to see a (~16 bit) Forth backend for LLVM... so I don't have to "think RPN" myself.

just load the binary into serial flash, point the interpreter at it, and of it goes.

reminds me of jvtoups emacs-lisp stacking monad (IIRC)

abc

Reply on news.ycombinator.com