XL: An Extensible Programming Language
xlr.sourceforge.io
xlr.sourceforge.io
An interesting derivative of XL is Tao3D, which shows what you can do with it. https://tao3d.sourceforge.net
A FOSDEM workshop about Tao3D (which includes initiation to XL): https://www.youtube.com/watch?v=uE9LwSuZD64
The design philosophy, and why it's not "just another Lisp" is here: https://xlr.sourceforge.io/Concept%20Programming%20Presentat...
Iirc backend and goals seem quite similar.
What is the current status the project? Is there any ongoing work and plans?
Here are factors playing a role in my current thinking:
1/ the LLVM debacle. I just can't follow them changing the APIs all the time. I gave up on LLVM for now.
2/ Rust being the first language introducing a concept that was not trivial to introduce via an XL library, lifetimes. I think that I nailed a design now, but it annoyed me for a while, and the design is not implemented.
3/ I spent some time documenting where I wanted to go, notably the type system. As a result, I found that the language was becoming complicated, which annoys me. I'm trying to get back to super-simple roots, but I have no clear path towards this goal yet.
4/ I want to to unify the self-compiling compiler and the dynamic one. The self-compiler only compiles an older dialect of the language.
https://www.piumarta.com/software/id-objmodel/objmodel2.pdf
The focus is on the object runtime where almost any operation including method lookup can be changed inside the end user environment.
Spring Framework & other Java IoC/DI stuff was pretty amazing. I kept finding new cool layers of depth as I dove into the model.
I love the idea of programming languages that might better integrate some of these ideas, of instrumentability. I'm not sure if it's actually necessary though; maybe having all these code assemblers living in userland libraries is fine.
Thanks for the link! Not sure if I've run into this VPRI / Warth combo paper, & worth revisiting either way.
Anyone with recommendations for reading would be welcome to dump those here.
I have a little compiler from 10+ years ago I was working on. And I recall after I had put it down, and tried picking it back up just maybe a year later(?) The interfaces I was using in llvm all changed. It really took the wind out of my sails for jumping back in.
I use LLVM since v9, nowadays I'm stuck on v15 (that's not because of LLVM btw).
Between the two versions there's been a radical change too, i.e "opaque pointers", but the transition was rather smooth as we were provided, for a long time, the two versions of the functions affected by the change. Maybe the LLVM team got more serious since the author experienced the said difficulties ?
Other thing I note is that the author uses the CPP API. I use the C one which exposes only a high-level subset of the CPP one. This encourages a saner use of LLVM, a more concrete separation between the front-end and the mid-end, although sometimes there are limitations.
A simple example of what encourages the C API, especially since opaque ptrs are added, is not to rely on LLVM to retrieve the IR type of an IR value. That should always be done using the AST, eg with an `.ir` field in your nodes.
Another one I remark, after a brief overview of LLVM-CRAP, is that the author had to change the internal data structure used, depending on the LLVM version [0]. Using the C API that would never had happened. The C API essentially allows to create block refs, instructions refs, value refs, type refs, contexts. Then you choose the containers you want to use to hold them. No need to switch to another stdcpp one, even if internally LLVM does so.
[0]: https://github.com/c3d/xl/blob/master/src/llvm-crap.cpp#L265
Really, all use cases. I'm actually a fan of the parenthesis. Lisp feels like aesthetic perfection to me. Languages like Haskell and XL here just look like line noise to my brain.
Are there any that come close to the usage of lisps with proper s-expressions? I've seen them come and go, but no "lisp without parens" that seem to stick around for longer period. I guess that should say something? Maybe we just haven't found the right way of exposing it though.
Personally, when I've given it a try, the indentation-based syntax always makes it hard to use without faults. Maybe it's just a "getting used to" thing, but it seems like a mistake to depend on invisible characters for syntax.
FWIW, I hardly ever write macros (mostly use Clojure/Script), so not sure the main purpose of the parens is macro writing for me. S-expressions with something like parinfer just makes programming a lot more enjoyable and easy, especially compared to all the C-syntax-like languages.
I’ve been doing a recent deep dive in languages as part of my own overly ambitious language attempt and only recently found out that Logo was in fact Lisp based.
I’m guessing that the reason it might not have taken off is because, unfortunately, it was looked upon as a ‘kid language’.
The goals of Papert and others seem a far cry from where tech has landed today.
edit: for an egregious typo
SUM 2 3
[SUM 2 3 4]
If you consider Julia a Lisp, which I would argue you should, it may be the most widely-used Lisp currently. The only way you'll see an s-expression is is to call Meta.show_sexpr.
If you consider Dylan a Lisp, you should consider Julia a Lisp as well. If you don't consider Dylan a Lisp, I have to conclude that you consider the s-expressions to be part of the definition of a Lisp. Which isn't an unreasonable taxonomy, but isn't the one I use.
Maybe, but would you consider Julia a Lisp without parentheses, which was the context here? Because after looking at their documentation (https://docs.julialang.org/en/v1/manual/methods/ for example), it seems to have as many parens as any other lisp, just postfix notation rather than prefix notation, `f(Float32(2.0), 3.0)` vs `(f (Float32 2.0), 3.0)`.
Arguing about what makes or doesn't a lisp is a conversation I don't think will lead to any useful results, I feel like that's ongoing since the 60s or something. Out of my league :)
> lisps with proper s-expressions
It's off-putting for you to fly in and pretend to be surprised that Julia uses parentheses in a way consistent with other languages which are syntactically Algolic. Apologies for expecting a serious conversation, didn't know who I was dealing with.
Have a nice day!
Also, I do not find Lisp unreadable - you train your brain to ignore the parens and look at the indentation instead pretty quickly. And you can always tell Emacs to gray out the parens if you really want. ;-)
But I don’t want to train my brain to ignore useless syntactic noise, I want my brain to be guided by helpful syntactic guides.
https://old.reddit.com/r/lisp/comments/1aw7jim/rhombus_a_new...
You'd think the ability to define custom syntax would be the most important, but paradoxically lisp makes it a lot easier simply because there is no syntax.
Instead of dealing with chains of single assignments as in older C-like languages, you've got a bunch of compact calls fully in-context.
That is not much different from a mathematician that creates a new notation like "∞" or "∫_d_" because existing notation is capable but not convenient (to cluttered, shorthand needed to focus on domain concepts) for what needs to be done.
You may have been referring to Lisp beginners, as opposed to overall programming beginners, but I’ve heard anecdotally that overall programming beginners take to Lisp better than those with experience in Algol style languages.
I could see that. And will prob be testung that hypothesis out soon.
And I too have become a fan of the parentheses.
And then we get to observe: If instead of modeling instructions and memory semantics, we first teach someone to model abstractions and computations, how do they think about programming differently even when eventually exposed to C and its lineage?
Being faculty / wordy, there is probably much written about it attempting to draw conclusions. There were definitely some "smart, but total beginner programmers" in the mix, although I don't know how scientifically separated the performance of the "two 'some's" might have been.
The audience for the series I’m thinking of is adult professionals- programmers I believe. And I’d heard they were from DEC or HP or some such. So in addition to the material itself, you have the blessed quirkiness of its two authors, alongside a whole lot of late 70s/early 80s attire and rudimentary graphics. Quite a feast… lol.
Simplifying for teaching is good. Making useful things completely unavailable… not sure I agree.
I prefer a syntax like in Pure[1], where the ambiguous, hard to parse indentation-based syntax is replaced by explicit semicolons (Yeah, you can use braces/semicolons in Haskell as well, but most code doesn't).
[0] https://github.com/ghc/ghc/blob/master/compiler/GHC/Parser/L...
A few guiding principles are:
- simplicity above all, with as few fundamental elements as possible
- the parser is a separate issue, just write your own syntax to avoid the most divisive bikeshed element of PL design, or pick the C like or ALGOL like one out of the box. You very likely want your own syntax anyway as you write extensions.
- every language element, from modules down to function calls, are first class, ie have an (implementing) type, can be stored in variables and used in expressions, be introspected and evaluated/deployed.
- runs at compile time, compiles at run time (code generation/partial evaluation/dynamic code)
- generates C, Java, Python and various bytecodes to maximise interoperability, code availability and deployability
- has no standard runtime or standard library of its own, is entirely parasitic on other environments
Even if it ends up being completely useless, it's a really interesting exercise in design.
Some notes on my lack of progress: http://www.nuke24.net/plog/43.html
I’ve come across Refal[1], who’s creator Valentin Turchin seems like someone who hasn’t gotten the recognition he maybe deserves. As an aside, his son Peter Turchin does some interesting work in a different direction that’s def topical these days.
I know SNOBOL gets labeled as one.
Would Prolog count? I haven’t actually written any myself. They def make a strong appearance in Erlang, though it isn’t the first thing that comes to mind if describing. There’s XSLT of course.
Any others?
I think if one were only able to pick one metaphor for a language, I think a strong case could be made for:
pattern => behavior
This was what convinced me to switch to LLVM. I now regret that decision, but at the time, it sounded like a lot of fun.
typedef _Load("libvector.so")(int) vector_int;
I think my only worries would be how to handle life-time analysis, i.e., the moral equivalent of destructors, copy constructors, move-semantics, etc; maybe the dylib ('libvector.so', in this case) has to register stack-clean-up, copying, and move-semantics code-generation for the compiler?
[1] Modulo the introduction of `_Load("...")(__args__)`, and a few other function-like invocations.
EDIT: Joking aside, I would love to hear a compare & contrast between Galaxy/GalaxC & LX/XL, though!!
EDIT2: and via that first link, I think you can get to a full source code distribution compile-able with C on 32-bit Linux. Might have its own woes like LLVM hosting, but those might also be more solvable as part of a 64-bit port.
I have my own little (overly) ambitious language effort in a different direction and tend to beat myself up a bit since I feel like I should have more to show after almost 4 years.
Then I noticed in the author’s history section towards the end that they’d been working on this sine the early 90’s and felt better about that.
Having recently joined that tribe, I think it does take a particular worldview and type to be willing to undertake that. I suppose folks writing an OS perhaps even more so.
Maybe it's a quine?
Can you define an "isn't" operator?
"It is what it is".
https://www.google.com/search?q=Trump+it+is+what+it+is
2nd hit onwards.
./xl -nobuiltins -parse /tmp/glop.xl -style debug -show
(infix is
is
is
)
Now, that name is very unlikely to be usable because `is` as an infix is so central to everything. As a matter of fact, it looks like even "is is 2" actually crashes the current implementation.
Oh well. I wonder what a sensible error message on this would be. Probably: "Bill Clinton denied to comment on the meaning of that statement".
Oh man, that’d be hilarious.
I’ve been hacking around with my own sorta Forth/Postscript like attempt at a language that runs on WebAssembly and does graphics in the browser.
Odds are it’ll never see the light of day. But if it does, you’ve now triggered all kinds of ideas for ridiculous ideas for error messages. Like starting them of with ‘hey bruh…’ or trotting out good ol’ Mr Clippy from Microsoft.
And it's not compile-time only, I'm pretty sure you can't do this as a macro:
>loop Body is { Body; loop Body }
Scheme macros use pattern-matching.
> loop Body is { Body; loop Body }
This, though, is definitely strange. I’m not quite sure how it works.
https://github.com/c3d/xl/blob/fast/src/builtins.xl#L222
How it works is what I tried to document in the document above. The trick is to have some fixed semantics on how rewrites are done, but a lot of freedom in how this is implemented. And this is far from perfect ATM.
For example, if you write "X is 2", this can be implemented as a constant, as a function returning a constant, or as a macro. However, if you write "X is seconds" in Tao3D, where "seconds" returns the current number of seconds in the clock, then it can no longer be a constant value. You still can use macro replacement (or inlining) or turn it into a function.
More details here: https://xlr.sourceforge.io/#compiling-xl
uint64_t alignBitsLeft(uint64_t x) {
return ((x >> 63) == 1)) ? x : alignBitsLeft(x << 1);
}
This works, because of a linking step at definition completion, where the "alignBitsLeft" reference gets linked to the now complete "alignBitsLeft" definition.
Similarly for:
> loop Body is { Body; loop Body }
The "loop Body" definition beings, including a reference to itself. Once the whole definition has been built up, the "loop Body" reference gets linked back to the "loop Body" definition. Ready for use.
--
Another way to look at this is memoization (at compile or run time). When you evaluate the second line of this:
n is 1;
loop { print n; n is n + 1; }
{ print n; n is n + 1; loop { print n; n is n + 1; }}
Herein, macros seem to be invented from first principles, having only been exposed to imperative languages like C++. It is interesting to see macros here invented in the context of a language that is not homoiconic. I wonder how it stacks up against Rust macros.
Postscript would count as homoiconic, I’d think. Though I haven’t actually seen programs that so this.
Compiled Forths might opt to wall off some of those options behind the compiler interface, in effect making the language slightly less far-reaching.
It is trivial to collect those Xts as data and execute them later.
CREATE XT-LIST ] THESE WORDS COULD BE FORTH CODE [
XT-LIST now contains six XTs. They can now be read one XT at a time and executed.
So if one accepts that those XTs are data, BUT they are also code since they can be "executed" by the Forth VM then I think we can say Forth is homoiconic.
However if that doesn't suffice it is perfectly legal to pass text strings to EVALUATE.
: TEST S" THESE WORDS COULD BE FORTH CODE" EVALUATE ;
That's all I got.
I’m going to have to do more Forth. And I have a sudden urge to go write some silly self-modifying assembly, if current processors even allow that anymore.
Definition of 'if' statement: https://github.com/c3d/xl/blob/fast/src/builtins.xl#L155
// If-then-else statement
if [[true]] then True else False is True
if [[false]] then True else False is False
if [[true]] then True is True
if [[false]] then True is false
// Loops
while Condition loop Body is
if Condition then
Body
while Condition loop Body
until Condition loop Body is { while not Condition loop Body }
loop Body is { Body; loop Body }
for Var in Low..High loop Body is
Var := Low
while Var < High loop
Body
Var := Var + 1