Carp lang: statically typed Lisp, no GC
eriksvedang.com
eriksvedang.com
https://github.com/carp-lang/Carp/blob/master/docs/LanguageG...
This part hits a real sweet spot for me:
> Carp borrows its looks from Clojure but the runtime semantics are much closer to those of ML or Rust.
One of my current side-projects is a little Civ-style game I'm working on in Racket, which I decided to start coding in on a whim. Will see how it goes moving pieces of it across to Carp.
And yes is Open Source (Apache Licence)
How come we don't have Google Maps-like zooming with LOD mapping to abstractions at different levels?
You don't even really need s-expressions to do what you're talking about. You can do it off of any grammar, which all current programming languages have[1]. S-expressions are slightly easier, but they aren't easier in the place where the real problem lies. The real problem lies in the fact that the code isn't shaped the way you think it is and it isn't as nice as you think it is and it turns out nearly every attempt to project code into something other than our current plain text representation flounders on the pathological nature of source code.
Personally I think this is a fundamental problem, but there is an avenue to where I would be wrong. If somebody who respected text and understood why it is currently so dominant, in a positive sense, also took on the task of trying to transcend it based on that understanding, perhaps they could find something, revealing the current lack of such editors to be accidental after all. But as long as the only rebels trying it also labor under the crippling error that text is a weakness that needs to be discarded as quickly as possible, rather than the only methodology currently known to be even remotely strong enough to support our programming endeavors [2], they're never going to get anywhere in trying to deploy a system that for anyone to be even remotely interested in it pretty much has to immediately be able to scale up to some reasonably fraction of what text editors already do.
I find it telling that the stickiest such effort I've seen to date is literally to simply provide the programmer with a completely zoomed out graphic of the text file in the scrollbar area.
[1] Since nobody has yet had the resources to try to figure out how to take a language model and turn it into a programming language. Patience.
[2] Or, to put it another way... make a list of all the weaknesses text putatively has for programming languages. Then sit and stare at it for a while until you grok that that is still the shortest list of weaknesses we know of, and all other efforts have even worse weaknesses at scale. It isn't that I deny the weaknesses; they are manifest and obvious to pretty much everybody. It is that it's still the shortest such known list at scale.
I’d say they are:
1) Lack of need for a proprietary tool to view/edit. 2) Density of information. 3) Nearly every existing attempt I’ve seen at visual programming is insultingly bad. So I get it why folks would be hesitant to waste time on some schmuck’s new invention.
Am I missing anything on the advantages of text? This is an honest attempt to see where I’m missing perspective, not an attempt to convince.
I don’t think the target audience is developers with an existing working workflow- the potential upside prob wouldn’t justify someone spending the time/mental effort to grok a new model. But I think the audience is those who have some aptitude and could be coding and aren’t yet. In my mind, primarily so that maybe individuals can have a little more influence over the code and algorithms in their own life more so than industry roles.
Given that the ways you can convey information graphically are a super set of what text provides, I can’t help but think there’s some room for improvement- especially with concepts like literate programming around.
Overall, I’d like to give your average personal a little more agency. Maybe that’s a graphical approach. Maybe its getting folks to do basic Rails/Django/Flask/PHP whatever.
But I am curious if I’m missing obvious rationales for text…
Well-defined semantics for search, diff, insert, and replace.
Source code is the best way to allow arbitrary evolution of code. Visual programming cripples the kind of evolution that is possible or reasonable.
Now, what might work would be a component designer where all the state of the designer is stored as the source code in human-editable form. It might be interesting to be able to right-click a class or function call in source code and "Edit as Component". If that improves my productivity, it might be worth paying for. OTOH, such a thing now has to compete with AI code generators.
But still, maybe someone didn't just find the right approach, so don't feel disillusioned!
I've seen a lot of people in business point to it as something to uphold - that if we were writing software correctly, it would be capable of that.
But it still requires fairly technical people, they can still create abominations, and oftentimes they're used as prototypes for designers until the programming team can come in and write something that is actually maintainable and performant.
Also my brother works in business process management. They make similar tools but for businesses. So it does exist.
1. How many edits don't preserve the AST structure of a program. Changing a leaf expression to a different expression does, as does, say, changing a `for` loop to a `while` loop. But cutting the `(cond)` part of `if (cond)` out (including the parentheses) and hoisting it to a separate expression does not because now those parentheses mean something different.
2. How often the program is in a grammatically invalid state across multiple edits.
I suspect fans of always-valid visual editors massively underestimate the size of both of those.
Acknowledged.
My answer is, programming is n-dimensional and text is one of the few mechanisms we have for dealing with that. I believe this is actually the characteristic of it that has screwed so many other efforts, especially visual ones.
I don't know if n-dimensional has a formal, accepted definition, but the characteristic of an n-dimensional space that is of interest to me is that at any point, suddenly a new dimension can be introduced, and this is indefinitely recursive.
Let me try to demonstrate this with a web site first. Imagine there's a web site that has a home page with ten internal links. Each of these ten links then has a set of ten pages that they link to, all unique, and those pages link nowhere. You could represent this particular website in a clean 2D spreadsheet. In the A column, you have the ten top-level links, and then in B through K you can have the subsidiary pages. In this simplified model, a web page can be imagined as something like a linear collection of links, so I can describe a page as being "Link 4, Link 7", identifying the subpage from the 4th top-level link, then the 7th link of that page.
Of course, this is a highly contrived example. Perhaps only and exactly one of those links goes to another top-level page with another set of 10 links that each have ten pages. So now, in the middle of this nice two-dimensional layout, in exactly one cell, suddenly you have another two dimensional layout appearing, and neither of those two dimensions is the same as the previous two dimensions, it's a fresh set. Nothing stops these pages from having another fresh set of dimensions appearing. In n-dimensional space, you never know when you're going to get a fresh set of dimensions when you cross over some unassuming-looking edge in the graph.
In reality, web pages are so complexly interlinked that there isn't any visual way of representing them in the general case. My example is obviously super unrealistic. We normally wouldn't even dream of trying to represent a website in a spreadsheet like that.
That doesn't mean the situation is hopeless necessarily, but it does mean we will make compromises. A sufficiently static site might have a canonical "site map" that completely organizes the site into a tree. But that is a simplified projection of a human conceptualization of the space; it isn't the real linking map of the web site. It is valuable, but it's only a particularly humanly-compelling slice.
In general, trying to collapse an n-dimensional space into a finite number of dimensions is not something you can do. Especially when the number you're trying to collapse it into is as small as "two" or "three"! This, again, doesn't mean the situation is hopeless. There's a lot of angles you can take on that "in general". "In general" is, as the saying goes, doing some heavy lifting in that statement. But it is an important aspect of understanding the problem.
How does text handle the n-dimensionality of code? Consider it from the point ofview of the language grammar. If I have
x = 1 + 1
x = 1 + (readint() + readint())
And then this nests infinitely recursively. And programmers keep coming up not only with new dimensions through the basic n-dimensionality of text conforming to tree-based grammar structures, but then new ways of adding dimensions; "x.y" as a field reference of a particular type is one thing, but then x can be polymorphic. Or decorated. Or monkey-patched. Or perhaps the language has multiple dispatch. Or overloading operators, meaning that operators now need to be be something you can follow to an implementation. Or... the list goes on.
You end up with a statement like "x.y.z = a.b.c(q, q2)" having potentially dozens of graph edges departing from it, each to a declaration that may itself be a rats nest of graph edges, each to a further declaration that may be a rats nest of edges... eventually, everything has to terminate in some sort of literal or atomic value of some sort, but we want those complex things. It's why we're not programming in raw assembler. (Though that would raise its own problems.)
Generally, we don't even think of these spaces as being "dimensional", which is to say, able to identify members by a specification of numbers in some number of "dimensions". But it is a thing we do. You can see this in programming in a very simple form pretty much anywhere you see dot-separated values of arbitrary length. A number of standards have some sort of 2.8.183.2.18.1377273 embedded in them. DNS names function this way too; behind every DNS dot could be another arbitrarily large and complicated subdomain scheme. You could proceed down a dozen simple elements and then only suddenly see an explosion of complexity. If you imagined a perfectly static web, you could theoretically (non-uniquely) identify web page locations by starting with a top-level catalog like Yahoo, and identifying web pages via the path you need to take to get there, like the number above. In reality, the space itself is shifting too fast for this to work, isn't constant, etc. But that's what n-dimensional looks like; you can identify the location of things, but it's not well-defined within that space how many "dimension" components you'll need.
Now, there actually is a natural visual mapping to all this. You have your "x.y.z = a.b.c(q, q2)" in a function, and you make it so you can click on anything, and it "zooms in" on that portion, in a way that the original expression is now left behind and gone. The problem with this natural visual mapping is you end up with a lot less context shown rather than more in the obvious, naive implementation. (Another problem is that this is basically nothing more or less than "Jump to definition", a thing I generally already have in my textual IDE.) The next obvious elaboration is to try to "expand" instead of zooming in, but this rapidly becomes impossible for all kinds of visual reasons. Maybe if we had 5 or 10 or 20 spatial dimensions to work in this wouldn't blow up so immediately, but holy crap is a mere 2 or 2.5 dimensions a kick in the pants to work with here.
There's actually two possible solutions. One is to come up with some clever visualization, the usual approach. The other approach is to attack the root of the problem and try to come up with a programming language with a much lower n-dimensionality. The latter is arguably what having s-expressions has to do with any of this, but it's still not really low enough. The problem with the latter is the resulting language is pretty unpleasant still. We kinda had them in the past, and nobody's rushing back to them. Even the crazed whackos writing Turing-incomplete languages aren't doing what you'd need for this. The need for n-dimensionality, and the value of it, just come out too readily in programming. I don't think this is a result of us having text and leaning into it, though there may be an element of that; there is intrinsic benefit in being able to take what was a constant int in the previous version of the code and inserting an arbitrarily complicated function to obtain that int some other way instead. I might just be too blinded by my own experience to be able to conceive of programming any other way, but I am even so inclined to think that's a fundamental thing everyone, everywhere will always want. And if you have that, you've introduced n-dimensionality. Kinda like accidental Turing completeness; sealing this away and still having something useful seems a non-trivial challenge to me.
Text has that characteristic where it's a well-defined operation to wedge new dimensions in anywhere, e.g. replacing a 1 with a new more complicated expression, with reasonable, if not perfect, cost/benefit tradeoffs. I haven't seen any other solutions to this. Given the number of whacks taken at it, it is at least a rational conclusion to think there may not be one. But even "rational" conclusions can still be wrong (the great rationalist conundrum, but that's a separate post). If someone does crack it, from either angle, I won't complain.
We can tell that in constrained problems it can still do OK. Obviously visual programming is used in some places completely routinely in some places today. But I've never seen an approach that would handle, say, the Linux kernel any better than text.
I would personally say that super-high-dimensional spaces are challenging. There are ways it will trip you up for sure, like the well-known "spiky spheres" which are themselves not "real" in the way we would traditionally think of them. But n-dimensional spaces can be actively perverse; they can trip you up very effectively even with relatively small instances, your every attempt to apply some simplifying assumption (even for the purposes of visual display) being rapidly and frequently destroyed by some falsification of that assumption, e.g, "function calls usually don't have more than 3 or 4 parameters" or "functions generally only have 50-100 lines tops" or... the list goes on. Which is another way of viewing why visual approaches to this problem have fared so poorly historically.
At least not in terms of current technology; we can all imagine the Star Trek future where you can just tell the computer that you want an analysis of all the currently-interesting factors related to this diplomatic situation and a collection of AIs turn that into something effective. At that point "programmers" aren't staring at large blocks of text. Though don't be fooled; someone is going in deeper than the initial instruction. "Programming" may instead be a matter of working with an AI almost like you'd work with a coworker, but someone's going to be dedicated to the task of maximizing output. You've seen we've recently taken a big step in this direction, but we've still got a number of steps to go, and even this big step is still being processed by the world at large.
But in terms of current tech and the world of programming as we know it (which I actually believe will always be with us; while it has its manifest limitations, the ability to take concrete instructions and chew through them at light speed will always be useful; even if you've got an AI that can use teraflops and petaflops to have a real-time conversation with you, even that AI will have uses for conventional programming techniques to use those teraflops & petaflops just as we'd use them today, rather than for high-level AI functionality), I don't know if there's a huge paradigm shift that can scale to, say, the size of the Linux kernel.
But if anyone wants to create it, I think they need to start from an understanding of the positives of our current paradigm, rather than the false assumption that it's all just some sort of big mistake and it's so obvious what the better answer would have been.
...
"Our goal is to build a format that compares to text files in the way that text files compare to bitmaps."
>"We're writing these trees ... and parentheses are just a way to write this two dimensional structure as a linear character string. Of course, when LISP first started and people had teletypes or punchcards or whatever and (the parentheses character structure) was more convinient. But maybe if LISP started today the syntax would look more like (a 2D syntax tree)."
Another demo since, showing how I write tests: https://merveilles.town/@akkartik/109742488349122478
With macroexpand-* family of emacs+slime, which I use all the time, I do have access to this. I haven't done any Aspect-Oriented Programming, but suppose it wouldn't be impossible to get similar functionality with it too.
A lesser form I use often with non-Lisp languages is narrow-to-defun and narrow-to-region, especially when I need to look at more than one area of code simultaneously.
But for the broader idea I think you can also just as well ask why Bret Victor's vision has gone nowhere (and probably never will). I'm satisfied myself with an answer like this:
"Linux supports the notion of a command line or a shell for the same reason that only children read books with only pictures in them. Language, be it English or something else, is the only tool flexible enough to accomplish a sufficiently broad range of tasks." -- Bill Garrett
But I also think that for many abstractions, the unabstracted form is often just fine too, and if it's common enough it becomes a 'pattern' that is read quickly in the same way that adults read words or chunks of words and not letter by letter. rpg's free book "Patterns of Software" gets into this near the beginning. For a simple example, in Common Lisp you might at some point write something like
(let ((result (mapcar f list))
(length (length list)))
...)
(let ((length 0))
(let ((result
(mapcar
(lambda (x)
(incf length)
(f x))
list)))
...))
Initialize some local variable result
Loop
Within the loop, update the result (in the above, it's the "length" variable)
Outside the loop, use the result
Tim Dévé has even created a game for the Nintendo Game Boy Advance by using Carp's C interoperability; you can play an emulated version online[2].
[1]: https://github.com/carp-lang/Carp/blob/master/docs/CInterop....
I was trying this from `nix-shell -p carp` and immediately ran into issue with this when building the hello world example from the README. (Package sdl2 was not found).
Not sure how to fix this just adding `- p SDL` to the nix-shell prompt does not help, as the invoked compiler does not know where to look.
Any recommendations here? I want to avoid using a global clang/gcc installation for various reasons. Is running this under `nix` a good idea? Should I move this into a container?
That's not really relevant here, as carp itself seems to basically do code-gen and then compile stuff on its own (outside of any Nix build), so the environment in which you run carp must be set up such that the compiler and linker running in it can discover the needed libraries. Very often the mechanism used for that is pkg-config.
If you wanted a more "Nixy" way to do this, you could write a `buildCarp` suite of functions in Nix that correctly sets all this up inside of a derivation build and takes the Carp code you're compiling as an input, yielding the compiled one as a Nix store path.
For just playing around, it seems more reasonable to just add the needed dependencies to the runtime environment of carp.
The author of Cakelisp notes that Carp requires writing bindings, but Cakelisp apparently does not:
https://macoy.me/code/macoy/cakelisp/src/branch/master/doc/V...
Edit: I think Cakelisp has you explicitly generate C code, and maybe Carp does it behind the scenes?
Cakelisp is an S-expression representation of C++, kind of like how Assembly language is a nicer representation of machine code.
Carp is its own language which transpiles to C, like a bunch of other languages that get transpiled to C (e.g. Nim, Chicken Scheme,...).
I suppose the main trade-off (besides Clojure syntax) is RAII and MMM versus a borrow checker, consult your nearest Rust article for much back and forth about that stuff.
Come to think of it, that could work well with S-expressions:
(with-arena my-arena
(let ((foo-a (allocate foo)))
;; yadda yadda yadda
))1. Do static typing with manifest-typed values that can be assigned to arbitrary symbols, or 2. Do Lisp in any real sense without #1
There was a claim a while ago that gradual typing is inherently slower than static or dynamic, but that sounds like an implementation bug to me, not anything fundamental.
Don't ask what "static typing" is, you are simply supposed to love it, because it's good.
Metaprogramming in C or Java is not fun. It's either code generation integrated with the build system, preprocessor (yuck), annotations processors or runtime weaving. Which all suck at different levels.
Arrest this man,
He talks in maths,
He buzzes like a fridge,
He's like a detuned radio.
Source: I contribute to the language.
I've heard about Carp before, but haven't ever tried it. I've longed for an embedded ML language, and I like Lisp and Scheme, so this seems right up my alley. Apologies for all the questions, but I'm just curious about potentially applicability to some things I have in mind.
As it compiles to C, any microcontroller with a compiler that supports something that looks like C can work. I've run Carp code on a GBA (as mentionned somewhere else in the comments), esp32, esp8266, as well as Arduboy & Pygamer via Arduino.
There is some more information about running Carp on embedded platforms in the docs[0].
One last thing I wanted to mention is that Carp is still very much in flux so it might not be the best choice for longterm projects, but if you're interested in playing with the language there is usually always someone to answer questions on the Gitter[1].
[0] https://github.com/carp-lang/Carp/blob/master/docs/Embedded....
[1] https://app.gitter.im/#/room/#eriksvedang_Carp:gitter.im
> The first version of the compiler was made 2015 - 2016, attracting a fair bit of attention in the programming community. In the spring of 2017 I rebooted the project, rewriting it in Haskell (it used to be C).