Carp: a statically typed lisp, without a GC, for high performance applications
github.com
github.com
Yes, but the question was how it does that. Does it reject code where it cannot reliably detect lifetimes? Or does it generate code that just does not deallocate (i.e., that leaks) objects whose lifetime the compiler can't determine? Both of these are "GC-free"...
Well, Carp is a dialect of Lisp so it's free to impose whatever semantics it wants. It's not a dynamically-typed language (where it would indeed be impossible to statically manage memory).
Harder is of course the ffi, which doesn't generally know if the extern call allocated something. This is undecidable for the compiler and he has to trust the user/program. Other lisps have an attribute for those externally malloc'd args.
you only need a GC to recycle cyclic data structures, with links back to previous nodes. since Carp does not create such data structures in the generated C code the reclamation scheme can be kept simple.
heap vars to be kept global (such as closed-over upvals you are complaining about to be leaking) are kept global and released at the end. every dynamic language does keep it functions and closed over values global.
however, lexical closures are released at the end of scope, and remaining tracked heap vars are copied.
In other words, they leak if the program no longer references them.
> every dynamic language does keep it functions and closed over values global.
Yes, but most of them collect closed-over values when they are no longer used. Using a garbage collector. To avoid memory leaks.
Tracked global vars are of course free'd at the end of the program. There's no leak.
2nd closed-over values inside global closures can only be freed at end. A common problem in non-GC'd languages, which can only be fixed by adding a GC. Yes. The goal here is to avoid a GC. So don't reference big memory inside closures (as upval or return val, local is OK), or undef them explicitly, as in every dynamic language.
Using local closures would fix that, but Carp has no syntax for that I think. It would need an assignment from lambdas. Most langs prefer a simple global deffun variant instead. Lua got that right. That's why lua is the preferred game VM. But being able to use the easier and more simple lisp syntax should be preferred.
Cool, we're agreed then.
Also: will be/is the lifetime-checking done in the same way in the interpreter, even though it uses GC? Otherwise, those are in effect two, I think significantly different dialects of the language.
I'll try to do as much checking as possible in the interpreter so that the transition from interpreted to compiled code is smooth. Right now there is none though.
That is, it's not just "rewrite your program until the prover succeeds".
• Regarding FFI, is it possible to pass pointers to carp functions/closures as arguments to external dynamically loaded functions? are those features available in REPL?
• What's the story/approach regarding structs inheritance vs. composition? also custom types and implicit type casting?
• What are the plans regarding multithreading/parallelism? Esp. interesting with respect to memory handling.
At its current state of claims, the language sounds awesomely close to the "holy grail" for me at first glance — though with a fair bit of vagueness and uncertainty.
2. No plans for inheritance at the moment, probably some kind of interfaces but that's not a top priority right now. Custom types are limited to structs, I will add union types soon
3. This has not been decided yet, I want to do more research before settling on any particular solution. It will be a high priority though, since I want to use it for the games I write
Thanks, Erik
I was originally envisioning having the compiler being able to infer parentheses based on code indentation.
Also have you considered compile time AST evaluation and simplification?
Re 1: are there any examples of this in the codebase? I haven't seen anything in the "examples/" subdir, did I overlook, or should I dive somewhere into the compiler/runtime codebase?
Also, some more questions still, if you wouldn't mind:
• Could you possibly answer junke's question? https://news.ycombinator.com/item?id=12041555 I didn't repeat it, as I hoped you'd see it too;
• Do you have tail-recursion? Also, to tell the truth, a list of "what we don't have yet" and also "what we don't plan to have" would help to confront my dreams with reality...
Regarding (2), personally I like Go's/Rust's approach, but I'm not an expert in the domain by any means, and I understand everybody has one's personal taste and view. Just couldn't resist, sorry :)
edit: Also I see you're the author of Else Heart.Break() - awesome, congrats!
I'm being very conservative with adding different type constructs but sure, Rust & Go certainly gets a lot of things right in that department and I'll consider their solutions for the future development of Carp.
https://en.wikipedia.org/wiki/PreScheme
I still don't see a clear answer to junkie's question despite a huge thread. Let's be specific: "Where no GC is used, how exactly do you handle the issue of memory safety? Is it no safety like C, a specific analysis like Rust, or something different entirely?"
Far as concurrency, you probably know about Rust's scheme. The ones that came before it that you might want to consider were Ada Ravenscar and Eiffel's SCOOP.
https://en.wikipedia.org/wiki/Ravenscar_profile
http://www.sigada.org/ada_letters/jun2004/ravenscar_article....
https://en.wikipedia.org/wiki/SCOOP_(software)
http://cme.ethz.ch/publications/
SCOOP might be the more interesting of the two for you. It was deployed in real-world apps in Eiffel first. Then, a team ported it to Java. It had a performance penalty. One of the many works in publications... don't remember which... did something ("slices?") that knocked out almost all that overhead. Another model-checked it to eliminate a few errors in SCOOP model. I see one on message-passing. So much badass work and results on SCOOP model I've been annoyed that it was ignored by mainstream for so long.
So, hope those help in your thinking on tackling concurrency. Personally glad I looked it up for you as I found this very, interesting project for safe use of GPU's:
http://se.inf.ethz.ch/people/poskitt/publications/Kolesniche...
The short answer is that safety is handled similar to Rust, using lifetime analysis. It's quite a bit more simplistic than Rust at the moment though. Also, some checks are done at runtime (like bounds checking on arrays, when turned on).
The closest I have come to finding something similar for livecoding visuals and audio, or games is Extempore [2], which has xtlang, a Scheme, with manual memory management and types. The only two criticisms I have, and they are small, are the build and number of dependencies, and how to distribute standalone executeables. In all fairness there are binaries now available for all platforms, but standalones are still problematic.
I curse Fluxus [3] for starting me down this crazy road many years ago in search of the best creative coding environment. Shame it is not refreshed every so often, because the interface is brilliant.
I'll have to look over Carp this weekend more closely. I am intrigued how it is like Rust's semantics. And if it is manual memory management all the way, then I guess that is where it differs from Rust's safety features?
Now I don't have to try Clojure again! I don't like the JVM, and prefer going to C.
[1] http://www.buildyourownlisp.com/
I've also looked at Lobster, which is now statically typed by default, not optional.
Anyway, it sounds neat, but even after playing with it a bit, I'm not sure how necessary it is.
SBCL (and supposedly the commercial Common Lisps) are pretty fast nowadays. Not C fast, but I can usually get about 1/2 - 1/3 C speed and 4-8x faster than Python without doing anything special.
For example, I made an animation app (https://github.com/jl2/qt-fft-viz) that reads an MP3 and generates an animation while it plays back, and I get 90+ fps. It's not a big graphics/CPU intense video game, but it's doing a lot of FFT calculations and some basic graphics.
So, not to diminish the Carp project, but I think regular old Common Lisp is faster than most people think, is better supported, has more libraries, and works on more platforms and operating systems.
> Carp borrows its looks from Clojure but the runtime semantics are much closer to those of ML or Rust.
> – https://github.com/eriksvedang/Carp/blob/c762e9ff07544b40c8d...
It really looks like a great language!
https://en.wikipedia.org/wiki/PreScheme
I imagine some techniques behind Chicken Scheme in area of optimization might help it too.
ECL (also typed), Vicare, Larceny, Ypsilon, the new guile, Gambit-C, Chicken, Chez, Bigloo, stalin, Corman CL, or fast typed CL: sbcl, franz, lispworks and few more.
Or the ones compiling to LLVM or JVM or .NET. Most prominent Clojure with a very unlispy syntax.
Of course libgc is pretty simple and slow, good for foreign code (i.e. ffi's needed in games), slower than Cheney-2 finger copying allocators which I got down to 10ms, compared to 100-300ms for mark-sweep. But copying collectors need 2x memory, so not usable for small devices such as phones. And a bit complicated for foreign memory, which prefers conservative mark & sweep.
I wonder what Lispers would say about that since I heard that not having a homoiconic syntax might cause some troubles at metaprogramming(macro) level.
I think vectors for argument lists is mainly a usability affordance. Which gives you aesthetics for free. (If you happen to find it more aesthetic.)
> Common LISP and Scheme are not simple in this sense, in their use of parens because the use of parentheses in those languages is overloaded. Parens wrap calls. They wrap grouping. They wrap data structures. And that overloading is a form of complexity by the definition I gave you. (https://github.com/matthiasn/talk-transcripts/blob/master/Hi...)
Elsewhere, he critiques Clojure's use of vectors for argument lists. Because a vector implies order, so every caller must put arguments in the right order. (To decomplect, you'd use maps instead of vectors. But vectors win you brevity. Many notice that with longer argument lists, maps increasingly become more attractive than long argument lists.)
struct {
int x;
} ...;
int foo () {
return 0;
}
In Clojure, take [a b] out of context. Which element is evaluated, a, b, none or both? The fact that it is a vector does not help you, because it could be a binding, an argument list or a vector constructor.
Also, how something is stored inside the AST has nothing to do with its meaning at runtime. Argument lists could be passed using the stack, but you don't actually write a stack inside the source code. The fact that they are written as vectors or lists does not matter either, you still have to know the semantics. And semantics is almost never context-free.
I've never found the use of parenthesis to be an inhibiting factor in the complexity of Lisp code. It's a piece of syntax that has a single use and isn't ever over-loaded. A parameter list... is a list. A form... is a list with a specific structure. Surprise. The hyper-spec is very clear on this (i.e.: there is no over-loading).
I don't think his justification for [] was really necessary. If you want a different syntax for the defun macro you're free to have at it in Lisp. Clojure did nothing special there. Most schemes could interchange braces if you wanted to. You could easily write your own defn macro with your preferred syntax. It is no great innovation to use a different syntax so I don't know why he bothered making that remark about Lisp.
ThinLisp is not a typical Lisp implementation in that it does not implement a
garbage collector or many of the other run-time development features of other
Lisps. ThinLisp is designed for producing high quality deliverable C libraries
and executables from Lisp sources. Originally designed for real-time control
applications, ThinLisp stresses run-time performance at the expense of some
development time conveniences.
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.3.4031
http://people.csail.mit.edu/jaffer/Schlep/scm2c.html
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.570.5677&rep=rep1&type=pdf
For example, you can automatically convert back and forth between s-expressions and something else, like i-expressions http://srfi.schemers.org/srfi-49/srfi-49.html