s7 – Scheme implementation intended as extension language for other applications | Hacker News Reader

s7 – Scheme implementation intended as extension language for other applications

ccrma.stanford.edu

80 pointsazhenley29 comments

I use S7 in my open source Scheme-for-Max project. I have built a Max/MSP external enabling one to script and live code Max/MSP (a computer music platform) in Scheme. I love it, I checked out many options before settling on S7 and have been super happy with the choice (guile, gambit, chez, chicken, chibi, racket, ecl). It's in many ways similar to Janet and Clojure: a single-namespace lisp 1 but with keywords, CL style macros with gensym, and environments. It is also close to Guile, but is liberally licensed.

The best part of S7 in my opinion is how easy to embed and extend it is. The C FFI is really easy to use, and all of S7 fits s7.c and s7.h, making building very easy. You can see some S7 in action in Max/MSP here: https://www.youtube.com/watch?v=ErirIFCTdjg&t=2s&ab_channel=...

For those interested, Scheme for Max lives here: https://github.com/iainctduncan/scheme-for-max

It will be seeing new releases this semester as it's now my Masters in Music Tech thesis project at the University of Victoria, working with Dr Andrew Schloss. I plan to have full support for running the Common Music algorithmic composition toolkit in Max/MSP this year.

I have also used S7 in a Juce applications, which is what the normal version of Common Music does in its "Grace" host and Christos Vagias has been using it in iPlug 2 for VST plugin development.

The with-baffle is interesting (and call-with-exit).

My personal take on Scheme is that call/cc is elegant and powerful in the same way that assembly language is elegant and powerful—and just like it’s hard to reason about programs that make creative use of assembly language constructs, it’s hard to reason about programs that make creative use of call/cc. Just like you can implement nearly any control structure in assembly, you can implement many with call/cc. I was once working on a Scheme compiler but abandoned the efforts once I realized the enormous negative impact that call/cc has on writing compilers, interpreters, and library code.

Do you want to write a reasonably fast and general mapcar? Do you want to do some basic escape analysis to see if you can allocate a value on the stack? Better have a good, long think about things. The withering gaze of call/cc will make your face melt.

So, years ago, I was hopeful that there would be an officially sanctioned Scheme without call/cc. Call it what you will. It never materialized.

You can think of it as two sides of the same coin—just like objects are a poor substitute for closures and closures are a poor substitute for objects, call/cc is a poor substitute for exception handling, and exception handling is a poor substitute for call/cc. R7RS adds SRFI-34, but you still see stuff like with-baffle and call-with-exit precisely because call/cc is like a stick of dynamite that you can put in your code, and there’s no way it can work across any kind of FFI boundary without severe restrictions.

Good point — especially since as opposed to full continuations, which are essentially impossible to implement efficiently, coroutines / one-shot continuations, have a completely obvious and intuitive implementation: each coroutine is a separate stack and when one coroutine yields to another, it's just a jump to executing code on a different stack.

And with coroutines, it's frequently possible to implement a "multi-shot" continuation, by cloning any mutable parameters, and returning a generator which returns a fresh coroutine each time its called. It's not the same thing as a delimited continuation (is it?) but it's close enough for my purposes.

Normally, you just want functions, but sometimes, you want coroutines. Normally, one-shot coroutines are fine, but sometimes you want multi-shot. I have never once felt like a full-blown continuation was the construct I was missing.

From what I've read and heard, call/cc is generally considered strictly inferior to delimited continuations.

> I was once working on a Scheme compiler but abandoned the efforts once I realized the enormous negative impact that call/cc has on writing compilers, interpreters, and library code.

Isn't implementing call/cc trivial if you have CPS?

The other criticisms are common and valid. Racket uses delimited continuations instead because of the pitfalls of call/cc.

> Isn't implementing call/cc trivial if you have CPS?

You’re right that call/cc itself can be trivial. The problem is that everything else becomes way more complicated because call/cc exists.

The critical thing to remember is that if you have call/cc, ANY user-supplied function can return multiple times. In general people expect library functions to “just deal with it” and in practice that means that (1) half your library functions might just be broken but you don’t know it because they fail some weird test case involving call/cc that you didn’t cover (1) you can’t do anything clever to speed up your library functions and (2) your compiler won’t be capable of doing the escape analysis to fix the performance issues in your library functions.

The thing is, you can conceptualize of call/cc as being just “free” if you think of the language as a way to program a continuation-passing machine, but when you do that, you’re basically one step away from some weird garbage-collected assembly language with unlimited goto.

In order to reason about programs more effectively we largely abandoned goto except for the few cases where it’s the right tool. Most of the time we like our control structures to match the stack—you go IN to a function exactly once, and then OUT of the function exactly once.

Scheme is properly tail recursive. That means that CPS conversion is going to work. Seen that way, call/cc is just a convenience feature that spares you the work of the conversion.

Introducing weaker language constructs and claiming that they are easier to reason about seems spurious, because there still may be full fledged continuations coming with CPS. I guess this so called reasoning will only work for a limited subset of all programs.

I actually think that call/cc has the advantage of forcing people to realise what the ramifications of proper tail recursion are.

Nobody likes call/cc except for the chicken scheme people where call/cc is more or less free. Delimited continuations is the bee's knees. They are faster, more easy to reason about, captures only what you tell them to (and thus are better in a GC language).

Given enough time they will replace call/cc entirely.

Buy, hey, don't take my word for it. I will appeal to authority and post this: http://okmij.org/ftp/continuations/against-callcc.html

What's a good language and textbook / lectures to pickup delimited continuations and how to use them?

I don't know really. I was introduced to them through Andy Wingo's blog wingolog.org and ported some call/cc benchmarks to use delimited continuations tomlearn howmto use them (and how they differ). Then I went head first into the source code of guile-fibers, a parallel concurrent ML implementation for guile that uses delimited continuations. It implements a full CML system as a library, which really is a testament to the power of delimited continuations.

There is a lecture by Matthew Flatt held at Microsoft research about delimited continuations where he uses a web server to show the differences of delimited vs undelimited continuations somewhere in YouTube.

Edit: btw the speed difference with regards to overhead between call/cc and call-with-prompt was something like an order of magnitude for simple things like generators and the rather well known fibcc benchmark.

There's a nice collection of material here:

https://github.com/rain-1/continuations-study-group/wiki/Rea...

I would rather see delimited continuations, but the Scheme community doesn't seem to have settled on a way to implement and represent them in the language.

Nevertheless, having some form of continuation mechanism is more in line with Scheme being a small but powerful core upon which any programming construct can be built. Sure, most programmers out there would rather express things in terms of try/catch. But the idea is not to have continuations replace try/catch, but to have try/catch expressible in terms of continuations. Then it becomes a library you load and depend on, and theoretically runnable in any Scheme implementation that supports the continuation mechanism the try/catch library uses.

What's a good language and textbook / lectures to pickup delimited continuations and how to use them?

Other small, embeddable schemes that are worthy of notice:

* S9fes: https://github.com/bakul/s9fes

* Chibi: https://github.com/ashinn/chibi-scheme

Both are just a couple of files to add to your project, and easily extended with good built-in features.

I've also had a lot of luck embedding Chicken. It's quite straightforward to do, and with the boon of being able to compile Scheme to C to include directly.

Note that the original version of s9fes is to be found at Nils M Holm's site https://t3x.org/s9fes/index.html

My copy has a few plan9 specific extensions. Though more is is needed for anything like building a user level filesystem. I track Nils' original version on the t3x.org git branch.

Because Guile was such a great success? Or because it isn't?

Guile has has a very great deal of new development in recent years, including truly massive optimization.

I am curious what people who know Schemes think about how it now compares to other efforts.

While S7 is quite similar to Guile, it is actually a descendent of TinyScheme, not Guile. I looked closely at Guile too before adopting S7 and while it was probably the second easiest to work with as far as embedding, for me the license was the main issue. S7 is liberally licenses (can't recall OTOH if it's MIT or BSD)

Another interesting pro of S7 is that it compiles to webassembly! Chistos Vagias has made a neat demo of that to allow you to try it in the browser. https://actondev.github.io/s7-playground/

s7.c is a 98,182 line C file.

So um. Yeah. Good luck wrapping your head around that. Eesh.

I though for sure s7.c was an amalgamation file like sqlite3.c, but evidently it's not. Eesh indeed.

This is on purpose so that embedding is simple. And embedding is really really simple. I use it extensively, and have only had to look at s7.h and the docs file.

Because plonking ten .c files into a project is that much more complicated than one? That's where the complexity of embedding lies?

I get what you are saying... but S7 is so simple to embed that yes, that would make a (minor) difference. It is really, really easy to embed. Like, takes 10 minutes to get going level of simple. So having to do nothing other than include one header and link one object file (or add to the build) is nice. I don't really have a strong opinion one way or the other on the all-in-one-file issue, but as someone who uses it in an embedded context all the time, I'd have to say the fact that it's all in one makes zero negative difference to my work. And it's really nice to be able to just leave s7.h open and grep over it to figure out how to do just about anything. This simplicity can not be said about Chicken, Racket, or Gambit, mind you. (Guile though, also has a low barrier, I really like Guile too...)

There's a whole class of Lisp and derivatives, dialects etc that get embedded in applications that are in itself written in such languages, and anyone involved in such applications thinks this is just fine.

Out here in the real world, nobody will touch Lisp or anything remotely like it with a 10-foot pole. The real world embeds Python or Lua.

S7 was designed for serious computer music, and comes out of CCRMA at Stanford. While I like Python (I did it professionally for 10+ years!) generally speaking in computer music embedding Python is a non-starter because of the GIL - it does not play well in apps where you have thread and performance issues, which in real time computer music, is basically a given. S7 on the other hand is very easy to use in that context, you can turn the garbage collector on and off, and have multiple instances in different threads with no issue. I really like Python, but have totally given up on it for music applications. There exist Python music libraries, but honestly, no one who really knows that field uses it in apps, though they may use it as a preprocessor for score data.

BTW, out here in the real world where I evaluate tech companies for major investments by private equity firms, I have personally worked on a deal that ended with a $100M investment in a company with a Common Lisp DSL in their modelling engine under the hood. They aren't publicizing this fact, but it happens more than one would think, a lot of tech companies are told by investors not to share their secret sauce.

I've done 50+ diligences in the last two years, ranging from $2M to $1.5B transactions. Only two had embedded DSLs, one in lisp and one in their own language that wrote an interpreter for in C. I don't think Python gets embedded much these days to be honest, I've done it and the experience was not pleasant compared to using S7. Though I do know Lua gets used in the game industry.

Related question - what are today's extension languages that are secure and safe-by-default and also portable?

Reply on news.ycombinator.com