Zero Feet: a proposal for a systems-free Lisp
applied-langua.ge
applied-langua.ge
The user can easily obtain an executable C compiler from their distro or whatever. With that, they build your language.
If your language is written in itself entirely, then the user needs an executable of the compiler for your language.
There is a second benefit: C is an external language. Nothing that you do in your program changes the C language. If your language is written in itself, and you make changes to it, you're changing the language in which your language is written. Mistakes in this can cause hair-pulling bugs.
Bootstrapping with another language like C doesn't have to mean that anything remains that is written in C in the boostrapped product!
A self-in-self thing can be turned into a C boostrapped thing by simply adding an independent, minimal implementation of that language written in C that the user can compile and run, which is capable of running the real self-in-self compiler to compile that compiler and the run-time. Once that is done, the real compiler can be used to again compiler the compiler into a faster compiler and the run-time into a faster run-time. After that, the C boostrapper is not used for anything, and there is no evidence of it in the product.
It may be that just being in a position to use the language a lot (because you're implementing it) and to change that language (because you're the one building it) is enough to get the effect you've observed, but I wouldn't be at all surprised if you would only start down that path in the first place if you already had strong opinions about what a good language looks like.
Whereas, some languages are really good at making compilers. Then, some tools are specifically designed for program transformation and analysis. Using those languages can dramatically improve your compiler which might indirectly increase the number of goals the other language hits.
Fully dogfooded: C++, Rust, Go, D, Zig, [Java with Graal or Jikes], [Python with PyPy]
Not dogfooded: JavaScript, Python, Ruby, Lisp, PHP, Lua, Swift
Compiler is dogfooded but VM/runtime is not: Java, C#, Haskell, Kotlin Native
The compiler (i.e. whatever parses source files and generates either native code or bytecode) is almost always written in the language itself, except for purely interpreted languages where that would be too slow.
If there's a VM or runtime, it's written in C or C++ if the language is slow; it's written in the language itself if the language aims to be as fast as C; and for the many languages in the middle, it varies.
The biggest exception I can think of is Swift, which is mostly not dogfooded, although it's becoming more so. And even that is explainable by other design decisions. Like C++ or Rust, Swift requires the compiler to do a lot of work to compile it properly. Like Java or Haskell, Swift is really slow at runtime. The combination of those would be deadly for a dogfooded compiler.
Besides that, C and Objective-C are technically exceptions because the most popular compilers for them are all written in C++. But I didn't count them because C++ is in the same language family.
- CoreLib is entirely C# (let alone anything outside it)
- Low-level facilities like thread-locals, instead of C++, are now C# + compiler idioms + C# to unmanaged exports mostly
- Type system facilities have more of their code moved to C# as well
- Memmove/memcpy and other performance-sensitive paths, that are part of CoreLib, are either pure C# using its SIMD API or special-cased in compiler (like unrolling of copies, comparisons (including text) of constant lengths and similar last-mile optimizations)
In addition, ILC (IL AOT Compiler) is fully written in C#, although uses the same back-end as JIT.
JIT could have been reasonably written in C# now that there is NativeAOT but would require more work (also considering that it's a GC-based language), apparently it was considered at some point but ultimately there isn't just capacity to do so - .NET compiler team is fairly small.
Some members of that family are fully self hosting. E.g. to build SBCL, you need an implementation of Common Lisp. It doesn't have to be SBCL.
No, that’s exactly what Ken Thompson was describing in Trusting Trust. You always have to bootstrap the compiler, but once you do, you use the compiler to compile itself and all future versions. So you use that circularity to hide the evidence of a back door in the compiler. It applies to all bootstrapped languages.
Now, sure, if your chain does not involve C, then it is immune to a compromised C compilers; how could C compilers do anything to it, that are not installed or run?
However, you're vulnerable to a doctored binary of your language.
Bootstrapping your Language X from C helps you here also, for multiple reasons.
Reason 1: there is no recursive cycle when you externally boostrap from another language. Even if a C compiler is doctored in order to recognize that it's boostrapping Language X, and do something malicious, that malicious thing will not propagate to new Language X installations. Reason being that the build output of Language X is not used for further bootstrapping new Language X instances. The other language is always used for boostrapping. All proliferation of the hack has to propagate through the ecosystem of that other language.
Reason 2: Language X is less popular than the boostrapping language, which makes its ecosystem vulnerable. If a vulnerable language with a tiny ecosystem is boostrapped using ready-made binaries of itself, those binaries are likely only available from one site. If those are infected, it's game over.
Reason 3: Language X development does not develop the host language (such as C). C is not of interest to the Language X project, and can be conservatively used, so that Language X will build fine with a 20 year old GCC. Even if a bad actor attacks Language X by getting malicious code upstreamed into GCC, that attack won't appear in old versions of GCC. (By the same token, Language X written in Language X could also use a conservative dialect of Language X for boostrapping, so that a many-years-old implementation of it with a widely known SHA-256 can be used.)
I think obscure language X being written in language X is more of a security benefit because it is more likely that a much greater portion of those who use language X are better able to recognise weirdness than if there are parts of your computing environment that are essentially blackboxes.
There is a defense against the trusting trust attack. It basically involves using two different compilers and checking their output matches. You can find the article about it if you search.
I thought Dr. David A. Wheeler's 2009 doctoral dissertation "Fully Countering Trusting Trust through Diverse Double-Compiling" had been generally accepted as proof that you could?
If the compiler which understands it is in some widely used language, things are easier because we likely already have it. If we don't have it, it's likely nicely packaged.
We also likely have it on a platform where that language we are trying to build has not been ported. That gives us a big head start in porting.
Fair, but, hold on, shouldn't you just make sure that if you're working on version 1.2 of your language, you compile it using your compiler from a stable 1.1 build? (other than maybe in tests where you need to test new features etc)
This is not a problem specific to programming languages, but to any other system that uses part of itself
This is where I suggest reading Ian Piumarta's Open, extensible object models [1] and later his COLA paper [2]
Piumarta tried to imagine an entire late-bound, meta-circular system based around message passing, starting from as little C as possible to build a Smalltalk-like language. It is a small paper that had be obsessed for a year trying to imagine how to implement an entire OS based on it. Worth a read.
And in the theme of meta-circular Lisp, he also worked on Maru [3]
1: https://www.piumarta.com/software/id-objmodel/objmodel2.pdf
[0] https://github.com/Clozure/ccl/blob/master/lisp-kernel/gc-co...
I disagree with a lot of what Jonathon Blow says, but he’s correct when he said that even the most trivial of projects are regarded with awe, as if they’re some insane and innovative skunkworks project. There are problems that need solving, that are in dire need of solving, but nobody seems to care about them. What we end up with is more and more bloated frameworks and languages that nobody uses after a week.
In line with that, there needs to be a good FOSS replacement of Unity. I know Godot exists but more open game engines will help.
Those are just a few off of the top of my head.
The answer to your earlier question, of course, is that—to some people, anyway—trying to make something good is far more satisfying than trying to make something popular.
There is an infinite number of problems out there that "need" solving. The thing is I don't really care about those problems. They'd have to pay money to get me to even think about them. These "trivial" projects though? Those provide me with plenty of intrinsic motivation. I like working with the unseen infrastructure that everybody takes for granted.
What’s old can easily become new again if a technology change in one corner of the system propagates outward, modifying the constraints & requirements on other components.
Someone could write a modern lisp that sat on top of a Xen unikernel that would work in a similar way, the compiler could generate native instructions though.
I think you would need to examine the Symbolics microcode carefully to categorize what it was doing.
Nothing high-minded is needed, you just implement what you implement. Operating systems (including Lisp ones!) existed before C-and-Unix and they'll exist after C-and-Unix (haha just kidding there is no "after C" or "after Unix").
(Offended Rust fans, please note that your pet language is just as free to escape Unix, or not, as any Lisp ever was.)