The compiler for Teal, a typed dialect of Lua
github.com
github.com
One of the biggest difficulties I have with managing a project as large as Grid is, is that you just barely start to become large enough for one person to have difficulties not only keeping all the finer details in your brain, but writing extensive documentation is exceptionally time expensive. And we do a really bad job of it.
We're supposed to be launching Grid Engine 10 sometime between now and March, but I don't think we're going to make it, because we haven't finished writing documentation for the next new major version.
So I looked into tooling to get around these problems, but you have to hint at types in Lua comments. I also looked into typed variations of Lua, but there's a few, and some of them are all-or-nothing solutions.
Teal has been out for a while, but I'm always pleasantly reminded of what exists in the Lua space, and how people continue to create great solutions for what I accept to be a more niche language.
I hope that these solutions continue to be shared around on HN, because even though Lua's ecosystem is small enough for many of us to recognize one another, there's a lot more tech out there than Lua and you forget what people are working on.
For as mature as Lua is, there are still complimentary solutions that are lacking in maturity or solutions that still don't exist all together. It's a very exciting space for developers.
Yeah, Teal was motivated by my experiences developing LuaRocks, the Lua package manager, which is also a significant pure-Lua application, as well as other experiences working with large Lua codebases.
We've been making some good progress on the compiler and the overall tooling (VS Code and Vim plugins, soon general LSP), and I can say that the overall development experience in Teal already feels smoother than pure Lua thanks to the compiler assistance.
My intention with Teal is to be able to support use-cases such as yours, Love2D, and so on. In my FOSDEM talk earlier this month ( https://www.youtube.com/watch?v=OqXbnaDR8QY ) I mentioned some of the challenges involved in supporting more existing Lua codebases, and how I think we'll need to add a bit of a metaprogramming layer in order to support more smoothly things like homegrown class systems (like the one I see in Grid, in the very first example in your homepage). So I would love to get feedback as we start to explore this space. I want Teal to remain a minimalistic and pragmatic project, so the goal is not to develop a super-general meta-programming system, but one that solves concrete problems Lua developers face when integrating Teal.
We'll continually be making efforts to push through our documentation and marketing struggles, so I hope to provide some feedback later this year when we've made the time to try implementing Teal as a solution.
Thanks, Hisham!
If it is possible, one could run same language across system level, game logic level and GPU shader level. This would be fantastic because you often don't know at which level it would run fastest. When algorithm at game logic level is not fast enough it would be possible to move preparation stage to compile-time (prebaking) and parallelizable sections into GPU, all by using same basic language.
There has been a fair bit of discussion about this design decision/limitation recently in other channels. Seems like that's one of the main safety checks you'd want/expect from a compiler.
That said - Pallene[1] and Nelua[2] are two very promising projects in similar spaces (although both are offering more than Teal's scope)
And there is a productivity gain from not having to consider nils in your types, because handling often requires code, annotation, or thought. But there is a loss of productivity to fixing nil errors that aren't caught by other typing features.
Places where the typing in Teal is still helpful: accessing the wrong thing, typos of variable names, wrong number or type of arguments to functions. That covers most of the statically-detectable errors.
Pallene, on the other hand, is more focused on performance. The tradeoff is that it doesn't cover the entirety of the Lua language, just the features that we can generate fast code for. The intention is that you can write the performance-critical parts in Pallene and glue everything together using Lua (or Teal).
That said, I certainly wouldn't be surprised if we saw more cross pollination of ideas between these two languages in the future :)
We definitely like to encourage the use of types (we <3 types!), but, just to clarify, Teal does support hybrid Lua/Teal codebases well, by use of definition files a la TypeScript. The Teal compiler itself is currently a "main program" written in Lua and the "core compiler" written in Teal. :)
> A Typed Lua 2.0, if you want to call it that way.
please don't, people are already confused as is. :)
(For those not well-versed in Lua history like Hugo and me :), "Typed Lua" was an earlier academic project, which had different design criteria such as being able to type check unannotated "idiomatic" Lua, which made it a much more interesting research project and a much more difficult industrial one — Teal is very pragmatic, with the intention of being immediately usable but no intention of being academically interesting, more in the TypeScript mindset. This talk goes through the history, for those interested: https://www.youtube.com/watch?v=VUThGgxOf28 )
> That said, I certainly wouldn't be surprised if we saw more cross pollination of ideas between these two languages in the future :)
For sure! There are some exciting possibilities there!
Hey maybe the linker could be called goauld.
We need better and more rigorous terms in computing science. This use of the compiler word blurs the meaning of interpreted vs compiled languages.
I was under the assumption that it would generate executable machine code, not Lua source code.
You can have terms that are precise and unchanging, and terms that are useful in today's computing landscape, but you can't have both.
Grace Hopper, who coined the term "compiler" used it for what we'd today call a "linker-loader".
Many C/C++ compilers emit textual assembly code which is then assembled as a separate step to machine code. The machine doesn't execute assembly, so are those still "compilers"?
Nikaus Wirth's Pascal compiler compiled to bytecode, not machine code. Likewise, almost all Java and C# compilers today target bytecode and not machine code. Are they compilers? What if I take that bytecode and run it on a Java processor (https://en.wikipedia.org/wiki/Java_processor)? Was javac not a compiler before Java processors existed but spontaneously became a compiler once the chip was shipped?
The V8 JavaScript engine parses your JS, compiles it to bytecode, runs that in an interpreter, uses the results of that to re-compile the code to machine code, and runs that again. The user never explicitly invokes a compiler or sees any machine code written to disk. Is V8 an interpreter? A compiler? Both? If you never run a JS program long enough for the JIT to kick in, is it still a compiler?
Some emulators for game consoles take game ROMs compiled to machine code for a different architecture and interpret them a virtual instruction at a time. Others take the ROM and recompile its machine code to new machine code for the host architecture, so they just treat the original machine code as an input format. Does that mean whether the ROM contains "executable machine code" versus "source code" depends on which emulator you run it in?
When Apple transitioned from 68k to PowerPC, newer PowerPC Macs ran old 68k apps in an emulator. Did those apps still contain "executable machine code"?
Modern CPUs implement large instruction sets by translating them down to microcode. If you compile to the larger ISA which is not what the chip actually executes, are you still outputing "executable machine code"?
The definition of compiler has never assumed generating executable machine code. Already in the 1970s, Pascal compilers have generated P-code (a form of "bytecode" in Java parlance), which was then interpreted. In the 1980s, Turbo Pascal produced machine code directly.
I've seen the neologism "transpiler" being very frowned upon by the academic programming language community precisely because a compiler is a compiler, no matter the output language — my use of "compiler" there was precisely because of my academic background.
I don't mind the term "transpiler" myself if it helps you understand it's a source-to-source compiler, but then, you don't see people calling the Nim compiler, which generates C code then compiles it into machine code, a "transpiler", even though it is a source-to-source compiler. In the end, "compiler" is the all-encompassing term for a program that takes code in one language and produces code in another, be it high-level or machine language — and yes, that means that technically an assembler is a compiler as well. And since we're talking assembler, most C compilers do not generate executable machine code either: gcc produces assembly, which is then turned into machine code by gas. So gcc is a source-to-source compiler? Is Turbo Pascal more of a compiler than gcc? I could just as well add an output step in the Teal compiler to produce an executable in the output using the same techniques of the Pascal compilers of the 70s. I don't think that would make it more or less of a compiler.
As you can see, the distinction of "what is a transpiler" reduces to "what is source code" or "what is a high-level language", the latter especially having a very fuzzy definition, so in the end my sociological observation on the uses of "transpiler" vs. "compiler" tends to boil down to people's perceptions of "what is a Real, Hardcore Compiler". But being a "transpiler" or not doesn't say anything about the project's "hardcoreness" either — I'm sure the TypeScript compiler which generates JavaScript is a lot more complex than a lot of compilers out there which generate machine code.