The Koto Programming Language
koto.dev
koto.dev
I found rhai's syntax very straightforward, and I could almost accomplish my needs just by looking at some basic examples.
I use Rhai in wasm, and it can handle real-time audio blocks, which is really impressive:
Rhai also offers some safety guarantees: no panics, no stack overflows, etc. Rhai seemingly requires slightly less ceremony when interfacing with Rust: direct use of many things, as opposed to implementing a trait to interact with Koto.
(Disclaimer: I spent 5 minutes skimming the docs of each.)
On the other hand, you can always write your own Rhai interpreter if necessary. And if you restrict the language to a limited set of features, which you need to do anyway to keep it realtime-safe, you could even compile it to native code.
> and it can handle real-time audio blocks, which is really impressive:
Any scripting language can do this as long as you stick to operations that don't cause memory allocations, system calls or other non-realtime-safe operations.
For example, you can use Lua to write process functions for Pd objects: https://agraef.github.io/pd-lua/tutorial/pd-lua-intro.html#s...
The question is rather how much you can do in a given audio callback.
---
All that being said, Glicol is very cool!
I gave Lua a shot, but getting the toolchain set up in wasm was a hassle:
https://bytedream.github.io/litbwraw/introduction.html
So at least I can say Rhai has some advantages on syntax and compatibility with wasm
I just don't really understand why Rhai uses an AST walking interpreter, that's basically the least efficient way of implement a scripting language. Once you have an AST, a byte code compiler/interpreter is not really hard to implement, so I'm wondering why they knowingly leave so much performance on the table...
I'd start by trying UniFFI [1] which looks much simpler than the approach of manually writing a C API and using that as a foundation for higher-level language bindings.
This would also likely be the starting point for a package management system (if there ends up being demand for one). Rust doesn't have a stable ABI so to make sure that dynamically loaded Rust packages are compatible, either Koto would need to be in the business of Rust toolchain management so that packages can be recompiled when needed, or an API layer would be needed. There are some projects that provide ABI-compatibility shims but I don't like the idea of having two separate approaches to FFI, so I'd want to try to build on the foreign-FFI layer once it's in place.
I'm half hoping that by the time I'm interested in working on this Rust will have decided to pursue ABI stability. And there's also something in the back of my mind that's yelling 'Wasm!' at me but I would need someone wiser to convince me that it would be the right direction.
I'm fine with the Rust API breaking compatibility.
Before 1.0 I'd want to address at least:
- the FFI / package management topics mentioned above
- async support: https://github.com/koto-lang/koto/issues/277
- extend the parser to support an autoformatter: https://github.com/koto-lang/koto/issues/286
...and then have a larger number of people using it in projects without major issues coming up for a good while, e.g. a year+.
At the time we did it with Lua.
We extended Nginx and Envoy-proxy with a Rust library (and a server), and added a Lua interface, so users can further tweak the config and the flow.
https://github.com/zuriby/curiefense/tree/main/curiefense/cu...
After looking at this for 5 minutes it seems it is better than rhai according to my metric. But not necessarily better than lua.
If it helps here's a git blame helper script I made for Helix (as an example of practical code without any effort put in to making it readable for others): https://github.com/irh/dotfiles/blob/main/scripts/.scripts/g...
e.g. https://github.com/irh/dotfiles/blob/main/scripts/.scripts/t...
Nice! Now, if it only had type support.
Edit: Sure enough, "whitespace is important in Koto, and because of optional parentheses, `f(1, 2)` is not the same as `f (1, 2)`. The former is parsed as a call to f with two arguments, whereas the latter is a call to f with a tuple as the single argument."
One of the problems of terse syntaxes is that one typo away from a syntactically valid program lies another syntactically valid program with entirely different semantics.
I prefer a syntax that has enough "gaps" between syntactic constructs, so that a single typo usually leads to an obvious syntax error. In this regard, Python or Java or TS are comfortable, Haskell or Lisp or C are okay, and Coffescript or Scala or C++ are terrible.
I'm guessing because "it's cleaner/simpler", but that's a shallow understanding of those words. Just because there are two characters fewer on the screen doesn't make the code simpler. Simple semantics are what you should aim for and inconsistencies like these throw a wrench into that.
For example, strings in JSON vs YAML. Isn't it "simpler" to not have to quote every string? So long as your string isn't "no" that may be true. So now instead of a simple mental model of "every string must be quoted" it's "strings don't need quotes except for these specific exceptions that cause issues: <list>". So much simpler... sigh.
<virtual high five>
My thinking is that the potential footgun here is outweighed by the win of paren-free calls for quick scripting / rapid iteration, but it certainly counts towards the language strangeness budget [1] so I figured it was worth pointing out in the guide.
[1] https://steveklabnik.com/writing/the-language-strangeness-bu...
I don’t understand why… inferred typing is nearly as easy to use while being more robust.
For me the biggest gap in programming languages is a rust like language with a garbage collector, instead of a borrow checker.
Rust has a lot of features that should be strongly considered for the next generation of programming languages such as
result/sum types
Inferred typing
Not object oriented or overly prescriptive with functional programming.
I think the closest language to filling that gap is occaml (I am not familiar with it).
I have coworker's that are more skilled in domain logic that can write basic self contained programs, but stuff like traits, OOP functional programming is a bridge too far.
I feel like a language that fills that gap could be useful, context is a manufacturing ERP system.
Written by Bob Nystrom, author of Crafting Interpretors.
At first glance it appears to be object oriented, which is against preference but not a deal breaker.
However error case looks to be try catch which is a deal breaker.
Wren uses coroutines ("fibers") for error handling, which is unusual. But yes, ultimately it does seem to be equivalent to try-catch.
Besides, it's "done" let it be done.
Also has pattern matching which I should also have mentioned in my top level post.
[0]: https://blog.nestful.app/p/why-i-rewrote-nestful-in-gleam
[1]: https://porffor.dev/
Also, Elixir is working on gradual types, which is something I would keep an eye on. https://hexdocs.pm/elixir/main/gradual-set-theoretic-types.h...
Such a great programming paradigm that sadly has few breakout successes
I otherwise am unsure what you mean
If roc ends up being even close to that productive for me, I really don’t care how widespread it is. I like these thoughtful unrushed languages that I can support for years without stress.
Thats the real tragedy. Elm was a good language but the culture around it means it could never achieve critical adoption. It is great technology for building frontends but how easily will I be able to maintain that app as the web changes over the years?
I doubt it's a silver bullet for everyone, but it's been phenomenal for me as a solo dev w/ my own product. I feel so much better about my elm code than I do about my React code. I have to do all the other things, like marketing, sales, training, support, back end, etc. It's nice to launch a UI and know it'll last (and also that it's easy to refactor and augment too!). Elm has turned my into a statically typed functional fanboy. It's been a gateway drug to Haskell, OCaml, and F#.
I see the same promise with Roc, although I bet Richard will update it more frequently than Evan w/ elm after 1.0.
That said, I wouldn't compare it to scripting languages. The lack of implicit conversions / traits / ad-hoc polymorphism means it's not that convenient for scripting.
I also want to point out that Araq quickly took the criticism and his commits look a lot better now.
Also, core Nim team is currently working on their own rewrite of Nim compiler, called Nimony, that should evolve into Nim 3.
Refs:
fork - https://github.com/nim-works/nimskull
commit 'drama' - https://news.ycombinator.com/item?id=32021299#32023998
nimony - https://github.com/nim-lang/nimony
result/sum types = enums whose cases have associated values
inferred typing = Swift "type inference"
Not object oriented or overly prescriptive with functional programming. = Uh, yes
Those features map to Kotlin too
But if I were to create a self-contained (that is, non-JVM) "Rust-like but with GC", I think it would look a lot like Kotlin.
Rust-like but with GC is already Scala. The reason is that Scala (unlike Kotlin) focusses on immutability, which makes it more similar to Rust. It's actually even easier to use (no borrow checker) but at the cost of performance.
After Rust, I would see F# as the next closest language, quite far before Kotlin.
Maybe like a third of Scala. But yeah, Scala has a lot of good parts, it's also just a huge surface area. I agree that you could pluck a subset out of Scala and make it this "Rust-like but with GC" language.
But it's nigh-impossible to actually make that "only use this subset" idea work in practice, because it just ends up being a bikeshed.
But you're right that the same can be said of the OO focus of Kotlin.
> But you're right that the same can be said of the OO focus of Kotlin.
The difference is that Scala focuses on immutability, which makes the experience much closer to Rust than to Java. Whereas Kotlin does it exactly like Java does. Were Kotlin to focus on immutability like Scala does, than I would recommend Kotlin over Scala for what OP asked for.
That's why C++ is so badly designed, you can do anything in it, so people do and no two C++ codebases are the same. Your codebase at work is written one way, the libraries it uses in another, your personal projects use something else entirely. It's a huge mess where nothing is compatible with each other and the mental load of switching between projects is untenable.
For example, look at Golang. The syntax is very plain and it's by design so that everything looks the same.
Then take a look at Lisp. On the surface the syntax looks all the same but in reality, everyone can write their their own macros and people usually do. Then, you can barely understand what's going on if you are new to the codebase - but on the other hand, the flexibility is enormous.
Scala is somewhere in between. When you hire someone for Golang, you don't really need to make anything clear in terms of code style (I believe). In Scala (or Lisp) you should definitely clarify the style that you use. It's actually normal (I've hired pretty big number of Scala developers over my career and I have also rejected job offers due to the style the company uses which doesn't match what I like).
C++ probably is alike, but I don't really any C++ experience.
There are lots of ways to write Scala, to its detriment.
This isn't meant as a knock on Scala! It's a great language and in particular I think it's been influential on other newer languages. It's just that it does have this "multiple dialects" issue.
Rhai Rune Dyon
Mun is not dynamic, however it does not have string support afaik.
Kotlin and Swift may be better candidates than these scripting languages for my imagined usecase.
Come to think of it, maybe I don’t have a point other then there is so many scripting language’s inspired by rust that is dropping a major convenience feature, that I am surprised is negotiable (inferred typing ).
Rust's dyn Any corresponds better to C#'s Object; dynamic exists to interface with dynamic languages and is rarely used.
Its ahead of its time in basically every aspect, it’s 100% compatible transparently with the whole C# ecosystem, it’s mature yet still evolving.
The type system is something I never saw before : creating types is so ergonomic and fast that you can create custom type for basically any value of your program without boilerplate if you want.
It’s really a refreshing language that anyone should try.
What I really love with it is that it’s hard to write (when you are learning it) but incredibly clear to read.
It looks like Koto supports type hints - not sure if these are checked at compile-time, run-time, or both.
> I don’t understand why… inferred typing is nearly as easy to use while being more robust.
Is it (nearly as easy to use)? Every static type system seems to have special cases and exceptions which are avoided by dynamic typing. I'd love to find one that's actually simple.
Also, it's definitely not nearly as easy to implement - which is important for a language being designed and built by a small team and targeting a lightweight runtime.
It looks like Koto only checks types at run-time. That means its type annotations are essentially shorthand for something like Python's `if not isinstance(...): raise TypeError`.
The hints aren't used for other purposes at compile time yet, but they could enable some warnings.
I think this is the real reason why there are so many dynamic language implementations. If you want to implement a dynamic language, you just slap a type tag on your runtime objects and boom, your "type system" is done.
Dynamic languages get a lot of expressiveness "for free", whereas having a really expressive static type system requires a lot of work. It's not that hard to get a type system on the level of C, but if the language is interpreted, it's still going to be pretty slow.
I do think there can be benefits to having typing in a scripting language (and not a bolted-on type system like typescript or mypy). It's much easier to work with an FFI if the type system of the scripting language maps closely to the implementation language. It also does make it much easier to optimize the language down the line, if that becomes a priority. Making a fully dynamic language efficient is very, very difficult.
I agree, though I often think Rust is probably good enough. You can use RC or grab a GC crate. It's not as ergonomic as just assuming all values are GCed, but I think it gives the flexibility and fast iteration of working in a GCed language.
Both come with very powerful features, but you don't need to use them and you can use libraries accordingly. Especially Scala can be made to feel very similar to python
Source? AFAIK there's no confirmed upcoming allocators APIs, and even if there was they would just allow reusing the builtin `Box`/`Vec`/etc etc with custom allocators. This is not much different than what you could do with a custom type, so I find it hard to believe it would allow garbage collectors that are not possible right now.
I cannot agree more that's the much needed sweet spot/Goldilock/etc. Personally I have been advocating this approach for some times. Apparently the language is already widely available and currently has stable and wide compiler support including the venerable GNU compiler suite (GDC). It also one of the fastest, if not the fastest programming in existence for both compilation and execution [1].
It has been beating Fortran in its number crunching territory, no small feat given the Fortran pedigree with many languages still depending on Fortran based infrastructure for their number crunching capabilities including Matlab, Julia, Rust, Go, C, C++, etc [2].
It also has a nice bulti-in REPL system due to its very fast compilation and execution [3].
For an excellent overview of D programming language please check this presentation at ACCU conference [4].
[1] D website:
[2] Numeric age for D: Mir GLAS is faster than OpenBLAS and Eigen:
http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...
[3] Why I use the D programming language for scripting (2021):
https://news.ycombinator.com/item?id=36928485
[4] How DLang Improves my Modern C++ and Vice Versa - Mike Shah - ACCU 2024:
"Everything should be built top-down, except for the first time" (See #15 in https://www.cs.yale.edu/homes/perlis-alan/quotes.html)
Because dynamic typing has its own advantages, which are worthy of experimentation even if you perceive absence of static typing as a weakness.
Gradual typing can offer us the benefits of both worlds.
> inferred typing is nearly as easy to use while being more robust.
Implementing type inference can be fairly trivial if your types are all disjoint. Hindley-Milner type inference is well studied and there's plenty of literature.
But as soon as you introduce subtyping, the traditional methods are not sufficient. It's only in the past decade that good solutions have been discovered, notably Dolan & Mycroft's MLsub[1], based on Dolan's Algebriac Subtyping thesis[2], and Parreaux & Chau's MLstruct[3], which uses a boolean algebra approach[4]. Type inference with subtyping is not a solved problem - these developments are big steps forward, but there are still open problems under research.
Subtyping doesn't imply object-oriented. Structural typing (ie "static duck typing") is a form of subtyping.
[1]:https://github.com/stedolan/mlsub
[2]:https://www.cs.tufts.edu/~nr/cs257/archive/stephen-dolan/the...
Gradual typing has much the same overhead as other kinds of dynamic typing. It's broadly appropriate as part of the interface between separately-developed software components, and not very much otherwise.
A program in a gradually typed language which does not use `dynamic` is fully statically checked and has no overheads. The overheads only appear when `dynamic` is used.
This is why it offers the best of both worlds. We get static typing everywhere where we're not using dynamic, and when we do use dynamic, we can basically use it however we want and have the benefits of dynamic typing, because from the static perspective, dynamic ~ dynamic, regardless of what the runtime type is.
The important innovation is that consistency (~) is not transitive - so it doesn't allow us to implicitly convert one type to another - only conversions to/from dynamic are implicit, and other conversions must be done explicitly.
Obviously, this provides an "escape hatch" from static typing where we do use it - we can get around some static type check by casting to/from dynamic explicitly, but this works out well in practice.
C# is an example of a gradually typed language since v4 which introduced `dynamic`.
Haskell is close , in that it has the type `Data.Dynamic`, but it doesn't support implicit conversions to/from it, which would be possible if it had a consistency rule. We have to do the conversions explicitly.
GC, sum types, result/option types, interfaces, no OOP, fast compilation.
Favors minimal text entry. You keep the type information in your head and enter fewer tokens. Historically you also reduced the work and improved responsiveness for the interpreter, which might have been running on an 8- or 16-bit computer at 5 MHz.
It makes perfect sense to use Rust as the main language for your application but have areas which are either in the prototype stage, need to be written quicker or which simply don't need the performance. But Rust does not offer a way to enter such a less strict context and such proposals keep getting shot down by the community, even when they are made from core members of the Rust team.
Contrast that with C# which has a dynamic keyword, allows enabling a checked context (not just in code where you can't miss it but also from csproj), has reflection, etc.
I really want Rust to succeed but sometimes the attitude borders on zealotry.
It is like saying browser should be coded in C# because C++ can't be use instead of JavaScript...
Your tone is insulting and has no place in technical discussions.
Yes, it is very multi-paradigm and can be used in many domains, but it's not trying to be C# and it can't be C#. I would love to see Rust# as a language, but Rust itself cannot be that language.
Dynamic can be implemented in a compile-to-native lang. Contexts with different rules as well. Reflection support would likely have overhead which would need a granular opt in mechanism but is very likely possible.
Similarly many features rust is missing like compile time reflection, field in traits, ...
Nope - and indeed, we're seeing Rust used in a more diverse set of applications than C++ (e.g. there are several Rust web frontend frameworks, it's a popular WASM language in general, etc)
However, Rust is targeting the same kind of general constraints as C++ for development and deployment, which means it can't add anything that would depend on a runtime or impose an undue burden on users. (Of course, C++ cheats in this regard - RTTI and exceptions - but Rust matches that, and doesn't go beyond.)
> Dynamic can be implemented in a compile-to-native lang.
Requires runtime functionality, and it's not really clear what the resulting semantics would be, anyway: aside from the usual type-safety concerns, how do you deal with lifetimes and other compile-time constraints?
> Contexts with different rules as well.
What kind of different rules? The problem is that any deviation from the rules needs to be reconciled at some point, and that reconciliation has to be watertight: you can't weaken the guarantees somewhere that interacts with safe Rust code, because the weakness you've introduced can spread. This is already a pretty significant issue with unsafe Rust.
Similarly, moving to a higher level of abstraction has similar issues: how do you reconcile the interactions of GC'd objects with the rest of Rust, which expects deterministic destruction and somewhat-predictable object lifetimes?
> Reflection support would likely have overhead which would need a granular opt in mechanism but is very likely possible.
If you're already committing to a granular opt-in mechanism, you might as well use a library, which offers you more options: https://docs.rs/bevy_reflect/latest/bevy_reflect/
> Similarly many features rust is missing like compile time reflection, field in traits, ...
I'll give you compile-time reflection; that would have been quite nice to have, but the Rust Foundation alienated the primary person with a plan (https://thephd.dev/i-am-no-longer-speaking-at-rustconf-2023), so who knows when we'll see the next proposal? I agree that it's a shame, but there's usually ways to work around it (proc macros can be used to patch over a lot of Rust's relative deficiencies)
Field-in-traits has been discussed before, but is complicated due to the borrow checker: https://internals.rust-lang.org/t/fields-in-traits/6933/1
In general, the borrow checker is the primary impediment to copying features from other languages; it's just generally non-trivial to fit them into the Rust paradigm without significant R&D. That's why I think a higher-level Rust would have to be a separate language, not an extension of Rust proper: resolving the collision of semantics between abstraction levels is just too difficult in the general case.
I personally haven't found Rust that difficult to prototype in, when I need to I just liberally use clone/Arc/RefCell.
I see the main benefit of these scripting languages as being able to write/run code at runtime, e.g. live coding music or mods for video games.
Rust is actually a pretty nice language for prototyping IMO. I agree with your take - Rust has many escape hatches you can use to develop quickly. Then when it comes time to clean up, it's obvious where the deficiencies are (look for all the clones and unwraps, etc)
> I see the main benefit of these scripting languages as being able to write/run code at runtime
Thank you. Some commenters seem to think a scripting language somehow reveals some deficiency in the core language. Reality: not all code is available at compile time. Many applications need some way to inject code without recompiling.
You're right that it would be great with full LSP support, but for quick scripting, I mostly just want a linter.
`dynamic` has a very narrow use-case and generally shouldn't be used at all (even when prototyping)
I believe it was introduced to make working with poorly designed external libraries easier (e.g. Windows/Office libs)
For production systems, people just use Python + Rust when needing a balance between dynamism and strictness. The tooling to mix them is very mature and the communities are overlapping.
With uv becoming the defacto packaging solution for python, I expect the border to blur even more in the future.