Cool, and the paper mentioned in the document "Dependent Types for Low-Level Programming" [1] is such a gem.
[1]: https://people.eecs.berkeley.edu/~necula/Papers/deputy-esop0...
148 karma · joined April 27, 2016
[1]: https://people.eecs.berkeley.edu/~necula/Papers/deputy-esop0...
I used to make many tools with libclang Python bindings to automate some chores of refactoring. I don't remember if one could expand the macros using it, and I was told that lexer and preprocessor are messed together in Clang. So it should be quite hard to extend the existing framework.
I would definitely go into this direction, but for now it just looks like some final boss, let me finish some early quests.
I have a friend currently writing a GC in C and I was making it a project to test around my ideas (where problems with generic containers, compile-time eval and static reflection kicked in so early). But these you mentioned sound more important to me, I have more friends working on RISC-V projects, I should find some inspirations from their projects. Thanks for the mention!
Yeah! The criterion is just there like you said.
> Do you have an opinion on the dotnet version of generics?
I'm not familiar with dotnet languages, but I have much experience implementing generics, especially how to avoid getting the poor C++ templates. Also, I believe traits (not that of C++) and typeclasses (Rust/Haskell) are definitely needed, and it's already proven too, to pair with the generics feature.
> Ceph.
My former job was to write a distributed object storage in Go and it was heavily inspired by Ceph. But subset of the design could make much benefits for a smaller company, since the initial idea to write a new one was from the pain of doing rebalancing in Ceph... oh you want to know the solution? Manual partitioning and migration by our SRE/Ops team lol. It's unexpectedly effective.
I think I gotcha. Oh no.
> But I haven't finished fleshing it out and proving the type system, so I really can't share it yet.
Any profile I could follow to wait for this to happen some day? I've made several pen-and-paper attempts on some problems you just mentioned, and further ones would be "strict aliasing", "pointer provenance" and many which appeared in all the dicussions here. It feels like you've already done many stuff, but I can't find your profile anywhere.
I found it might be possible to tackle "strict aliasing" and "pointer provenance" with a type system and I would head down to it early. The approach might sound like Rust's `MaybeUninit` but I didn't think much about it.
I've already implemented procedural metaprogramming in a JS dialect of mine [1], it's also trivial to use it to implement compile-time format strings. I would improve the whole experience in this new C-like language.
Again, very very practical ideas here. Great thanks!
[1]: https://github.com/rowscript/rowscript/blob/16cb7e1/core/src...
Exactly, that's the most important takeaway I got from all the discussions here: I will be patching issues while people are flooding in for better general approaches.
I might shift much of my direction, but lucky that there are many "lessons" (like ooc) to learn.
So I was talking about the direct C codegen approach, but there's still much of some mess like one needs to choose a C standard and knows how to verify the generated code.
I got stuck at how to tackle "new language doesn't compile with a standard C compiler" for many times, but my solution is much worse than yours: Like LuaJIT, they left one unreadable "minilua" C file [1] to bootstrap some stuff, we could have a source-code version of the "new C" compiler, compile things twice. That sounds bad.
For languages with a very advanced type system that compiles to C, I could only think of Koka [2], which translates the "algebraic effect and handlers" code into pure C, achieving pure C generators, coroutines and async/await without the support of setjmp/setcontext. But the generated C code is unreadable, I would definitely think about how to handle the readability and debugging issue with sidecar files.
[1]: https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/host/minilua....
And the first sentence [1] of its pointer type introduction, exactly says everything I said here...
> [...] greatest motivation behind ATS is to make it employed to construct safe and reliable programs running in OS kernel.
Also found this interesting Reddit thread [2]. Time to bring some old gems back with good ergonomics now, it won't be that hard. Hold my beer for a while.
[1]: https://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML...
[2]: https://www.reddit.com/r/ProgrammingLanguages/comments/uacib...
I also learned that in LLVM IR, they have the implicit null checks [1] to replace the guard with just a signal handler, so it won't hurt the branch predictor too much.
So I believe there are many options upon "debug" and "release" profiles for the bound and null checking here. Very good design space.
[1]: https://llvm.org/docs/FaultMaps.html#the-implicitnullchecks-...
It's pretty fun to think about "Carbon to C++ is Kotlin to Java". One very important takeaway from all the discussions here is that, I cannot ship a language right to the target (small) community, as it's impossible to control how people decide to use this language. Which means, I have to focus much on how to improve the experience of application writing. Carbon would definitely be one of the inspiration.
Oh yeah, and I don't need to handle seemless integration with templates, I'm lucky.
Sorry that I didn't much clarify the "pain" though:
It's quite like the experience of using parser combinator in Rust, where you could happily define the grammar and the parsing action using its existing utitlies. But once you have to do some easy wrapping, e.g. to make a combinator called `parenthesized` to surround an expression with parentheses, the "pain" kicks in, you have to leave as many trait bounds as possible since wiring the typing annotations become tedious. That came up while I was using framework like `winnow`.
Async Rust kinda shares some similar characteristics, utility functionalities might bring in many "typing wirings" that could terrify some people (well but I love it though).
You don’t. Read the features I listed. One ends up with a C alternative frontend (Cfront, if you love bad jokes) including type system like Zig without any standard library. No hash tables, no vectors. You tended to write large games with this.
Like I said the main 3 groups of users, if you’re concerned about application writing, ask it. Rest of the comments talked about possible directions of langdev.
> Modules.
You write C++ and don’t know what a standard is. Motivating examples, real world problems (full and incremental compilation, better compilation cache instead of precompiled headers), decades spent on discussions. Economy would come for projects with modern C++ features.
> Threading.
If you know Rust and Go, talk about them more. Go creates tasks and uses futexes, with bare-bone syscall ABI. Higher level primitives are easy to use. Tools and runtime are friendly to debugging.
I wrote Go components with channels running faster than atomics with waits, in a distributed filesystem metadata server.
On CPU intensiveness, I would talk about things like automatic vectorization, smarter boxing/unboxing, smarter memory layout (aka levity, e.g. AoS vs SoA). Not threading niche.
> Strlen implementation and plan of low level programming.
Because I keep talking about designing a general purpose language. One can also use LLVM IR to implement such algorithms.
The design space here is to write these if necessary. Go source code is full of assembly.
> Pointer provenance.
Search for Andras Kovacs implementation of 2ltt in ICFP 2024 (actually he finished it in 2022), and his dtt-rtcg, you would realize how trivial these features could be implemented “for a new language”. I design new languages.
> libc.
Like I said, your happy new APIs invoke malloc.
> Headers.
C++20 modules are left unstable and unused in major compilers there, but it’s a standard. And C is ironically perfect for FFI, as I said, almost every programming language speaks C: Rust WebAssembly API is extern C, JNI in Java, every scripting language, even Go itself talks to OS solely using syscall ABI, foreign-function calls are only possible with Cgo. C was not just an application/systems language for some sad decades.
> Big elephants.
Since I was in the zoo watching tigers:
Mostly three groups of people are served under a language: Application writers, library writers, compiler writers (language itself).
I narrowed down and started “small” to see if people writing programs crossing kernel and user space would have more thoughts about C since it’s the only choice. That’s also my job, I made distributed block device (AWS EBS replacement) using SPDK, distributed filesystem (Ceph FS replacement) using FUSE, packet introspection module in router using DPDK. I know how it feels.
Then for the elephants you mentioned, I see them more fitted into a more general library and application development, so here we go:
> Threading.
Async Rust is painful, Send + Sync + Pin, long signatures of trait bounds, no async runtimes are available in standard libraries, endless battles in 3rd party runtimes.
I would prefer Go on such problems. Not saying goroutines and channels are perfect (stackful is officially the only choice, when goroutine stacks somehow become memory intensive, going stackless is only possible with 3rd party event loops), but builtin deadlock and race detection win much here. So it just crashes on violation, loops on unknown deadlocks, I would probably go to this direction.
> Optimization, hardware.
Quite don’t understand why these concerns are “concerns” here.
It’s the mindset of having more known safer parts in C, like a disallow list, rather than under a strong set of rules, like in Rust, an allowlist (mark `unsafe` to be nasty). Not making everything reasonable, safe and generally smart, which is surreal.
C is still, ironically again, the best language to win against assembly upon an optimizing performance, if you know these stories:
- They increased 30% speed on CPython interpreter recently on v3.14.
- The technique was known 3 years ago to be applied in LuaJIT-Remake, they remade a Lua interpreter to win against the original handwritten assembly version, without inline caching.
- Sub-techniques of it exist more than a decade even it’s in Haskell LLVM target, and they theoretically exist before C was born.
It is essentially just an approach to matching how the real abstract machine looks like underneath.
> libc.
Like I said, C is more than a language. Ones need to switch a new allocator algorithm upon malloc/free, Rust quits using jemalloc by default and uses just malloc instead. Libc is somewhat a weird de facto interface.
Yes! Zig has done a great job on many C-related stuff, e.g. they've already made it possible to cross-compile C/C++ projects with Zig toolchain years ago. But I'm still quite stupidly obsessed with source-level compatibility with C, don't know if it's good, but things like "Zig uses `0xAA` on debugging undefined memory, not C's traditional `0xCC` byte" make me feel Zig is not "bare-bone" enough to the C world.
> Micron and Oberon+ programming language.
They look absolutely cool to me! The syntax looks inspired from Lua (`end` marker) and OCaml (`of` keyword), CMIIW. The features are pretty nice too. I would look into the design of generic modules and inheritance more, since I'm not sure what a good extendability feature would look like for the C users.
Well BTW, I found there's only one following in your GitHub profile and it's Haoran Xu. Any story in here lol? He's just such a genius making a better LuaJIT, a baseline Python JIT and a better Python interepreter all happen in real life.
That's so great! But sad that no enough ideas and argument came up here. :'(
> How metaprogramming would work?
When it comes to "tactics" in Coq and Lean 4 (i.e. DSL to control the typechecker, e.g. declare a new variable), there are almost equivalent features like "elaborator reflection" in Idris 1/2 [1] (e.g. create some AST nodes and let typechecker check if it's okay), and most importantly, in Scala 3 [2], you could use `summonXXX` APIs to generate new definitions to the compiler (e.g. automatically create an instance for the JSON encoding trait, if all fields of a record type is given).
So the idea is like: Expose some typechecker APIs to the user, with which one could create well-typed or ready-to-type AST nodes during compile time.
[1]: https://docs.idris-lang.org/en/latest/elaboratorReflection/e...
[2]: https://docs.scala-lang.org/scala3/reference/contextual/deri...
> Lifetime and compilation speed.
Yes exactly, I was considering features from Featherweight Rust [3], some subset of it might be partially applied. But yes it should be super careful on bringing new features in in case of compilation speed.
It's also worth to mention that C compiler itself would do some partial "compile-time eval" like constant folding, during optimization. I know some techniques [4] to achieve this during typechecking, not in another isolated pass, and things like incremental compilation and related caching could bring benefits here.
[3]: https://dl.acm.org/doi/10.1145/3443420
[4]: https://en.wikipedia.org/wiki/Normalisation_by_evaluation
> Every feature must be crucial.
I want to hear more of your ideas on designing such language too, and what's your related context and background for it BTW, for my curiosity?
It is really interesting that in China people often ask their friends '吃了吗? (Have you eaten?)' rather than '嗨 (Hi)' in daily life. So initially I thought this post was describing something in Chinese w/o my noticing the URL.
And also, it's better to check out some metasyntax notations, e.g. EBNF https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_F...
And this is the link of the report from QQ (Tencent) official website: