Adding ANSI C11 C compiler to D so it can import and compile C files directly
github.com
github.com
He implemented the ANSI C compiler here in just 5 days.
Semi-related project, using libclang to import C++ directly (with C++ templates and all the other features, regular D has C++ linking and exception support built-in): https://github.com/Syniurge/Calypso
Some quotes from the forum https://forum.dlang.org/post/s7a37h$2bbv$1@digitalmars.com :
- Writing a C preprocessor is a nightmare. Fortunately, I already did that https://github.com/facebookarchive/warp and we can use it if we choose.
- The D code will be able to inline C code, and even CTFE it.
- There are a lot of wacky C extensions out there. We only need to implement currently used ones (not the 16 bit stuff), and only the stuff that appears in C headers.
- Without a C compiler, we're stuck with, wedded to, and beholden to libclang. I wouldn't be surprised that the eventual cost of adapting ourselves to libclang will exceed the cost of doing our own C compiler.
This is good work but let's be realistic - it doesn't work yet on any real C code I've tried it with.
Yeah, it doesn't support C headers yet. But an excellent prototype nonetheless.
Lol, he is a paid hype man for the Dlang community, and you should ask him about Symmetry Investments and if they are hiring.
I'm starting in July.
====
make
dmd -c -O -inline -release -ofwarp.o cmdline.d constexpr.d context.d directive.d expanded.d file.d id.d lexer.d loc.d macros.d main.d number.d outdeps.d ranges.d skip.d sources.d stringlit.d textbuf.d charclass.d
context.d(222): Error: Implicit string concatenation is error-prone and disallowed in D
context.d(222): Use the explicit syntax instead (concatenating literals is @nogc): "# 1 \"%2$s//\"\x0a" ~ "# 1 \"<command-line>\"\x0a"
context.d(223): Error: Implicit string concatenation is error-prone and disallowed in D
context.d(223): Use the explicit syntax instead (concatenating literals is @nogc): "# 1 \"<command-line>\"\x0a" ~ "# 1 \"%1$s\"\x0a"
macros.d(329): Error: Built-in hex string literals are obsolete, use std.conv.hexString!"FD 61 FD" instead.
macros.d(333): Error: Built-in hex string literals are obsolete, use std.conv.hexString!"FD" instead.
macros.d(338): Error: Built-in hex string literals are obsolete, use std.conv.hexString!"FD 61 62 FD FD" instead.
ranges.d(71): Error: Implicit string concatenation is error-prone and disallowed in D
ranges.d(71): Use the explicit syntax instead (concatenating literals is @nogc): "Buffer overflowed. Possibly caused by forgetting to " ~ "complete a git merge in your code."
Makefile:44: recipe for target 'warp.o' failed
make: ** [warp.o] Error 1
====
====
make
dmd -c -O -inline -release -ofwarp.o cmdline.d constexpr.d context.d directive.d expanded.d file.d id.d lexer.d loc.d macros.d main.d number.d outdeps.d ranges.d skip.d sources.d stringlit.d textbuf.d charclass.d
util.d(20): Error: function `loc.Loc.write(File* f)` is not callable using argument types `(File function() nothrow @nogc @property ref @system)`
util.d(20): cannot pass argument `& makeGlobal` of type `File function() nothrow @nogc @property ref @system` to parameter `File* f`
directive.d(470): Error: template instance `util.err_warning!(string, string)` error instantiating
context.d(264): instantiated from here: `parseDirective!(Lexer!(Context!(LockingTextWriter)))`
main.d(39): instantiated from here: `Context!(LockingTextWriter)`
Makefile:44: recipe for target 'warp.o' failed
make: * [warp.o] Error 1
====
Small price to pay for an elegant language.
That isn't why Warp was discontinued. (My last change to Warp was 7 years ago.) Even with my C code, it's unsurprising to have to update a few things after 7 years.
My version of the repository is:
https://github.com/DigitalMars/dmpp
and compiles with DMD 2.067. I'll see about updating it later today. Stay tuned!
I saw this addition in zig and thought that it was amazing, a lot of languages have this need to create definitions in their own language, C# with [DllImport](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.i...), [crystal with lib](https://crystal-lang.org/reference/syntax_and_semantics/c_bi...), I think since C is so ubiquitous and being able to just consume header files just removes so much boilerplate creation that is easy to get wrong.
zig also decided to just call an existing compiler (clang) and if you do zig cc --version then zig thinks it is clang, because the zig compiler just calls into clang main. D has gone with the route of calling gcc.
An added benefit is you get ffi-reflect -- https://github.com/corsix/ffi-reflect -- over the resulting types / functions...
to bystanders, odd autocorrect: it's marshalling. marshalling data. although fight and war are indeed also appropriate associations for binary data exchange (struct conversions) between languages.
Not mocking the spelling of the previous post; it's a strange term anyway and in this instance either spelling works.
Marshal = mariscalcus = "horse servant", compare mare and seneschal. Martial = from Mars, god of war.
these CS gals of olde really had a hand for naming :-)
Unfortunately there's a lot of disagreement between the bigger players as to how this should be handled:
> > 1) The authors appear to be deadlocked
> No need to soften it with an "appear", we are intractably deadlocked.
* specifying target cpu features (important!)
* features such as thread sanitizer support
* LTO that crosses the libc boundary
* targeting any glibc version
...all of which are supported out of the box by zig.
You can absolutely set up a cross-compiling C/C++ toolchain today as he said - but you're going to have to find the right packages in your distro, deal with the MacOS linking issue, get a copy of the android NDK, make sure you're installing the right package versions, muck around with libc, etc. That doesn't make for a delightful start to a new weekend project.
It's often underestimated just how substantial "setup pains" are IMO. I suspect most people who have a strong distaste for C/C++ mostly despise the tooling and installation hell.
The novel part of what Zig, (and maybe Rust/Go to a lesser extent) are doing with cross-compilation is not 'making cross compilation possible' - it's having 'the best experience' out of the box, with zero installation pain.
Zig has a ton of issues and is super early stages still.
But shipping out-of-the-box with a Zig/C/C++ cross-compiler, with libc, static libc(musl), compiler-rt, libunwind, libcxx, etc. for ~50 cross-compilation targets, and solving the MacOS cross-compilation linking issue (with their customer linker, zld) is IMO a huge deal because it lowers the barrier to entry to systems programming quite substantially.
For me the best example I currently have of such attitude is how an internal WinDev group managed to push for getting C++/CX replaced with C++/WinRT without it being production ready.
After 4 years, still no Visual Studio support for IDL files, one has to manually copy/merge generated files into the Visual Studio project, no two way editing with XAML files, we get continuously told that it only will be fixed when ISO C++ gets compile time reflection.
Apparently they feel that is a productive experience and never came close to try out either C++ Builder or QtDesigner.
I guess they were still using raw C to deal with COM to consider this kind of downgrade as something positive.
It's not bolted in the compiler so less nice than Zig but the jar produced by jextract is not ABI dependent so more portable. The ABI dependent code is generated by the JIT using the the Foreign Linker API (the equivalent of C# DllImport) at runtime.
https://docs.microsoft.com/en-us/previous-versions/visualstu...
Despite being told numerous compile-time options to use by different manuals and documents and people on IRC, nothing appeared to change the outcome. It might be due to some undefined behaviour in bzflag, of course, but it made zig cc a non-starter for C.
https://github.com/ziglang/zig/issues/4830#issuecomment-6054...
Not exactly. D is going the route of actually implementing a C compiler in the D compiler. Here's the C parser:
https://github.com/dlang/dmd/pull/12507/files#diff-2fc171ca9...
You could argue that writing a C compiler instead of using libclang is an indicator of brain-damage, but I lay out my reasons here:
https://digitalmars.com/d/archives/digitalmars/D/Add_ImportC...
And besides, C is such a simple language it's kinda fun.
echo hello.c
int printf(const char*,...);
int main() { printf("hello world\n"); }
dmd hello.c
./hello
hello world
But then there's the question "should D features be allowed in the C code?" This is a very subversive question. Is it really C if that is allowed? Already, the C code allows for Compile Time Function Execution, which comes for free by passing the C code through the D front end. You can see it in action in the test cases in this file:https://github.com/dlang/dmd/pull/12507/files#diff-b94776a02...
Scroll down a bit and you'll see it executes C functions at compile time. I decided to flirt with heresy and leave this in, because it sure makes adding tests convenient!
I'm less sure about adding unittests to the C compiler. Certainly, you can always add the unittests to the D code that is importing the C code. This will probably hold the charges of abomination to a minimum.
Oh my good gosh. D and C could be seamless together. This is great.
> I decided to flirt with heresy and leave this in
I'd love to see more C language extensions in ImportC.
> I wouldn't be surprised that the eventual cost of adapting ourselves to
> libclang will exceed the cost of doing our own C compiler.
This is a really insightful point. I had to learn this the hard way :)
We might follow your lead on this, as we have done with so many other great ideas implemented in D.
Ironically, Vexu started from the other side as you, with the preprocessor mostly done, but the backend left to-do: https://github.com/Vexu/arocc
One thing that might make libclang worth the cost, however, is its ability to compile C++ code as well. On Zig's end of things, all we have to do is provide libcxx, libcxxabi, libunwind, compiler-rt, and linking, and then libclang is really pulling a lot of weight by compiling C++ code into object files. Sadly this ability is just too useful in practice to ignore. For example, LLVM itself is C++ so if Zig wants to be able to bootstrap itself, it needs this capability.
Still, I think your maneuver here is the best long-term approach to tackle this problem, and I imagine as time goes on we'll start to migrate towards D's solution here. Maybe someday the Zig distribution that does not have LLVM extensions enabled will be the more popular one.
I'll be watching the evolution of this new feature in D with great interest!
1. Even though I have written a C++ compiler, it's a C++98 compiler and C++, like the Winchester House, accretes new features constantly, meaning keeping it up to date is impractical.
2. C++ semantics are different enough from D that it is impractical to mechanically map C++ to D. You can do bits and pieces, and dpp does that, but seamlessly doing the bulk of it isn't going to work. (C can be mapped nearly 100% to D.)
dpp uses libclang to try and map what it can to D.
Major props to Walter and the rest of the D developers for being so seemingly in tune with what people are looking for in a language.
That's a step in the right direction! I love D!
Here is one example from Apple in that regard, search for "Memory safe iBoot implementation".
https://manuals.info.apple.com/MANUALS/1000/MA1902/en_US/app...
Are you sure you're not thinking about C++? C is a small language.
I a way I'd rather use C than lua. I wish I could easily integrate python as a scripting language, but python is slow and its runtime is heavy. TinyCC, however, is very light.
Which is more important?
https://dlang.org/blog/2018/12/04/interview-liran-zvibel-of-...
For me currently D provides a good level of balance between those worlds, but the aforementioned GC isn't an issue for me at all so I have it easy.
"Eric Bier Demonstrates Cedar"
I think D should latch and promote more on the idea of having on-demand performance when it's necessary with GC convenience for the rest of the application. Soon having borrow checker capability ala rust and cyclone will only strengthen the narrative.
- https://en.wikipedia.org/wiki/Modula-3
- http://joeduffyblog.com/2015/11/03/blogging-about-midori/
- http://www.projectoberon.com/ (see https://www.astrobe.com)
- Eric Bier Demonstrates Cedar (https://www.youtube.com/watch?v=z_dt7NG38V4)
There are a lot of people in 2021 who - for whatever reason - feel like they need to program personal computers without using GC. As you point history shows that's a lot less true than people think, but nevertheless - that is the sentiment.
Dlang tries spends a lot of effort trying to attract these people, by making more and more of the stdlib nogc, having @nogc annotations, experimenting with deterministic manual memory management etc etc. But I feel like they'll never convert most of them.
I also agree with the sentiment, the language issues with some not so well worked out features should be fixed, instead of trying to please everyone.
This. For average programmers like myself, latest and fanciest language features are of little value. C++ and (unfortunately) D are in the same quest: to incorporate every programing language paradigm available, the extra complexity be damned. In the specific case of D, I'd rather have a good assortment of native D libraries than more bells and whistles added to the language.
The obvious examples are Ddoc, the builtin documentation generator, and unittest, the builtin unit tester.
The standard library won't be a variant you can use without the GC but rather one that is allocator agnostic wherever reasonable
Binding C++ libraries is quite risky in the sense that the libraries usually aren't meant to be consumed via that API. For that reason it's mainly used (publicly at least, there are things I know that I cannot say) for exposing D to C++ in a sane way (Mangling, vtables, templates, all work, exceptions even!).
ldc and gdc are a mix of D and C++ due to the respective backends.