Next C compiler is a D compiler: Introducing DMD's ImportC | Hacker News Reader

Next C compiler is a D compiler: Introducing DMD's ImportC

briancallahan.net

189 pointsWalterBright83 comments

Note that the one feature holding this back from "true greatness" during development is that the tooling for providing LSP completions for D symbols in editors doesn't appear capable of detecting C code in your project yet

One of the major appeals of ImportC is that it does away with needing to write (and maintain) D bindings to C libraries -- by hand, or generated.

I was trying to experiment with io_uring in D and using liburing headers included in a C file + ImportC

My D app compiled and ran fine, and I could call liburing but in my editor I was in the dark as to what the methods I could call or their signatures were.

For hardcore vim programmers this is likely the norm, but for me I consider that a step backwards, and I went back to manually generating D bindings and using those.

Waiting for the day we'll see ImportC fully integrated into dub and pure-d/workspace-d!

> For hardcore vim programmers this is likely the norm

Just FYI, not anymore.

This article is about using the D compiler's ability to import vanilla C code to compile entire C projects.

This might actually make my life a lot easier compared to how I am currently doing certain things with some of my libraries that need to interoperate with C.

Going to give ImportC a spin there!

Can DMD compile C projects directly similar to the Zig compiler?

I haven't used Zig, but DMD can compile C projects directly using makefiles or by simply putting the .c files on the dmd command line.

Do you plan on giving Zig a try? (They seem to be doing a lot right.)

Taking the opportunity here to point people to "Adding Modules to C in 10 Lines of Code" from NWCPP earlier this year:

<https://www.youtube.com/watch?v=2ImfbGm0fls>

(The pitfalls you mention running into there wrt to different compiler instances is itself a decent case study in not relying on singletons <http://www.object-oriented-security.org/lets-argue/singleton...>—even if it ultimately wasn't so inconvenient that the work never got done, it would be even more convenient if it didn't need to have been done in the first place!)

> Do you plan on giving Zig a try? (They seem to be doing a lot right.)

All this Zig spam is getting very tiring.

In a conversation about languages with this kind of C interop, how can you omit Zig?

> In a conversation about languages with this kind of C interop

It's not. It's a discussion about "the C compiler built into the Digital Mars D compiler".

Please stop with all the Zig spam.

Instead of being petulant about it, you could ignore it and move on

Hard to ignore it when it's always in your face:

https://news.ycombinator.com/item?id=33331549

I don't mind languages getting hyped but seeing posts every week on the front page of HN about an unstable esoteric language with 0 adoption - yes, it's getting tiring.

Maybe you should appeal to the community and tell people to stop upvoting it, and explain why it’s wrong for them to think Zig is interesting.

How come it's always accounts made a few days ago which seem too have this problem with other people's interests

Literally from the thread I shared:

- https://news.ycombinator.com/item?id=33337381 (posted from a 12-year-old account)

- https://news.ycombinator.com/item?id=33333904 (posted from a 2-year-old account)

10241024 is without a doubt our friendly neighbor troll https://gist.github.com/andrewrk/9aed922a6210e57243af8be8d10...

simplotek is most probably a legitimate user annoyed by Zig news. I don't agree with their point, for what it's worth, but at least it's an opinion shared earnestly, unlike the troll.

> Instead of being petulant about it, you could ignore it and move o

Not wanting to see gratuitous self-promoting spam degrading the signal/noise ratio of an otherwise interesting discussion is hardly petulance. Replace any Zig reference shoe-horned into this discussion with, say, Java, and see if you have any point.

Reminds me of this post just from few days ago[1]. I wish HN had a rule or two dedicated to PL tribalism.

https://news.ycombinator.com/item?id=33382003

It's not "Zig spam" to ask, in direct response to a compiler guy who has just stated that they have not tried it yet, whether they plan to. For further context, the video I linked to is Walter showing up to a C++ users group to present a talk about the D compiler infrastructure—again, at a C++ event, for a bunch of C++ programmers. Considering that, if what I've done here by asking the natural follow up question is a faux pas, then I hesitate to conceive of what you must think of those actions by the person I'm asking it of.

Even all that aside, comparing and contrasting contemporaries in a similar space is natural. That's just, like, scholarship, you know? Imagine someone who's defending their thesis responding to a question about alternatives to their research by getting into a big huff about it. That'd be pretty silly, right?

And the comments about "PL tribalism" are a little odd, then, since the most tribalistic behavior we have evidence for in this thread is someone reacting exactly like this. As a fan of Walter's contributions, including D, in the past I've gone so far as to put a non-trivial amount of effort into polishing the articles on D, Walter, and Digital Mars on Wikipedia. Meanwhile, I've never even written a Zig program, dude; if there's a tribe I belong to, it's not Zig.

> It's not "Zig spam" to ask (...)

It is Zig spam, not because of the discussion or the question or the topic of the time of day. It's Zig spam because it's yet another Zig post among the barrage of Zig posts desperately shoe-horning offtopic gratuitous references to Zig. It's tiring and adds nothing to any discussion.

If you feel that strongly about the need to promote Zig, submit discussions. Please don't spam each and any top submission with these gratuitous references.

It's not off-topic though. D has the facility to compile C projects, as has Zig. OP asked a genuine question about whether Walter Bright was able to compare experiences.

Please don't blow past the comments already addressing your complaint.

What I responded to was a comment from D's own creator who himself directly mentioned Zig. I didn't shoe-horn shit. I've been waiting a long time for a hit on Corncob TV.

If you're upset about Zig mentioned here, your quarrel is not with me, the person who spent over half the words in their comment bringing it back to the original topic by linking to a useful talk; your quarrel is with, if anyone, the original questioner who got Walter to comment on Zig. (And even then, your ire would be tenuous at best, for the reasons already mentioned.) I also cannot "feel that strongly about the need to promote Zig"—because I am not promoting Zig. As I have already said, D is closer to my tribe than Zig is.

Have you heard of Rust?

If they've heard of zig, then they've almost certainly heard of Rust. I generally prefer Rust, but C interop is something that zig does significantly better.

Agree about the excessive spam. To include attacks and bashing of other programming languages. It's very obvious what's going on and looks like it's going over the top, as if a concerted effort and mode of operation.

Zig can be used as a standalone CC replacement and not just for FFI purposes.

A lot of people might not know that before doing the D language, Walter was already a world-wide acknowledged expert on the compilation of C and C++.

I don’t use D seriously but I’d still bet my ass that something smart happens when you feed C code to the D compiler toolchain.

> A lot of people might not know that

Personally, that would surprise me, because C++ compilers is the context where I first read the name. I'm not saying this is incorrect, but I would find it amusing somehow.

It's fair to say that Walter is best known for his work with D rather than his work with C++, though.

Depends on how old you are. ;-)

Indeed. He was a legend a long time before there was a D programming language.

Yes, Zortech C++: about 1991 IIRC. If this was not the first commercial compiler available on Windows, it was one of them.

The article's title is "Your next C compiler is a D compiler", which makes sense, though it's a bit clickbaity. The HN submission title is "Next C compiler is a D compiler", which barely makes grammatical sense, and to the extent that it does make sense, it sounds like there's a C compiler called "Next C compiler" which is also a D compiler.

My mistake. Thanks for pointing it out.

I don't think it's your mistake. It seems HN automatically removes certain leading/trailing words from titles so as to reduce clickbait but it doesn't seem to be very smart about it.

It gets annoying, because in cases like this it doesn't make sense, but in other cases it actually changes the meaning of the title. I honestly don't understand it given the prohibition against submitters changing the title.

More importantly, this change is silent.

I'd be much less annoyed by it if dang finally made it a pre-submit check (maybe with a checkbox "Overrule") like the "too long" check. But no, it seems to be important that the submitter has to check manually whether the title he just typed in is still there.

That's impressive, i thought the preprocessor wasn't planned, looks like it takes care of it too, i guess it's time to revisit D, C but better

Preprocessing is now fully supported!

My experience with D is always a joyful one. If it isn't because it is a relatively straightforward language to learn then it is learning something new now and then that just makes life to much more easier.

Infact, D was the first language I learnt templating in and I really like it (or rather, the way D goes about it - made it much more approachable for me).

I wasn't going to add templates to D until I figured out a way to make them simple.

Hey walter, have you thought about adding first class types to the language? that would enable us to use CTFE for type based algorithms instead of templates like std.meta. If you did, what are the reasons against them?

You definately made it simple! I recall the very first time I tried learning then in D (was trying to understand the `to!` from `std.conv` and it seemed simple enough).

The syntax for inferring a type for function calk arguments is nifty too.

Lastly, today I finally understand what mixins are , I just read some stdlib file somewhere and thought, wow that's rather useful. I definately think I will be making use of mixins sometime soon, been working on a small compiler in D itself and there are times were duplication (or perhaps inclusion is a better word) of code would be nice in such a manner.

Does ImportC support extensions like attribute cleanup yet? It is basically the same thing as D's scope(exit).

More broadly, are there plans to support C extensions that are available in GCC and clang?

ImportC already supports some extensions, what we figured were the most used ones. We plan to support others on an as-needed basis.

> Error: C designator-list not supported yet

I guess only a subset of C11 is supported? Is there all of C89?

Designator-list support has since been added. The multi-layer designators are still to be done, though (they're rarely used, so are a lower priority).

I think something along those lines (rarely used...) was also given by the C++ Committee as justification why so much of the C99 designated init feature set was left out of C++20, which is a bit of a shame. All C99 designated-init 'sub-features' complement each other nicely once you start using them. It's a great way for building 'data-centric' APIs.

For instance check out this call to create a 3D-API pipeline-state-object in C99:

https://github.com/floooh/sokol-samples/blob/ad9047e228a8441...

The C++ reason has to do with the construction order of fields, which C naturally doesn't have.

Sokol's API's are the most ergonomic C API's I've ever seen, for what it's worth.

Interesting.. i personally use that a lot in C, this lib for example:

https://github.com/floooh/sokol#sokol_gfxh

I couldn't live without it

https://dlang.org/spec/struct.html#static_struct_init

D support it, what's blocking it?

Hello Walter, do you have any advice on integrating Rust and D? Especially regarding GC

Rust is pretty clearly the ascendant language in this (big) niche.

I forecast that as it’s pragmatic, industrial uses (inevitably) grow sharply, that the FFI story will just have to improve. Rust is going to be the next “low-level” language.

The “Pure Rust” talking point will have to recede a bit but it will be a net win for Rust’s truly broad adoption, which is going to happen whether cranky old-timers like me like it or not :)

Bitching about downvotes is generally frowned upon around here, but I’ll burn a little of my old-timer credit.

How is acknowledging that Rust is clearly and obviously the next big thing an insult to anyone.

It’s not my favorite language but it’s a good one and I know it’s going to displace C/C++ in my lifetime. What more does the downvote brigade want?

Using Rust code in D should be relatively easy: expose C API, compile it into a shared library and use it on the D side as any other library. cbindgen (https://github.com/eqrion/cbindgen) can help with generation of header files. Static linking is possible as well.

Using D code in Rust probably will be much more painful and I am not sure if it can be done 100% safely...

> Using D code in Rust probably will be much more painful and I am not sure if it can be done 100% safely...

Can you elaborate? Saying it will be "much more painful" without anything further is not especially helpful. I've added D functions to C programs more or less trivially, including compiling to a shared library, with no change to the code.

Because I am not knowledgeable about D runtime I can not say that it will not cause any conflicts with Rust assumptions (thus the use of "probably"). For example, IIRC there are serious issues with integrating Go code into a Rust project. If D code is compiled in the betterC mode, then there should not be any problems.

You can link in and interop fine with C/C++ as long as you link in the D runtime, kind of how you can mix C and C++ by linking in libcxx.

D's betterC mode ensures that the binary can run purely on libc without D's runtime library. It's kind of like being able to link and execute C++ code without needing libcxx.

There are serious considerations for including Go code in a rust project, but it's not like there are any showstoppers - one just can't use the default runtime and must rely on Cgo, and the FFI entry points are not performant. Other than that, it's relatively painless. Far less painful than calling into the JVM IMO.

I don't think Rust will ever consider FFI'in with another language "100% safe."

You can use C libraries in Rust, even compiled .so files where you dont have the corresponding C headers.

> Using D code in Rust probably will be much more painful and I am not sure if it can be done 100% safely

You can just as easily declare D code to be extern(C) and have rust treat it as a C library.

There is a question about C23 if you can see [dead] people:

1. We can support it, I suspect no headers will go near it for a while but we can.

2. C23 support actually brings some C semantics closer to D ones anyway...

C has head const and it's pretty common in C code bases. From what I remember, D had transitive const and not head const, so how does ImportC deal with this now?

Is head const now supported in DMD with ImportC?

https://dlang.org/articles/const-faq.html

What I don't like about LDC, is you have to install the entire Visual Studio if you want a static build [1]. Contrast this with Go, Nim, Rust, Zig and others, that don't put this burden developers. Is DMD any different in this regard?

1. https://github.com/ldc-developers/ldc/issues/4047

DMD doesn't require anything else. Except for Win64, where it needs the VS C runtime library and linker.

> VS C runtime library and linker

is that included with the DMD installer? If not thats the same situation as LDC.

For me, the next C compiler is a RISC-V assembler...

Walter, since you're here: what's your position regarding bootstrapping and Ken Thompson's "Reflections on Trusting Trust"?

I haven't read it.

The D compiler will steal your passwords, but only for old VAX BSD versions of passwd, so Walter can log in and fix your K&R C bugs while you're sleeping.

your daily edition of WalterBrightFacts™

Hmmm...once at Caltech someone got into my account on the PDP-10 and all they did, as far as I could tell, was fix some bugs in a program of mine I was having trouble with.

I never did find out who did it. I wonder if it was Walter?

I was an overly obnoxious frosh at the time and I'm certain I was quite annoying to the older students who frequented the terminal room such as Walter (I think he would have been a junior at the time).

And I'm certain Walter was aware of me, because he had once either targeted me in a prank or used me in a prank against someone else (or perhaps both--I've never been clear on who was the target). The someone else was one of the system admins, Bill. Bill really liked playing Walter's game "Empire".

So one day I was sitting at a terminal hacking away at something and I got a message via the system for sending a text message to another terminal from Bill telling me to not do what I just did again or I'd be in trouble and have my account suspended.

I had no idea what he was talking about. A while later I got another message. Bill seemed really annoyed with me. I then got summoned to his office. He was really mad. He said I keep sending him rude messages, like telling him to stop playing Empire and get his fat ass back to work.

I denied sending these messages and he was not buying it.

And then another insult from me came through while I was there in his office.

Bill then started to maybe believe me, and went to Walter's directory to look at the Empire source. Sure enough, Walter had recently added code that checked for Bill playing while I was also logged on, and would fake rude and insulting messages from me from my terminal to Bill.

And that is why I will never use a compiler from Walter if there is any way the compiler might be able to tell I'm the one using it. :-)

David Wheeler's Ph.D dissertation on the subject hasn't been refuted as far as I know, so the diverse double compilation method should be sufficient to mitigate the compiler subversion threat. I worked on a project to do this in OpenEmbedded to build firmware for the Power9 Talos workstation firmware and the BMC. Somewhat painful but successful.

Can you tell the D compiler to free a pointer returned from a C method when it goes out of scope? Like uniq_ptr

D has scopeguards similar to defer.

D can do RAII.

via structs, right? As for classes you should not rely on it I recall.

You can stack allocate classes, and then they behave like structs.

https://dlang.org/blog/2017/07/07/go-your-own-way-part-one-t...

This is just like Zig's C compiler

Zig doesn't have a C compiler yet, it bundles clang. Someone in the community was working towards a C compiler in Zig though.

ImportC is basically a C frontend for the D compiler, semantics are close enough but there are some changes to be made still.

Zig does have "translate-c", which attempts to convert C code to Zig code. Similarly to D, it's used for a feature to automatically import C headers into a Zig program. But even that's based on libclang's parser and AST.

ImportC has bitfields and #pragma pack but not translate-c.

The important part in Zig is that you don't need another compiler toolchain to build a mixed C/C++/ObjC project, how that's implemented under the hood (e.g. embedded clang vs self-made compiler) is an implementation detail.

I'd argue that C++ and ObjC support is just as important as C support for real world projects (because a lot of important libraries are written in C++ unfortunately, and ObjC is needed for calling into macOS system APIs - although this could also be done more awkwardly by going through the ObjC C runtime).

The one big downside of embedding Clang and the whole of LLVM is the compiler's executable size (somewhere around 170 MB).

Reply on news.ycombinator.com