Metaprogramming is less fun in D | Hacker News Reader

Metaprogramming is less fun in D

epi.github.io

124 pointsingve88 comments

It's a good article. The title is slightly bait-ey in that 'less fun' is a good thing in this context. C++ metaprogramming, at least for this example, requires a bunch of boilerplate. The D equivalent is built with standard language features and produces understandable error messages when mistakes are made.

It's a nice little anecdote about why you might want to use D in place of C++.

Does D have much of a use case left?

Java and C# are statically typed, garbage collected languages with a huge adoption and massive amounts of tooling.

D's attempt to become an alternative to C and C++ was eclipsed by Rust, which doesn't have the GC overhead and has a ton of other really compelling features.

There are a bunch of other new and exciting languages too: Go, Nim, ...

Why would anyone choose D today? (Serious question.)

D is GC optional, so that makes it pretty different than Java and C# and Go.

It also compiles to native exes as the normal procedure. You can do that with Java / C# in special cases though.

D is much easier to pick up than Rust. Rust seems to take the runtime overhead of Java/C# and put it into your brain overhead during typing time.

D has very fast compile times, Rust has very slow compile times.

I would like to use D if I could dump the GC, but it appears they don't make that easy, even though as you say it is possible. For me smart pointers make memory management seem not so daunting.

It's actually pretty easy. Just mark your code as @nogc, and it won't compile if there's anything that could potentially trigger a garbage collection. You obviously lose some functionality this way, but there's nothing difficult about avoiding the GC.

There are smart pointer libaries, and also implementing ref counting is much more straightforward than in C++ because the type system knows whether data is being shared or not.

I should add that the GC actually works perfectly fine for normal work, it justs not up to the competition (Partly because of having to work with traditional dynamic memory).

I wrote my thesis in D.

I considered C/++, Rust, Go, Nim and D.

D won because:

* Less manual memory management

* Awesome compile-time errors (this is absolutely key)

* Super easy tooling setup

* High-level abstractions in stdlib

* Still reasonably fast

* AOT compiled, which was a hard requirement for what I was researching

Interesting. I'm curious in particular where Nim fell short, any chance you could elaborate?

It was in a rough time for Nim, around the transition from the old name, when the documentation was still being rewritten, and metaprogramming was a magic few knew or used.

D and it's errors were just easier.

Not to say Nim isn't on my radar for new things, it just didn't quite fit at the time. (However, my first prototype was in Nim!)

Makes sense. Thank you for answering, is your thesis available for reading anywhere?

Anytime.

Unfortunately not, as I no longer have the right to publish it, and it hasn't appeared in the journal yet.

But basically, partial evaluation of the AST to create a fully type-inferenced statically typed F-expr Lisp. Makes JITing faster.

but this is really a poor argument for D your thesis code, isn't necessarily code that will be used by many people or shared with many developer (you didnt even share a link to the source code)

a thesis, just produce a proof of concept

D is unpopular , and I really believe that going after c++ was a bad idea .. if this was not clear at first, it should be clear by now, yet almost 100% of the article i see on D never stop from claiming how much better it is compare to c++

I guess the lesson learned from D's unpopularity is, when creating something dont focus on one competing tool

I recall how one time Linus Torvald, bashed subversion on how "Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System" -- Wikipedia

Was a horrible design choice from the start

By focusing on C++ they added a GC, as an advantage (because C++ didnt have a GC) but now that many people priase Rust for not having a GC and arguing they prefer C++ over D for it ... now they are trying to dis-integrate the GC ... very reactive, design decision ... and i doubt this will help much

> yet almost 100% of the article i see on D never stop from claiming how much better it is compare to c++

And I see the same from Rust and Go communities, hell I see it in Swift and Java communities a fair bit too.

That's not a real argument for or against a language. It just says the community will actually compare itself to other languages in similar fields. X vs Y articles are dime a dozen.

Some of Ds library inspired things in the new C standards, and D is influenced by other languages. I fail to see how that is a bad thing.

Common Lisp influenced Closure.

CoffeeScript influenced ES2005.

C has influenced Go.

Python, Nim.

That's just natural evolution. A language trying to become more flexible, is a good thing.

It's quicker to teach a language by pointing out how it differs from a similar language that the student may know.

As someone who contends against the horrors of C++ daily, I wish the best of success for both Rust and D.

C++ is an incredibly popular collection of footguns and its replacement is a noble design goal.

Conversely, SVN is one of the tools which I use to archaeologize within a C++ codebase. It might not be right for the Linux kernel or other FOSS projects, but in the corporate world it works fine.

> I wrote my thesis in D.

Really? So like your abstract goes in the main function, or how does that work? Do you use reference types for citations or just external symbols?

Good choice of language; though, who wants a "Ph. C++" or "Ph. Rust" affixed to their name, know what I mean?

You jest, but aren't as far afield as you might think.

I used a combination of Literate D [0] and pandoc to create the finished paper.

Ph. PLT still really doesn't stand out.

[0] https://github.com/zyedidia/Literate

> Really? So like your abstract goes in the main function, or how does that work? Do you use reference types for citations or just external symbols?

D has a documentation generator (Ddoc) as part of the language. It turned out to be useful as a standalone documentation generator. The dlang.org website is built using Ddoc, and I've even published a couple books in Ddoc. It feels a bit weird running a compiler over your marked up text, but that feeling passes after a while :-)

can D be used as drop in replacement for c++ in something like metatrader

i think, it might be worth it if you guy investigate areas like these, many trading platforms have interfaced to c/c++ if D and be added as a drop in replacement, with some more safety guarantees, you have a big niche market here that D can survive on for a long time

D interfaces very well with C/C++ code and libraries. It was designed to do that from the begining. Some D libs are actually bindings to C/C++ libraries. You don't need to replace existing C++ code/libraries, you can just integrate them in a D project.

> D's attempt to become an alternative to C and C++ was eclipsed by Rust, which doesn't have the GC overhead and has a ton of other really compelling features.

Rust isn't just a better C++, it imposes a new paradigm over resource management (lifetime oriented programming?). If you're coming from C++, you have to change your way of thinking. D doesn't try to make you follow any paradigm (for the better and for the worse) ; and its syntax will feel natural and terse for a C++ programmer.

My experience (10 people team) has shown all C++ programmers can learn D very quickly. Is this the case for Rust?

> Why would anyone choose D today? (Serious question.) Native speed, on-demand deterministic destruction, on-demand memory safety, metaprogramming, fast compile times, terse syntax.

I'm tempted to think that Rust doesn't really impose a new paradigm over resource management so much as it standardizes and enforces the common conventions of C++ resource management.

A major pain point probably is that people never really followed the "conventions" of C++ resource management as rigidly as Rust enforces them.

> A major pain point probably is that people never really followed the "conventions" of C++ resource management as rigidly as Rust enforces them.

That's exactly right. Rust does a great job moving idiomatic C++ best practices into the type system and actually forcing you to do it "right". And alot of the early pain people run into when switching to Rust is learning how to do this. I suspect very experienced C++ programmers have alot less challenge getting spun up on Rust. It's sorta front-loading the pain of discovering and learning C++'s best practices.

As a side effect, though, it means alot of Rust evangelism misses the mark as it mostly reveals their own C++ weaknesses (this is, on a different level, an indictment of C++, and its learning curve, but not really in the ways the evangelists often believe).

I'd amend this to say it's a subset of the C++ conventions.

While I kind of like D, it's hard to really make the case for using it. The thing is, it simply doesn't provide that much of a significant advantage over modern C++14/17 to make it worthwhile. At least Rust introduces an entirely innovative new concept with the borrow checker. D is just basically C++ with optional garbage collection and better syntax for meta programming, but without the vast Universe of existing C++ libraries to leverage.

D compile times alone are an advantage. I should probably point out that D can actually do some (More than anyone else) C++ interop.

D also has nice opt-in reflection as an advantage over C++ (though there are C++ tools to make up for it, to an extent).

The reflection is IMO the killer individual feature (well, hand-in-hand with the codegen to actually use it). The rest is just the benefit of 1000 small things that all add up to a greater whole.

Yeah, automatic static reflection for serialization of standard-layout structs in C++ would certainly be nice. You can achieve something like that with std::tuple, but it's certainly not comparable to first-class serialization.

PS, I used to enjoy your posts on SDN, before the site owner threw a fit. :-)

> D is just basically C++ with optional garbage collection and better syntax for meta programming, but without the vast Universe of existing C++ libraries to leverage.

Phobos, the D standard library, makes C++ standard library look very small. C++14 standard library still can't do "ls", "execvp", "socket" in a portable way (but we finally got std::thread and std::function!). Phobos can do all of this, and also haves support for md5, gzip, diff, sort, getopt, json, xml, ...

Thus the need for an external library is a lot less likely than it is in C or C++, and you almost never spend time rewrapping native stuff: it's already done by Phobos.

And in the end, if you need to use some specialized library, D can call C and C++ functions with zero overhead (I have D projects that use CGAL and SDL).

Productivity. D is a very pragmatic systems language. C++ interop is basically unparalleled, UFCS makes writing libraries less traumatic and CTFE is just plain useful. On top of this, the type system is both concurrency aware and has the best generics I've ever used (Subjective as it gets, but that's my experience). If you want safety guarantees just tag a scope as @safe. Also, static reflection works really well.

Compiles(with DMD, ldc and gdc are still fast but are not as fast) are fast and

D is not really comparable with Go (For example). D is much more complex (and I my opinion simpler to write through said complexity, but obviously opinions differ), whereas Go is supposed to be simple to learn and implement.

> Java and C# are statically typed, garbage collected languages with a huge adoption and massive amounts of tooling.

1. Java and C# are designed around JIT compilation, which results in a number of architectural choices that may or may not be to your liking.

2. The JVM and CLR are pretty heavyweight pieces of machinery.

3. Neither Java nor C# are particularly well-suited for working "close to the metal".

> D's attempt to become an alternative to C and C++ was eclipsed by Rust, which doesn't have the GC overhead and has a ton of other really compelling features.

This goes both ways: Rust doesn't have a GC, so if you want one, you're out of luck. For example, for most of what I'm doing – where GC works fine –, Rust offers me no benefit, but significant costs (note that this is a matter of application domains: the ones that Rust is designed for are simply not ones that matter much to me).

> There are a bunch of other new and exciting languages too: Go, Nim, ...

Go and Nim are indeed the languages that D is competing with more directly, but there's nothing wrong with having competition in this area, given that all of them are relatively new.

In my line of work (real-time), C++ typically reigns supreme but D is just as fast and more productive, so why not use it? It's not really about meta-programming, for me it wins on the basics.

And is we talk meta-programming, I think D goes further in the Stepanov style: https://www.youtube.com/watch?v=LIb3L4vKZ7U

Since the article uses an event system as an example, I'm wondering: how well do closures work in D? (Closures are very convenient when dealing with events).

Looks like they work pretty much the same way they do in most languages: https://dlang.org/spec/function.html#closures

They work fine. Everything is captured by reference, and is ultimately garbage collected. Unlike C++, you're not required to use a special "local function" syntax.

Interesting. Is this all guaranteed to work also in a multithreaded environment?

How efficient is the garbage collector, and are there pauses?

> Is this all guaranteed to work also in a multithreaded environment?

Yes.

> How efficient is the garbage collector, and are there pauses?

Not very efficient, and there might be pauses if you are not careful. Now if your closure doesn't escape (just a local function + context) it won't use the GC. So you can use delegates in @nogc code.

Full closures, function literals, delegates... Basically everything you'd expect.

I'm surprised someone hasn't chimed in with a Rust version of this by now.

Its a lot easier to do events based programming in Qt/C++.

Here is a Qt/C++ tasks[1] that greatly simplified async programming using Qt's event system.

[1] https://github.com/mhogomchungu/tasks

What does the []{...} mean in C++? Is it an array of anonymous functions?

It's a lambda expression ("closure"/"anonymous function"/…) that doesn't capture any outside state (see e.g. http://en.cppreference.com/w/cpp/language/lambda).

Many threads on D on HN lately. Is it a trend?

Wasn't the reference compiler open-sourced recently? That could've sparked some new interest.

I hope so :)

Also DConf is just around the corner (tickets close tomorrow?) so expect more.

i dont think going after c++ will help d's adoption it didn't work in the past

i dont know what have changed to make it work in the future

i dont remember Go or Rust ever focusing on comparing themselves to another language as much as d compare itself to c++

d should create its own path same way any other more popular language did

C++ will always have a leg up over D for me due to the GC stuff. I like not having GC in C++.

When I want a GC'd C-like I use Go.

D has to find a space between those two for me and a lot of other people.

I don't dislike D. I think D has a LOT of great work done in it. I just really want all my libraries to be able to be used without GC. And I'm not sure it's really feasible to have each foot in either world and not just get split down the middle.

Can someone argue against this worry? It's literally the only thing stopping me spending more time with D.

Also I've been coding in Ada (lord help me) for fun. Contracts are interesting.

I've been told (in other threads on HN) that you can disable GC in D. I don't know how that integrates with stdlib (or other library) bits that use GC (are they disallowed, or do they fail at runtime?).

You can't allocate with GC in @nogc functions, you will get compile time error. There is ongoing work in standard library (Phobos) to make it work without GC.

So if you @nogc main(), you're set? Thanks.

Affirmative.

You don't have to have GC in D, and manual memory can work side-by-side the GC with or without disabling it.

You can delete objects early, or call GC.disable

The safer approach is to mark that code @nogc. There is no guarantee that the GC won't run when calling GC.disable. You do get a guarantee with @nogc.

> There is no guarantee that the GC won't run when calling GC.disable.

If you have called GC.disable, the collector becomes `return;`. Sure, the GC function is called, but it doesn't do anything, it just immediately returns.

I'm not sure tbh. The documentation says "Collections may continue to occur in instances where the implementation deems necessary for correct program behavior, such as during an out of memory condition." My understanding from the forum discussions is that a collection could happen and you would not know about it.

An important factor in choosing a language is platform. I'm not going to choose Go or D to write an iOS app in, because it would be painful and buggy, if it's even possible in the first place. I'm restricted to Objective-C(++) if I want it to really be a high quality & native app.

Most languages with a C FFI (and ARM/Mach-O compiler support) can do pretty well on iOS. Rust bindings to Objective-C APIs are pretty painless for example.

D can interface with Objective-C just fine. The Apple-provided tooling will indeed be more convenient for the developer, but the result the end user sees will be a "high quality & native app" just the same.

I'm only lurking on the D forums, so this should probably be taken with a grain of salt. From what I understand the GC is required for various array, hash map and string operations, and things like closures and exceptions (maybe assertions?). Though there is work being done on fixes and alternatives (scope limitations, ref counted things).

This does seem to come up sometimes. I'm wondering if it makes sense to add a "end-vision for GC-less D" section to the D GC documentation, now that there are various undertakings to make it easier. Might also help with finding volunteers that care about @nogc to help out with various bits.

There is a pull request in now to make throwing exceptions GC-free. It hasn't been merged yet, but should be soon.

There's been a lot of work recently to enable GC-free D code. Yes, that means some of the standard library is off bounds, as are some of the built-in operators (new, array concatenation). However, you can have _static_ guarantees that there will be no GC allocations whatsoever. Writing an executable?

`void main() @nogc { /* ... */ }`

Done.

I've written a log of `@nogc` code lately.

Then what? How much of the standard library and the library ecosystem compiles with nogc?

Subsetting languages (and comunities) usually doesn't end well...

Depends on the kind of code you write. std.algorithm? No problem

Remedy Entertainment has a talk - actually two talks, one from 2013 and one from 2016 [0], about how they added D to their game engine, replacing a major subsystem, and then shipped a AAA title(Quantum Break) with it. The GC did not present a major issue although the talk notes that their solution was "unfinished" and got the result they wanted by wasting memory instead of what they initially aimed for(ARC, this is work-in-progress for D [1][2]). Most of the nuts-and-bolts work came from achieving smooth integration with the C++ codebase, and building for the release devices.

[0] http://dconf.org/2016/talks/watson.html

[1] https://wiki.dlang.org/Language_design_discussions#Automatic...

[2] http://www.digitalmars.com/d/archives/digitalmars/D/ARC_in_D...

The slides say the GC never collects, which is the same as having no GC.

After a while decreasing GC usage in D, I ended up removing it completely. I got reduced memory usage but no speed gain. What I've found is that before you remove the GC, you can turn the overhead from excessive to very affordable with optimizations. Especially today with std.experimental.allocator and -profile=gc

I'm a fan of Rust, but I think you must be reading a different HN if you don't think the Rust community focuses on comparing it to C and C++. The Go language creators have specifically stated that they created Go to solve problems they had with C and C++: http://web.stanford.edu/class/ee380/Abstracts/100428.html

I don't think "focuses on comparing it to C" is the right phrasing, but being a drop-in replacement for C is certainly a killer feature.

Rust has Mozilla, and the community does tend to appear in C/++ threads.

Go has Google.

Swift & ObjC has Apple.

D... Doesn't have a large mindshare/corporation backing it.

dont reduce Rust and Go relative success to just having google or mozilla

Rust and Go .. made better design choices that made them more popular

> Rust and Go .. made better design choices that made them more popular

[citation needed]

I found Rust by googling languages that compile to LLVM bytecode. It was version 0.6, but I was already excited by the fact that it was promising to be a fast, safe, concurrent language. Even back then it had a value proposition to early adopters.

I wrote a Fizzbuzz implementation. I did it to test language features. As language features and the standard library kept on changing, I kept updating my repository. It's had something like 70+ commits to it since 0.6 days.

https://bitbucket.org/iopq/fizzbuzz-in-rust/src/d4638b2ba3f0...

it's a little bit more complicated than necessary BECAUSE it's a test of new functionality (even now it only compiles on Nightly)

but it has a certain charm to it (fizzbuzz done in a few filter and map operations)

> dont reduce Rust and Go relative success to just having google or Mozilla

I don't think Go would have been popular without Google. It's easier to sell a solution to your manager when it is backed by a big org, and when people are paid fulltime to work on it. Same for Rust and Mozilla. The barrier for adoption of new languages is pretty high today in enterprise space.

so why did python and ruby became popular

and why isnt dart more popular

Python became popular because many people saw it as a cleaned-up Perl, and ESR wrote about it.

Ruby became popular because of Rails.

Dart isn't popular because it was designed to be a browser language but did little to court Web developers who like JS (contrary to the HN consensus, a lot of Web developers actually like JS) or other browser vendors.

so you agree with me, we python and ruby didnt need a huge corporate backing and dart even with some corporate backing didnt become popular

more examples that debunk that corporate backing is what made rust or go ... are scala and clojure, scala being way more popular than d and clojure ... and neither have corporate backing

in conclusion, you dont really need corporate backing if you have a nicely designed language that have its own path

I also said 'mindshare'.

Cs syntax can be awful. Similar symbols with different meanings. But, it was powerful, fast and simpler than some competitors.

Bash is frequently used today, for all sorts of things. Its even in production products, like git. However, even its creator thought it was a kludge.

There is no simple answer to language success, but a large group of people adopting and improving a language is probably going to be helpful.

d is not unpopular because of lack of corporate backing

d is unpopular because of poor project objectives and positioning and playing catch up with other languages rather than leading in any specific area

d needs a novel idea ... i guess they made bet on execution rather than innovation

Rust yes, Go just has Google.

i like your logic .. but i think Go have more than just Google, it also had Rob Pike and Ken Thompson as designers

two legendary programmers

And yet somehow they managed to ignore the majority of what has been learned in the last several decades of language design progress.

Given their experience, I strongly suspect they made the choices they did knowingly, rather than out of ignorance.

These two comments are not mutually incompatible. There is also a reason I've been heard to say that Not Invented Here syndrome was invented at Bell Labs.

Better design choices or better design?

D... Doesn't have a large mindshare/corporation backing it.

I had a strong interest in D from 2007-2009. The problem is that there were two competing, incompatible standard libraries for D1 (Phobos and Tango). Then came D2, which was a large improvement, both language-wise and library-wise, but basically split the ecosystem in three for some time.

I think the library and D1 vs. D2 situation stifled the growth of D. I gave up on D because the community was so fragmented.

That ended ~10 years ago. There's only D2 now. We have one user left using D1, and they're transitioning to D2.

Please note that, GC or not GC, you still can have deterministic destruction (e.g RAII).

I swear this title is generated by a Hacker News NLP bot. Heh :)

Reply on news.ycombinator.com