Compile-Time Sort in D
dlang.org
dlang.org
https://github.com/dlang/dmd/blob/master/src/dmd/backend/ope...
Notice how a static immutable array is created by a lambda at compile time. The lambda never appears in the executable. Before D, I'd use a separate C executable to generate the tables, which were then #include'd.
The lambdas can also include sanity checks:
https://github.com/dlang/dmd/blob/master/src/dmd/backend/ope...
More impressive: manipulating strings at compile-time.
Even more impressive: using string manipulation at compile-time to build valid D code, then use string mixins to inject it into the current program, passing it on to the compiler to be compiled as part of the program being built. This is done by std.regex to compile regular expressions to native code during compilation.
Even further impressive: using a string import to read a DSL from disk, transpiling it into D code, and passing it on to the compiler. This is used by Vibe.d to compile HTML templates to native code.
Granted, all this is fairly old news, and a few other languages have since caught up.
haskell templates are similar.
While C++ is not by any means a great meta-language, it's improved considerably since that time. D still has some interesting metaprogramming features, but on the whole I don't think it's enough to justify the rest of the baggage that comes with the language. Buggy implementations, releases that constantly break valid source code and make it a pain to work with dependencies along with a poor/unscalable garbage collector.
The ability to write a string literal that contains D code, perform raw string manipulations on it and then compile the result certainly sounds neat on paper and if you're working on a solo project it can be a fun exercise to blog about, but beyond the novelty you'd hardly find a mature or reliable codebase written by a team of professionals using hacks like these.
I do however work on a large-ish codebase with many professionals developers which makes extensive use of metaprogramming. We also don't have particularly big issues with moving from version to version so how old is your hearsay?
C++ doesn't even have proper modules yet, it's still not as good as D in more regards than just metaprogramming.
So visual studio has an implementation not many people use, what of the other compilers? I'm still yet to even compile a project that actually uses them, and they must have spent a significant proportion of my entire life trying to get them in the standard.
Naturally not every compiler is yet fully C++20 compliant but they will get there, VC++ is there, next release of GCC is looking quite good, clang well lots of issues anyway now that Apple and Google resources went away and not everyone has the same love for upstream.
As for the hack, go dive into D forums from like five years ago thereabouts, since import statements in D fully parse the source of the respective modules, the hack to speed compilation time was to move import statements into the function bodies so that they are only processed if that code is actually parsed.
If I am not mistaken, Andrei was one of the persons suggesting this.
That is a damp squib, of all the things I view as hacky in D that is not one of them.
Local imports are favoured for stylistic reasons anyway, they don't actually make all that much difference for code that isn't dead anyway. What's wrong with importing things where they are used?
IIRC Andrei proposed the local imports, but it was only a matter of removing an error message because they were banned at the time.
As for what is wrong with local imports, it makes impossible to track down module dependencies unless one goes grepping for imports all over the place.
Nobody is making anyone use them. It's a personal preference.
As for "impossible", `import` is a keyword, so it's trivial to find them all using, yes, grep. D was designed to be more greppable than C/C++. For example, it's really hard to find all the cast expressions in C, but since `cast` is a keyword, it's trivial in D.
(Cast expressions are all potential bugs, so being able to find them all in a code review is a good thing.)
> five years ago
Is an eternity in the software business.
In those 5 years, D's competition reduced even further the need to actually bother with D, slowly taking away all features that made D special.
If it were a hack, people would only do it for it's hacky-ness rather than as a style preference. The former is simply not true in industrial usage of D. In less words: It's bullshit.
What industrial usage?
A couple of companies that still haven't yet left the community, because each year something new is going to be the Thing to bring everyone into D?
Lets see how D manages to survive when the core team is no longer calling the shots.
What languages don't have something new added on a regular basis? Yes, we improve D consistently and regularly.
The location of an import has nothing to do with compilation speed. The only time it would ever make a thing compile faster is if 1) it is local inside a template function, 2) that template is never actually used, AND 3) the module is never imported anywhere else in the program.
That does happen - I use this pattern for optional dependencies which aren't even looked at if not actually used - but it isn't generally the case. You have to actively think about it to fulfill all three requirements.
Having a high performance GC requires inserting write gates into the executable code. This is only worthwhile if GC is the only allocation strategy the language uses. As the GC is optional in D, and is not actually used that much it is not worth the general slowdown of the write gates.
> you'd hardly find a mature or reliable codebase written by a team of professionals using hacks like these.
I wouldn't be so sure, as mature and reliable codebases use the hackish unhygienic preprocessor macros every day :-)
Isn't it funny how the most mature software (and often by extension, "reliable" e.g. have been in use the longest with the most bugs beaten out by years of hacks and patches) seem so far away from "best practices"?
I've always wondered about the root of this. Are programmers trudging along using "brute force" more effective, or is it just pure survivor bias on projects/products that people didn't give up on?
One kind is things dealing directly with the underlying thing you are trying to accomplish with a program. Say it is a program doing some sort of physics or engineering calculation that at some point needs to add a bunch of floating point numbers. Best practice might include using something like Kahan summation [1] to reduce the error.
I'd expect to see that kind of best practice in both mature programs and new programs because it is something that directly relates to what the program actually is trying to do.
The other kind of "best practices" are those concerning the environment and practice of programming itself.
Many of those seem designed to deal with the case where there is a lot of turnover in who is working on the software. People join a project, work on it for a year or two, and then move off to something else.
You adopt these best practices so that new people will be productive faster so you can get useful work out of them before they leave. So these best practices tend to change as fad and fashion come and go.
The mature software that has been in use for a very long time I think is more likely to have a much higher percentage of its work done by people who have been working on it long term, and have a lower turnover, and when someone new does come on they stay longer.
They don't need to have their best practices match the fad and fashion best practices used by the shorter term projects.
Libraries that don't use it at all are available. At work we have a large library, being a hedge fund, that must be consumed from Excel. This means it cannot use the GC because [reasons], it's not a particularly big issue.
Garbage collection is not a bug, by the way. It's one of biggest productivity wins D has over C++. Performance is often as near as makes no difference identical, only you now eliminate all memory safety issues without thinking about lifetimes.
D also has very nice composable allocators in the standard library
> While C++ is not by any means a great meta-language, it's improved considerably since that time.
C++ has also painted itself into a corner multiple times too, which despite being technically an improvement over the status quo are lacking severely in their utility. C++ screwed up "constexpr if" big time by always introducing a scope (which costs you a pair of {}'s in the rare occasion you need one) which means you can't conditionally insert declarations (i.e. variables, structs/classes, functions).
> but beyond the novelty you'd hardly find a mature or reliable codebase written by a team of professionals using hacks like [string manipulation and mixins].
They are a wonderful hack when you need them and nothing else will do what you want. This is not unlike resorting to macros in C++, except that its hygienic, unlike macros.
I'm not claiming the project is mature and I'm only one person, but reliable definitely out there. The most heinous set of string mixins i've ever written[1] has definitely got to be the code for generating wrappers to call the OpenCL object property querying functions (clGetDeviceInfo & friends). You need to pass a size and a void pointer to the address of the return object that you have to call once, twice or more (depending on the type of the queried property) to figure out how much memory you need to allocate to call it again.
The important thing is that the interface[2] you use to drive this code generation is very clean and return on investment for getting the generic case correct is large.
[1]: https://github.com/libmir/dcompute/blob/master/source/dcompu...
[2]: https://github.com/libmir/dcompute/blob/master/source/dcompu...
That really undersells the `static if considered paper` which has got to be the most offensive standards paper I've read.
> so there must be arguments against 'scopelesd constexpr if'
I've yet to find any. What I don't understand is: you hardly ever need to introduce a scope with Ds `static if` and in the very rare cases you do it takes all of two keystrokes, but with C++ you are completely unable to get rid of the scope.
Indeed. Never read something stinking so much of sour grapes from NIH-suffering old farts.
I had the misfortune to have to use D professionally for a little over a year, and I can confirm all of the above are sadly true. The garbage collector was a headache from day one, and was never not a problem.
TH/procmacros just give you lexical tokens, you then parse them (typically adding a monstrous dependency on haskell-src-exts/syn) and emit AST.
The way D is designed, you don't need such a special facility because CTFE is everywhere. You can run normal functions at compile time. You can have compile-time "static if" and "static foreach" statements, and use various introspection calls (confusingly called "traits") in those.
Zig has similar "comptime" functionality. C++ now has "if constexpr" but not any of the introspection features.
In fact, since C++20, you don't need any template metaprogramming for compile-time sorting - std::sort is constexpr! [0]
It seems like it hasn’t been actively worked on in quite some time, unfortunately, based on the commit logs.
import algorithm
const a = [ 3, 1, 2, 4, 0 ]
static: echo(sorted(a))The article itself is a D implementation of something done in C++ to show how D does it, and how much simpler it is. The Nim example is right in line with this.
C++ (gcc and clang) and D’s compile-time evaluation is a bit more limited in scope, and still uses a tree-walk interpreter, although D might also migrate to a bytecode VM someday (https://dlang.org/blog/2017/04/10/the-new-ctfe-engine/)
Many (most?) Lisps have had full arbitrary CTE since ... I'm not sure, probably before C existed?
Writing a lisp compiler in lisp is a very different task from writing an interpreter. Especially if one expects to bootstrap.
One example of how I use it can be found here: https://git.sr.ht/~bjoli/goof-loop/#speed in the first example where I contrast ,expand (macro expansion) and ,opt (the source->source optimizer.