Hacker News Reader

> Arbitrary compile time computation should be separated from the body of the program.

Here's some code that initializes an array at compile time with a non-trivial computation:

https://github.com/dlang/dmd/blob/master/src/dmd/backend/var...

This used to be initialized by a separate program that generated some .d files. It was nice to get rid of that. Even better, if you check the generated object file, there's no trace of the code used to generate the array - just the data for the array.

> > Precomputing values without I/O is conceptually just an advanced kind of compiler optimization, and that’s why Rust is basically seeking to make const fns only pure functions.

I agree that generating an array at compile time is perfectly fine. I was saying that I/O should be restricted in compile-time computation, which the person I was responding to made me believe was entirely unrestricted in compile-time computation in D.

>Arbitrary compile time computation should be separated from the body of the program. If you need to download files or read the file tree at compile time, that should happen in a build script. Rust provides build.rs for that.

This is an area we'll have to agree to strongly disagree. For me having support directly in the language is a huge improvement in ergonomics, compared to having to run a separate script. Compare e.g. generics in Go1 (which can only be done with separate build scripts at compile time) vs in any other language that has built-in generics. Also consider F# type providers, which guarantee at compile time the correctness of code querying a database (generate the schema/structs by querying the database at compile time). Certainly the same thing could be achieved by running a separate script as part of the build process, but I've never seen a language actually implement that, probably because it'd be much more painful.

>The fast compile times are a double edged sword, since those binaries are noticeably slower than Rust binaries, from what I’ve seen.

Note that D also has an LLVM-based compiler backend that is slower to compile but produces much faster code, which can be used for release builds.

A major difference is that the Go compiler will not run build scripts when you build the code. You have to manually run codegen scripts, which is error prone since people are not good at remembering to do manual steps. Rust will automatically run the build script before building your code. It just allows you to logically separate build time I/O from runtime I/O, which I find is nice for reproducibility of builds and maintainability of code. The build script is still written in Rust, and you can import Rust code from wherever into that build script. The plan is to support arbitrary non-I/O computation in constant functions anywhere in Rust, since those are just pure functions.

As far as your comment about databases, you should look at Diesel. It handles this very nicely in Rust, so... that’s not a problem at all.

> Note that D also has an LLVM-based compiler backend that is slower to compile but produces much faster code, which can be used for release builds.

Is it compatible with 100% of the code that the current DMD release is compatible with? That’s all I ask for in alternate compiler implementations... but it seems like a lot, sometimes.

>A major difference is that the Go compiler will not run build scripts when you build the code.

Go actually has a built-in "Go generate" command that will look for magic "//go:generate <command>" comments in the code and automatically run code generation based on them, although I'm not sure how ergonomic it is.

>It just allows you to logically separate build time I/O from runtime I/O, which I find is nice for reproducibility of builds and maintainability of code.

This is a reasonable argument. For what it's worth, compile-time IO in D is more limited than in e.g. Haskell, as it seems the only compile-time IO D supports is reading files (or at least I can't find any documentation suggesting otherwise). Most of the power of its compile-time code execution comes from non-IO code (e.g. generating an optimal lookup table for a dictionary of values known at compile time, hard-coded in the code, or automatically generating fast JSON serialisation/deserialisation code for a struct without need for macros). While Rust also plans to support this, full support could be years away, while D supports it now.

>As far as your comment about databases, you should look at Diesel. It handles this very nicely in Rust, so... that’s not a problem at all.

Definitely shows it's possible to get the benefit of type providers without language support for them, hats off to the Diesel team!

>Is it compatible with 100% of the code that the current DMD release is compatible with? That’s all I ask for in alternate compiler implementations... but it seems like a lot, sometimes.

It's eventually compatible, sometimes lagging a few versions behind. Using LDC for release builds is somewhat comparable to only using stable Rust for release builds, except D has a slower pace of new feature additions than Rust since it already has so many features.

D deliberately restricts what compile time code can do that involves the operating system, restricting it to simply reading files from the current directory and down.

This is to prevent the use of the D compiler as a malware target by sending it specially crafted source code to compile.

couldn't you just link to a c file in the same dir then use that to read up the file tree. I would be interested to know how D would prevent this.

D files don't #include .c files.

Unfortunately, these restrictions make it hard to implement F# style Type Providers. Though honestly the filesystem one is not nearly as annoying as the inability to reinterpret-cast.

That's fine with me. I don't want to make the cross site scripting mistake.

Which happen to be currently broken on modern .NET runtimes because it wasn't done on a clean way apparently.

> Go actually has a built-in "Go generate" command that will look for magic "//go:generate <command>" comments in the code and automatically run code generation based on them, although I'm not sure how ergonomic it is.

But it's still not run automatically at build time, you have to go out of your way to run `go generate`, which isn't gonna happen if you, as is customary, fetch and build third-party code with `go get`.

> which isn't gonna happen if you, as is customary, fetch and build third-party code with `go get`.

Except that the convention is to check in the files generated by go generate, so in practice, it's not actually a problem.

> It just allows you to logically separate build time I/O from runtime I/O, which I find is nice for reproducibility of builds and maintainability of code.

You can get the same effect if you really need it with a pre-build command with dub (the package manager/build tool, and with other build tools such as reggae), but compilation is supposed to not change the state of the system other than produce build artefacts, you can read files at compile time with `import("filename")`, but you can't write to files, or open sockets etc.

Things like mixing in a data file is incredibly useful and not something I would want to use a rebuild step for at all.

> Is it compatible with 100% of the code that the current DMD release is compatible with? That’s all I ask for in alternate compiler implementations... but it seems like a lot, sometimes.

Not strictly.

It is usually pickier about what is deems valid (which is usually corner cases that DMD really ought to reject). They share the same frontend so there is very little difference.

It has some of its own bugs, though nothing too major.

It generally has way less bugs both due to its release cycle being based off of DMD stable and thus it lags one cycle behind (~six weeks), so new features aren't available immediately, and having all the backend and optimisation handled by LLVM, though we occasionally hit LLVM bugs that clang for some reason does not seem to trigger.

(I develop both DMD and LDC (the LLVM based D compiler))

> Things like mixing in a data file is incredibly useful and not something I would want to use a rebuild step for at all.

It’s not directly part of the const function, but rust has compiler built-ins to import files as values as compile time, which makes it easy to declare global static variables that contain that data.

This is different from being able to walk a directory and read random files.

https://doc.rust-lang.org/std/macro.include_str.html

https://doc.rust-lang.org/std/macro.include_bytes.html

> They share the same frontend so there is very little difference.

That’s excellent.

> This is different from being able to walk a directory and read random files.

D doesn't allow you to do that either. You have to specify what files you are importing via a compiler switch: https://dlang.org/spec/expression.html#import_expressions

Yes. That's another barrier I put in to prevent a carefully crafted source code from being a malware vector.

> Is it compatible with 100% of the code that the current DMD release is compatible with?

For practical purposes: yes. We use DMD for development and LDC for release builds, and it works really well in tandem.

Becoming a C++ expert is impossible in a lifetime, becoming a D expert is a matter of months.

It is, from the very core, a way simpler language than C++. I'd argue it's simpler than Rust, even. IMO acknowledging a few function attributes is way simpler than learning all the flairs of ownership and lifetimes.

As for metaprogramming, I love D's introspection capabilities, while I kind of want Rust's ability to create unquoted DSL.

The only "non-uniform" part of D that I can think of is the oddness of `shared`, and a few outdated stdlib modules. Everything else (betterC, pure, live, etc.) is just a subset of a powerful, coherent multi-paradigm language.

I like D, but if you spend a some time in D forums, you would get that coherency is something that even D could improve upon, specially attribute usage, inline assembly compatibility among its implementations and some constructor/destructor corner cases.

> To become an expert in D or C++, you really need to understand what feels like a dozen different languages... and that’s not what I’m looking for in a single language.

Well, the alternative is to learn an actual dozen of language - one for the high-performance simd part of your app, one for GUI layouts, one for writing parsers, one with strong typing semantics for your domain objects (but how are you going to keep them aboard in the interfaces with your other languages !), one for network communication...

I much prefer (as a C++ dev, but the same applies to D), the ability of the language to make eDSL adapted to each tasks.

Rust’s macro system is incredibly powerful — much more powerful than C++’s, which is just text substitution. It’s easy to make DSLs in Rust. That’s not what I’m talking about.

> Well, the alternative is to learn an actual dozen of language

That’s a false dichotomy.

Rust macro system is much more powerful than D mixins.

I personally find Rust macro system infinitely easier to use than D mixins as well. In D, I need to learn a new pseudo-language to work with mixins, but Rust macros are just normal Rust code that gets executed at compile-time on other Rust code, and this code can do anything that any running Rust program can do.

The structure of the 2 fundamental Rust libraries around Rust macros are super intuitive to me (maybe its the CS background?). `syn` is a Rust parser from tokens -> AST, and it supports doing AST->AST folds and other common operations. And `quote` gives you semi-quoting.

With `syn+quote` most macros end up as 10 liners. Tokens->AST->AST fold->Quote->Tokens.

Huh? D metaprogramming is much closer to "normal code executed at compile time".

Rust procedural macros are like external tools, manipulating the AST as a structure. In D, compile time code is seamlessly interleaved with other code – you just have `static if`, `static foreach` etc. in your code.

I'm not sure what you mean by "new pseudo-language to work with mixins". The term "mixin" is unfortunately overloaded: the `mixin()` call just splices a string into the code, while `template mixin` is a way to expand a template where you want it. Neither introduces new complex structures.

> Rust procedural macros are like external tools, manipulating the AST as a structure.

Isn't viewing them as like external tools just a C centric view, based on C's (and C++'s) capabilities? Didn't Lisp have macros that manipulated the language as an AST prior to C even existing, inside the language?

Well, Rust is closer to C++ than Lisp :)

If you look at the API https://blog.rust-lang.org/2018/12/21/Procedural-Macros-in-R...

it works on the level of raw tokens, not even the AST. So you have to construct your own AST via the parser if you want to manipulate AST.

I don't know Rust macros, but I D had a old example that generates an image, using raycasting, at compile time.

C++ also has template metaprogramming and compile time execution nowadays.

Common wisdom is to leave macros for conditional execution, legacy code and the few cases that still aren't fully designed, namely reflection and metaclasses.

> Well, the alternative is to learn an actual dozen of language

I would say that this is actually almost always preferable, for two reasons:

1. I believe it's generally good for programmers to know multiple languages, to be exposed to different ideas, ecosystems etc. This broadens worldviews, makes you think outside the box more often, etc.

2. Each of those languages will probably be better at what they do than D's subset. For example, D's GC isn't really one of the best ones out there. Its pure isn't the same thing as FP pure. Its move semantics is crap compared to Rust (probably even C++ does them better). The quality of the borrow-checking remains to be seen but frankly I'm skeptical, it'll probably follow the same pattern as the other features.

If you don't mind, I'm curious about what "partially move structs" are? Structs have always feel like a chore to me, complicating things more when compared to maps in dynamic langs. Maybe Rust has something for dealing with the "explosion of types" issue and dealing with partial information in structs that I don't know about. Skimming through the rust book looks like Option<T> is the solution to missing data in structs.

Partially moved just means that the compiler tracks which fields have been “moved out” of the struct and prevents you from either accessing those fields or attempting to reuse the partially moved struct as if it were still a whole struct, since it is no longer whole... it has given up some of its members, conceptually.

In the article linked by this discussion, the author specifically mentions how fields can’t be moved out in D’s implementation.

Option is the correct approach for dealing with information that might be missing, but that’s not really related to partially moved structs.

Structs document what is available in a given value, to both the compiler and to you. If that’s not convincing, then I’m not going to try more here... it’s off topic. But I will say you can use maps in Rust or other statically typed languages. They’re just not meant to be an alternative to well-defined structs.

Sure, my intention was not to get in a dynamic vs static lang argument, when seeing "partial moved structs" and "ergonomics" in the same sentence my mind jump to think that there is something Rust that can make it feel closer to a dynamic lang regarding types ;) Thanks.

Can Rust now handle partial move semantics on structs? I remember reading it couldn't do it before, and there were threads about how you should make small structs because of that, or that you should not use methods, because they take ownership of the full struct and not just the fields that they actually use, etc.

It can, with the big caveat that it only works well if you don't cross a function boundary.

Ah, so passing a struct as an argument to a function will still take ownership or borrow the entire struct?

There are no plans at the moment for D to implement partial struct moving.

Rust doesn't have anything for that. Partial move means that if a function only really need to access one field on the struct, it would only take ownership of that field, even though it took the full struct as an argument. So two functions could have ownership of different fields of the struct at the same time.

One thing with Rust is that it is really focused on performance and safety. It also cares about ergonomics, but not over performance and safety. Unfortunately, type explosion does make for faster code that can be validated more easily for safety by the compiler. That said, Rust has good support for vectors, those are used a lot, but are homogeneous. It also supports tuples with nice literal syntax and pattern matching on them. While it favours structs over maps, they are pretty lightweight compared to objects, so very small syntax to define them, you can create one out of another, and have a nice literal syntax to create them. Still, this is not the magical world of dynamic data-structures, so don't try and think you can just map/reduce over your structs and tuples, compose and restructure them freely, print out their content, or merge and split them, etc..

So the only thing that is "moving" is the ownership of elements of a structure.

It seems "partial move" actually means "partial transfer".

> So the only thing that is "moving" is the ownership of elements of a structure.

Yup

In rust, you can move, copy or borrow a value.

Move - means that the responsibility of freeing the allocated memory is moving to someone else. Rust says the ownership has moved.

Copy - means that the full value is copied, and a copy is passed, thus now you have two instances of the value, and each can act independently.

Borrow - means that the access permissions to the value are temporarily transferred to something else, but will come back when done, and the responsibility of freeing the memory is still up to the owner (lender).

That's why it is called "partial move", cause it's using Rust's terminology. Transfer isn't in the Rust lexicon, so it wouldn't be as appropriate.

One problem with modern languages is that when complexity grows as more and more features are bolted on, old languages like C will look like a viable option. The hardest part for a maintainer of any software is probably to keep features out.