Rebooting a 15 year-old game written in D – Part 1 Compiling
speps.fr
speps.fr
For impatient, the game is now playable in the browser here: https://torustrooper.xyz/
[1]https://www.youtube.com/channel/UCK3fxU3_7tzBhZZ_6rljAQA/vid...
[2]https://forum.dlang.org/thread/r4lqd3$ad1$1@digitalmars.com?...
A lot of languages have a lot of "syntactic sugar" but actually what's important is more invisible basics: good GC, good concurrency primitives, good method dispatch, good and comprehensive standard library, supportive compiler, etc. Go has a few warts (the whole select / <-chan syntax, god-mode functions like `make', the interface{} problem, the whole go mod / dep kerfuffle), but it does tick all those boxes.
Ofc all that modulo the Google factor, but then there's Dart...
Just like we have WebAssembly because those same companies did not bought into PNaCL.
It's a weird way of enforcing structure and the last time I looked into it I couldn't get my head around it
I may be misremembering, but when I was learning D the only thing the spec was useful for was for metaprogramming, everything else was pretty obvious.
I.e., say we need ten features to make up any language. Perl arranges six of these into syntax. Go arranges say two of these into syntax. C++ arranges five into syntax. For D, I've had little exposure but for better or worse let's say it arranges 4 into syntax. For each, the rest are in the std lib. The extreme cases would be, maybe Scheme for 1/10; can't think of any 10/10 syntax languages.
In that light, we can say that Go requires less upfront investment in order to become productive and read code easily: learn two features + how to lookup docs. Versus in D: learn 4 features + how to look up docs.
That's pretty ad hoc, but I hope it helps get the point across.
D also solves a problem you can't really know you have unless you have a deep knowledge of programming. The idea that if you want to write small code vs. fast code rather than both in the same language in very strongly ingrained because we traditionally teach people C++ and/or Python and that's it unless you do advanced courses (to learn something like Haskell which still doesn't help), even though both are fairly crummy programming languages.
If all goes to plan I may be working to help popularise the language soon, so we'll see about all that.
Our academic group adopted it because it allows us to be both incredibly productive and have performant code, and both python and Go programmers were able to get up to speed quickly.
With respect to being ahead of its time, there are many features that have been in D for years slowly making their way to other languages, including C++. In that respect it is successful, but certainly I would like it to be successful on its own.
D is very similar to Lisp in many aspects. It's a powerful language which allows you to create your own minilanguage within a language. In Lisp, you'd do it with macros, in D you can use templates. This makes it a secret weapon which appeals to lone wolf programmers, but it makes it harder to interact within a wider community. Similar with the language features. D allows multiple paradigms, as a result, many libraries don't interact well together. Some people code in OOP fashion, some people refuse to use the garbage collector, some people even reject the entire runtime (betterC_mode).
[0] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
His request to "please stop submitting this to Hacker News!" should be interpreted in its strongest form; i.e. including comments. It's right there, at the top: the first thing you read.
D looks fine. It suffers from the curse that we cannot have 1000 languages be popular simultaneously. Popularity is a game of musical chairs. People who don't get chairs when the music stops are not always the clumsy ones who lack athleticism and reflexes. Sometimes, in that crucial moment, the nimble one is just far from the closet chair relative to the klutz whose butt is inches from the seat.
In popular culture, popularity wanes fairly quickly. Yesterday's idol is quickly forgotten. Like Billy Joel's line in "The Entertainer": "But I know the game, you'll forget my name / I won't be here in another year / If I don't stay on the charts."
In computing, we can't swap tools quite that fast. So for fairly long periods of time, popularity begets more popularity. Something is popular today because it rose to popularity some years ago. (Some things have longer cycles, some shorter.) Anyway, that alone creates a barrier to entry, similar to a monopolistic barrier. No matter how good something is that you create, it just doesn't have ... the popularity to make it popular.
I see the comment at the top about not re-submitting to HN, but I don't see disownership or disavowal.
Is it posted somewhere?
Because asking not to resubmit (which GP didn't), is not the same thing as saying "my conclusions were wrong".
I don't think I agree with its basic conclusion that language flexibility leads to a kind of social problem akin to a "curse."
From what I've seen over my career, both C and C++, which the essay claims are not cursed, have many libraries that don't interoperate well, re-invent wheels, are under documented, solve only part of a problem, and are basically solo/hobby efforts. In every job, the first thing I have to learn is how I'm allowed to write C or C++ at the company, since each provides such a huge surface.
The difference between C/C++ and Lisp/D is, I think, not language flexibility. They are all incredibly complex and flexible languages in their own way. The difference is funding and critical mass. C and C++ persist because companies pay people to use them and to improve them. Or, in terms of the curse essay, companies pay people to cooperate. The result is an enduring dominance. Those languages work better because more people use and improve them, so they stay relevant. Neither Lisp nor D seem to benefit significantly from that, so you have what is left: people without the time or interest in producing truly polished solutions.
Unfortunately, these microcomputers only had the memory and CPU power of big iron from 25 years before that. The best applications and games for these machines were written by assembly language wizards.
The C language played right into that. C came from an environment of "mid-grade iron" from about ten years before that. It was developed for the efficient programming of machines that Unix ran on, which were roughly similar in power to the consumer-grade microcomputers of the 1980's.
There were tools like Pascal and Modula also, but C came shrouded with a naive mystique thanks to its cryptic syntax, reputation for bending to the programmer, and creds due to its Unix connection. Naive programmers believed that expressions like i++ were faster than i := i + 1, or else, in any case, they liked it better. Moreover, in some situations, an expression stuffed with side effects in C would actually generate better code than less densely coded equivalent (the only equivalent available in a Pascal family language), due to the limitations of compiler capability. Coders whose ego was built on writing big programs in assembly language and getting them debugged were less threatened by a cryptic language in which you could write programs that were hard to understand, and those used to macro assemblers would also have liked the preprocessor. The ability to abuse the language to get the compiler to do what you want also would have helped. "C gets out of the way so you can get the machine to do what you mean (unlike those other things from the Wirthian family). Before there was an ANSI standard, that would have applied double. The printf function with its format strings also blew away Pascal and Fortran.
On the latter point, I started to learn C at about the same time I enrolled in engineering. There was a mandatory Fortran course in the engineering curriculum. In one of the assignments, I wanted the Fortran program to have numeric output with dynamic column widths. It was really difficult; I can't remember whether I even got it working. I also remember the issue that Fortran's output would replace a fixed field with * characters when the value would not fit, instead of just expanding it. A few months later when I was learning C, I made it a point to try to do that with printf. Wow, I was easily able to do that. I was thinking, why would anyone use Fortran for number crunching if we have C?
person * me;
Is this a multiplication statement, or a pointer to a person typedef?
I think there was an aspect of writing a C compiler that was easier, which was namely that certain optimizations in C were just left to the programmer. To get good code out of C, you depended on the basics, like decent instruction selection and peephole optimization, and jump-threading, and maybe elimination of dead code.
Then to otherwise get good code out of C, you stuffed multiple side effects into a single expression, which allowed the compiler the generate the computation and storage accesses any order without analyzing whether any particular order is correct (ensuring they are all correct was left to the programmer, who often didn't care as long as the one compiler he or she were using gave them the right results).
Even the assignment of local variables to registers was programmer-hinted: C had the (now deprecated) register keyword. With the register keyword, your compiler just had to place those variables in registers which were marked with it, and screw the rest. No analysis about which ones should be in registers: particularly good in register-starved architectures. The microcomputer world was characterized mainly by register-starved architectures, with the notable exception of the Motorola 68K series.
[1]: https://attractivechaos.wordpress.com/2012/02/28/timeline-of...
Which is quite unfortunate, as the language is quite appealing and the community always has interesting discussions.
They are heavy, bloated, and try to do everything, at the end of the day they pick the lowest common denominator, for everything, and they ship it
They want compile to a small executable, they are stuck and ended up just zipping everything and calling it a day
They want to interop with native world, but they introduce runtime overhead and their GC/memory model is a barrier
They want to AOT compile, but their runtime and libraries rely a lot on runtime reflection
They want to be tiny, but they have ton of legacy package that makes them very heavy
Just look at the success story of go, it invalidates any attempts to make both dotnet and java attractive, it is too late, even more too late in an ARM world where relying on a JIT is a weakness
What does this even mean?
> They want compile to a small executable, they are stuck and ended up just zipping everything and calling it a day
Both can compile executables the same size as go? See core-rt or the graalvm native-image compiler.
> They want to interop with native world, but they introduce runtime overhead and their GC/memory model is a barrier
Java's GC's are probably better than any on the market, including go's. And I will agree that currently native interop from the JVM is more expensive than it should be, but they're working on it.
> They want to AOT compile, but their runtime and libraries rely a lot on runtime reflection.
Easily solved by just declaring what you reflect upon or using runtime agents to find it. It's more difficult than a language than was designed to do it from the start, but eventually all of the big libraries will have it declared already. Hibernate already does [0][1]
[0] https://github.com/hibernate/hibernate-orm/blob/5eedda9a467f...
[1] https://github.com/hibernate/hibernate-orm/blob/5eedda9a467f...
Java has had AOT compilation since around 2000, in fact the free beer AOT compilation and JIT caches now available on OpenJDK originate from BEA J/Rockit.
Likewise in what concerns the free beer AOT and JIT cache on J9, it originates from WebSphere Real Time JVM.
As most FOSS people weren't willing to pay for them, so urban myths about Java compilation support models get cargo culted, while those of us that don't have any issues working for the man got to enjoy them.
Likewise .NET has had support for AOT compilation since the beggining with NGEN, then Windows 8 adopted the Bartok compiler from Singularity and Windows 8 .NET Native steems from Project N based on Midori learnings.
And then there are the older ones from CosmOS, Mono, IL2CPP and the upcoming CoreRT.
The success stories of Go are called Docker and Kubernetes, it hardly matters for anything else, while Java, .NET, alongside C++ and JavaScript, own the enterprise.
Even Google has decided to replace the Go written parts in Fuchsia with either C++ or Rust.
Java AOT story is all over the place, and it's not even good, remember the 300+mb executable from Excelsior? AOT low profile? yeah sure it invalidates the whole language features
Dotnet AOT? i don't call experiments, products, and they have the same limitations as Excelsior AOT, they produce insanly large executables, or if you want to target something like GO, you need to give up on the language features, i don't call this a good experience
Il2CPP, it's a transpiler; and same story as Excelsior AOT, it produces insanly large executable, the GC, and the code produced is very poor quality due to the nature of the language (lot manipulations at runtime, including reflection)
Go success story is that it now go beyong docker and k8s, people write cli tools in GO (github cli one good example), they write all their servers in GO (Riot for example), and some start to do gamedev with GO (ebiten)
Google is still using GO, it's not a system language it has a GC
And since the Rust guy left, their rust effort vanished, at least publicly https://github.com/google
oh and BTW, we'll see wich one get flagged, the one who call someone a TROLL because he give valid arguments, or someone who call someone a delusionist
XBox, Windows 8, 8.1 and Windows 10 aren't experiments, rather products used by more people than any software that you have probably ever produced.
So what that IL2CPP is a transpiler? So were Objective-C and C++ on their early days, and yet they have a market share that Go is yet to achieve.
Also it shows there is some learning required on the compiler design front.
People also write CLI tools in almost every language, hardly a measure of success of any form.
And for Go not being systems, that is actually something that it definitely is, as proven by F-Secure and Google.
https://www.f-secure.com/en/consulting/foundry/usb-armory
https://blog.arduino.cc/2019/08/23/tinygo-on-arduino/
Which still isn't the same as PTC Perc, Aicas JamaicaVM or Meadow.
Remember where was Python when 20 years old? It's just another scripting programming language on the block and was playing second fiddle to Perl and TCL. Ruby was hot on its heels due to RoR hipsters [1]. Replace Perl/Rust, TCL/C++, Ruby/Go and RoR/Kubernetes, see the pattern emerging for interpreted/compiled programming languages diaspora.
But now look how far Python has come despite the transition from Python 2 to 3 fiasco. At least D has passed the stage of growing pain while transitioning from D1 and D2 with minimum damages.
[1]https://insights.dice.com/2018/03/09/hipsters-ruined-ruby-fi...
Remember when Python went from 2 to 3? It was incredibly painful. But the transition is over now. Depending on who you ask, Python is now the most popular[0], third most popular[1], or second most popular language[2].
When I started learning Python, it was not popular.
[0]: https://www.tiobe.com/tiobe-index/ [1]: https://www.northeastern.edu/graduate/blog/most-popular-prog... [2]: https://insights.stackoverflow.com/survey/2019
Python is old.
I say this in all seriousness: this is probably a mistake, and the Dlang crew should make plans to exploit the opportunity to do a D3/3D release to the greatest extent possible—if only for marketing purposes and while holding onto the secret that there are no breaking changes involved.
Also, I think that stuff such as garbage collection and reference types seriously fell out of fashion, because there's no way in hell lots of C folks will ever consider a language with them, sadly.
And since we are speaking about games here,
https://docs.unrealengine.com/en-US/ProgrammingAndScripting/...
D is Lindy Effect. It's been around for 19 years, it'll probably stay around for at least another 19.
> The Lindy effect is a theory that the future life expectancy of some non-perishable things like a technology or an idea is proportional to their current age, so that every additional period of survival implies a longer remaining life expectancy.[1] Where the Lindy effect applies, mortality rate decreases with time.
So Rust may be hot now– but remember, NoSQL and Docker were hot once too, and they are now not popular.
Plus D is adding Rust's ownership and borrowing.
D compiles faster than C++20 and is easier to learn.
Is this for GC code or just @nogc and DasBetterC? Ownership seems a bit heavy when running the GC.
https://github.com/kubernetes/kubernetes/blob/master/CHANGEL...
Kubernetes alternatives (?):
The D programming language was trendy for a bit, 15 years ago. Now it's just old tech people make blogs about reviving retro games as a gimmick. It never got substantial traction because it sat half way between Java and C++ in a way programmers from neither camp would switch to it. Rust is fundamentally different, it attacks C++ at its heart and is not only a better language, but a superior programming platform. In a few years the ecosystem will be large enough that there are no reasons for anyone to choose C++ anymore, and at that point, what is the relevance of D?
Saying that "@live" feature is "adding Rust's ownership and borrowing" is technically accurate, but in reality I'd be surprised if it were used in production in major systems in the next 5 years.
The @live feature has multiple problems:
- The author wants it to be implementable incrementally (eg some code uses @live, some doesn't), but doing so isn't really feasible in safe code. From the moment you disable GC, you either need your entire program to respect some strict memory model (in which case you basically have a new programming language, and no incrementality), or your program won't be any safer than C++. This is not hyperbole: Rust-with-unsafe and D-with-@system is safer than C++, because you only need to audit an annotated subset of your codebase. D-with-@live (or more accurately, D-with-@trusted-@live) is not safer, because a memory error can come from anywhere in your program.
- Speaking of a memory model, D doesn't have one. Rust has Stacked Borrows, which gives library developers a framework to know whether their unsafe code might lead to undefined behavior. D doesn't seem to have nailed down semantics as to how @live might create/affect undefined behavior (which means the semantics will probably be "whatever the DMD/LLVM/GCC backend wants").
- Rust has Non-Lexical-Lifetimes (and soon Polonius). D's documentation is very sparse about how lifetimes in @live will be determined, but I think it's safe to assume that it will be scope-based at first. This means that, eventually, when @live ships, it will be roughly equivalent to Rust as it was in 2015. Not great.
And, finally, saying that Rust will decline because another language will add OB misses what makes Rust great.
The strength of Rust is that things work by default.
A quick search through D's forums will raise dozens of instances of people complaining about edge cases in the language, things that don't work exactly as expected or produce undefined behavior (autodecoding and uninitialized variables come to mind).
Rust has very few problems like that. If your Rust code compiles, it will work. You will never have a bug that happens on a coworker's machine but not on yours, or a crash that happens because you used two niche features that weren't designed to be used together.
If your code crashes, it will be in a predictable, reproducible way, with an error message and a stack trace to tell you what to fix.
Rust isn't perfect, but it's infinitely less fragile over time than D.
@live also has non-lexical lifetimes: https://forum.dlang.org/post/r742l6$2ofo$1@digitalmars.com
On @live, I'm not sure Walter is going in the right direction, but I think some of your arguments aren't the strongest. I have no issue with the incremental use of @live. People who use D know the difference between @system/@trusted/@safe and @live (esp. when @live is actually shipped and there is adequate documentation). The relevant question @live supporters could make is if @system/@trusted with @live is safer than without it. I think your point about D not having a memory model might be fairer, I'm not really in the position to judge it. Your final point is a bit weaker. Walter could easily argue that the idea is to move towards incorporating these features from Rust. Of course Rust is going to be further ahead on this. @live is also not thoroughly documented and there's a lot left to be determined.
I think on @live I've been most convinced by skeptics that something like a new type qualifier or storage class is better. This storage class would basically enforce the equivalent of Rust's borrow checker (as in @live) and only allow one mutable or unlimited const references but not both. Users can also apply it incrementally, but is a bit more significant of a change than @live.
All variables are initialized in D, so not sure what you have found there. A language that is in active use generates tons of questions though :)