DigitalMars C/C++ Compiler
github.com
github.com
If you want to build it, you'll need to download the C++ binary from digitalmars.com, and the D compiler from dlang.org.
Also related is that D is now completely under the Boost license as well.
Historically D was "partially open, partially closed"; the D frontend (syntax parser and some other pipeline stages) was open, but the backend/code-generator was built on top of the Zortech C++ compiler for DOS/Windows/OS/2. (I only have a hazy notion of the Datalight>Zorland>Zortech>Symantec transition; I'll bet the story would make for an awesome history book, if that doesn't already exist. Hint)
There was (highly appropriate) major fanfare when Symantec, who bought Zortech in Aug 1991, allowed the compiler backend technology to be open-sourced in April 2017, 1 year and 1 month ago: https://news.ycombinator.com/item?id=14060846
And that, fairly clearly, is what made this source release possible. Also https://news.ycombinator.com/item?id=17131274 :)
References (most insightful first):
- http://www.edm2.com/index.php/Zortech_C%2B%2B
- https://en.wikipedia.org/wiki/Digital_Mars
Would probably be interesting to convert it in C++17 :-)
Meta-comment: I delete my previous comment by error and want to put it again, and didn't manage to do it again. Is there some anti-duplication code that prevent to write 2 times the same comment on HN?
Can confirm, that's what I'm doing. Most of the code is written in an old-fashioned style, but rather than redo that, it's more productive to convert to D, and then into a modern style.
The D conversion as it stands is by rote translation, thereby retaining the technical debt. I don't plan on improving the structure until it is all in D.
Thanks for all your awesome hard work by the way!
Yes, it was Warp:
https://github.com/facebookarchive/warp
https://code.facebook.com/posts/476987592402291/under-the-ho...
You're welcome! It's reactions like yours that make it worthwhile.
Can you please add, if you have time, a few lines (the commands required) for building the compiler from source ?
make clean
make scppn .\code.d(16): Error: module `cc` is in file 'ddmd\backend\cc.d' which cannot be read dmd -c -betterC -noboundscheck -mv=dmd.backend=. -mv=tk=tk -version=SCPP -O -release -inline dt.d -ofdth.obj dt.d(36): Error: TypeInfo cannot be used with -betterC
--- errorlevel 1
--- errorlevel 1 dmd -c -betterC -noboundscheck -mv=dmd.backend=. -mv=tk=tk
-version=SCPP -O -release -inline -J. dfile.dBTW, I did all my tests on a clean Windows machine with nothing else installed.
c:\dmc\dm\bin\link /de @scppn.lnk
OPTLINK (R) for Win32 Release 8.00.13
Copyright (C) Digital Mars 1989-2010 All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
err.obj(err)
Error 42: Symbol Undefined _stdout
c:\dmc\dm\lib\NTSTUB.EXE
Warning 2: File Not Found c:\dmc\dm\lib\NTSTUB.EXE
Another potential problem is that I don't have Visual Studio installed on this machine.
On a more positive note, I think I will give D a try after this experience.
Try defining stdout as this:
enum stdout = &_iob[1];
So, I've commented: shared stdout = &_iob[1]; and added: enum stdout = &_iob[1];
Now, I have 5 executables HTODn.exe, msgsx.exe, optabgen.exe, SCPPn.exe, SPPn.exe.
Edit: Answered below in "May it have been better to start from scratch?"
https://www.joelonsoftware.com/2002/01/23/rub-a-dub-dub/
> 1. No New Features, not even small ones, would be added.
> 2. At any time, with any check in, the code would still work perfectly.
> 3. All I would do is logical transformations — the kinds of things that are almost mechanical and which you can convince yourself immediately will not change the behavior of the code.
I'm not sure what exactly you have in mind for "modern style", but I found the code as it is to be quite readable and a breath of fresh air from a lot of other code I've heard referred to as "modern" (which seems to mostly mean laboriously long identifiers, excessive amounts of abstraction, and CamelCaseScreamingAtYouEverywhere.)
It is bad enough that I considered not open sourcing it for fear of damage to my career :-) but then decided what the hell.
The good thing about the DMC++ source code is, it works. You can make anything work if you invest enough effort into it.
If you are also confused by the title, DMC = DigitalMars Compiler. It is a C/C++ compiler which seems to be almost completely written by one person
* Compiler implementation of the
* $(LINK2 http://www.dlang.org, D programming language).I haven't been able to find much on Tom Pennello besides some parsing papers from the 70s... I'm interested, but don't know how to find more about his work.
Was the "D as a better C" approach inspired by your want to transition DMC to D?
Yes, it's been oddly satisfying.
> May it have been better to start from scratch?
Ah, the age old question. I can guarantee, though, the answer is no. It took me maybe a couple weeks to convert the entire front end from C++ to D. Contrast that with the Warp preprocessor I wrote for Facebook, which took many months, and I knew what I was doing. And that is just the preprocessor!
Basically, writing a C++ front end takes 10 man years. I know of no effort that took less time, and this is assuming the best in the business doing the writing.
> Was the "D as a better C" approach inspired by your want to transition DMC to D?
Partially, yes. And certainly doing the transition work has improved betterC. Nothing like proving it works than translating a crotchety old horrifyingly complex piece of code written in 1980's style.
I plan on doing this to all my old projects that are still in use.
This is insane. No language needs to be that complex. I guess that's the power of backward compatibility: forcing an entire industry to dig itself into an ever growing hole, because in the short term we move faster by digging than by first clawing our way out.
(Yes, you can be productive in C++ in much less time.)
Back in the aulden days, it could also be run directly from the distribution CD without requiring an install. These days the "install" is merely unzipping the file.
You created the first "better C" out there to get any traction, it had a perfect name (B -> C -> D), but due to factors that were partially [2] outside your control [3], D's niche got eaten by Go and Rust.
[1]. The D programming language. [2]. I've seen people blaming the D to D2.0 transition, but D was a niche language before too. I've also seen people blame D for not having an Open Source compiler for years, but neither did Java, and it took off anyways. [3]. Such as Sun's marketing, or the general push to scripting languages between 1995-2010 (such as perl, tcl, PHP, Ruby, Python and Node)
"Certainty of death. Small chance of success. What are we waiting for?"
https://www.youtube.com/watch?v=8joT0oFuGoI
For a language without a megacorp pushing it, D is spectacularly successful. Consider:
1. It's developed by volunteers who work on it for the joy of it without remuneration.
2. People who spend the effort to get past the learning curve find it very pleasing and productive.
3. Other major languages have been continuously copying D's features :-)
4. Corporations that have adopted it for mission critical use have told me that D is their "secret weapon" that allows them to out-maneuver and out-innovate their competitors.
It's like the music business - you don't have to be the Beatles to have a very good career.
You are an inspiration Walter !!
Thank you for all your hard work, your products formed the center on my early career, and most of what i know i learned using your tools.
Note I particularly loved "Zed" the text editor that came with the early toolchain.
Regards Tim Hawkins, a grateful hacker.
That is an inspired quote. I shall steal that and use it whenever anyone argues "why x when y already exists".
Yes, seen many people argue that, including to me, e.g. about some of my blog posts, which show hand-rolled ways of doing things when a library for that exists. I guess often such people are not doing anything much themselves, and don't get that people can create something in the same area where something else exists, for any number of reasons, even just for sheer fun, love of the field, or for teaching beginners.
I think you're too humble here. The quality of D is no lower than the competing products backed by huge corporations.
I also vaguely remember using Symantec C++, it was a refreshing alternative to the competitors of the time, and I had very pleasant experience using it.
For someone who is in both the music business, but also a D programmer for many years, I got to say this was the most inspirational quote I have ever read and it'll be my inspiration in both fields.
That represents a misunderstanding of D. Unlike Go and Rust, which were built for one specific use case (Go for the work done at Google, and Rust for writing a browser), D is an all-around good general purpose programming language. If it did have a niche, it would be as a C replacement, and it is the best available tool for that.
Your comment also implies that D is not used. Download statistics, forum activity, and the existence of users suggests otherwise.
For my entire life as a Software Engineer the idea of a close sourced or proprietary compiler was just alien to me.
What set you down the path of writing compilers? Did anyone buy licenses to your compilers? Why was D started?
A more technical question: Was there any abstraction/pattern that has been deeply rooted into your code base that you are unable to change due to prevalence yet this pattern/abstraction makes it more difficult to maintain your code base? In any other words: have there been any mistakes made?
When I started, everything was closed source. Open source was inconceivable.
I started writing compilers in order to speed up my game Empire (www.classicempire.com). Compilers turned out to be more interesting :-)
Yes, many bought licenses.
D was started because I got bored with retirement (6 weeks of watching TV will do that do you). My ideas for improving C++ met with enormous resistance. So I did my own language. Ironically, D features have steadily been adopted by C++.
> have there been any mistakes made?
Haha, I could spend months responding to that!
Probably the worst abstraction in the DMC code base is the use of linked lists. They're slow, cache-unfriendly, and leaky. I learned my lesson with DMD, which uses adjustable arrays instead.
I've tried encapsulating the lists, like CodeBuilder in the back end, but it's a LOT of work to get that in. It wouldn't have been so much work if the list abstraction didn't leak the structure of the list at every opportunity.
That seems to be a recurring pattern in the history of software. Unix was created for similar reasons (to create a self-hosted development environment for game), and the Linux kernel started out as a terminal emulator.
Waw, you retired "young".
Who'd have thought that would ever happen? I wrote it simply because it was the game I wanted to play.
H/W: i386 machine 25 mhz, 2mb RAM. Internet: No OS: MSDOS IDE: MultiEditor + Zortech C++ Where: in the middle of nowhere - small town between Europe and Asia.
At some point upgraded the whole setup to Symantec C++ on Windows - for unknown now reasons.
Loved the compiler and the UI. Best UI ever. Esp. if compared to the Turbo/Borland stuff and then MSVC.
Thank you for the great work you are doing.
If you could get people to include D in the benchmarks / shootouts that get published it would help improve the popularity of D.
FYI - Thatneedle.com already uses it for part of the backend data processing workflow.
I stopped publishing benchmarks myself 15 years ago as people always assumed my thumb was on the scale.
Please share the secret of your tenacity and inspiration in the face of such "setbacks". How do you do it on a day to day basis?
Everybody likes to make more money :-)
Back when I was a brand new engineer at Boeing, I was talking to my lead engineer about the beauty of aerospace engineering. He smiled and said I didn't get it. Boeing wasn't making airplanes, they were building money making machines for airline companies. That the airplane turned out to be a beauty was just a happy side effect!
D isn't about my personal aggrandizement. It's about how effective it is for users at solving their problems and making money for them.
It can be though to shake "early license impressions" - D (dmd specifically) is one example, Ada (FSF GNAT vs AdaCore's GNAT Pro and Ada-gnat - GPL w/o runtime exception) is another.
[ed: if others are interested in "make your own measurements and host it yourself", relevant page with link to code etc appear to be:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
]
Not since I asked him to stop dropping by the D forum to remind us that he wouldn't include D in the shootout.
I've personally lost all interest in it.
(The benchmarks are also small and too easily gamed by the author of the benchmark code, and the compiler developer. I encourage people to run timing tests on their own code, as that is what matters to them.)
(That said, D is lightning fast for me!)
If you really, really want to, you could use https://crates.io/crates/cargo-script to do the way you're talking about.
https://doc.rust-lang.org/cargo/ has installation, getting started, a reference, all of that. It needs some work, but for the basics of doing this, it's got it all.
Gaming from the compiler/runtime side would be uglier - but I guess it is somewhat mitigated by real languages running the "general release" version.
No harm in "fastest way to list first 1000 prime numbers" being "print static list of first 1000 prime numbers".
Hand there's some value in having a standard benchmark harness that works easily across languages - as a helpful tool with "running your own benchmarks". Assuming the harness is any good, that is.
I just got tired of the vitriol leveled at me with no basis. Things like I must have "sabotaged" other compilers. Probably the absolute worst one was when the journalist decided that Datalight Optimum C ran the benchmarks so fast, it must be a compiler bug and removed the benchmark results from his compiler roundup, calling DC a buggy compiler.
The reality was Datalight C was the first C compiler on DOS to do data flow analysis, and it deleted dead code. (Benchmarks of that era did nothing useful, and dfa detected that.) No cheating at all. A couple years later, everyone did dfa.
This is the heart of the matter: benchmarking toy and unrelated bits of code amount to the same thing.
I think what people really need is a straightforward, accessible understanding of where a software implementation is inefficient. Design inefficiency can sometimes be "hacked over" (read "paved over") a bit like evening out technical debt; implementation inefficiencies, perhaps not so much. Making a complex database query faster by fiddling with (very) obscure SQL options because a given vendor's query planner is broken in a rare edge-case is one example that comes to mind.
I think it's especially sad when a software implementation becomes renowned for its inefficiency. It kinda takes the heat off in what I would argue is a very unfair way, like it legitimizes slowness like speed isn't a worthy pursuit... and then we wonder why our computers are slow. (Yeah, I'm thinking about Python here... and to a small extent many interpreted languages.)
In the context of D (and now DMC++ :D), so specifically compilers, it would be interesting to know what areas of the language don't generate especially fast code, or what bits might produce code that uses a little memory than it could, etc. Because that's what people really want to know before they take the time to write/port; and if they know about all the instances of "don't do XYZ in this very specific way" ahead of time, maybe they can write the best possible code on the first try! (Design and implementation are intertwined in practice.)
I suspect the list of such "avoid"s may not be long. It might make for a particularly efficient kind of developer user manual.
https://forum.dlang.org/search?q=benchmarks+game&scope=forum
For the simplest reasons that I have stated to you many times!
Including every language implementation that the language author would like to have included is more work than I am willing to do, period.
Sept 13 2008: 63 language implementations were shown-
https://web.archive.org/web/20080913030117/http://shootout.a...
- currently, 27 language implementations are shown.
You could truthfully say "he refuses to include [at-least 30 language implementations]". In that regard, there's nothing special about D.
What about you create a technical specification (including both technical depth and common-sense breadth) for what kind of benchmark you'll accept, throw the whole thing on GitHub, and then refuse 99% of pull requests? :) (ie, only accept really really good quality benchmark implementations)
Eventually, enough developers unimpressed that Language X is not adequately represented would step up to the plate and maintain good-quality benchmark code. (This could get pretty interesting with rapidly-evolving languages like Rust.)
Obviously this is all very ideal and I can see so many ways such an endeavor could go horribly wrong, sure. I can also very easily see you having floated such an idea then discarded it for reasons I haven't even thought of.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy.
What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X.
Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh.
And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis...
Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.
- To clarify, I meant the Gen1 Raspi.
- I now realize (after having actually had a proper look) that while the Micro:Bit does use "JavaScript" and "Python", they're different, microcontroller-specific implementations, and I should have used a different example there, like an Intel NUC or perhaps an ODROID board or similar.
Just systematically enough to be reminded that a different context may have different relative performance.
There are many of benchmark projects that include D and don't appear to be struggling under any kind of massive burden. For example:
https://github.com/kostya/benchmarks
What are your actual reasons for not keeping D... "scala... and Clojure and ..."? The results in the previous link show D as a massive competitor (sometimes 1st place beating C and C++) on both memory and speed. Wouldn't the purpose of benchmarks be... to highlight useful, highly-scoring languages? Isn't that one of the primary reasons people read benchmarks?
(The D implementations are also often smaller in lines of code, as per this benchmark project:
https://togototo.wordpress.com/2013/08/23/benchmarks-round-t...
)
Do you see any mention at kostya/benchmarks of why ".NET Core" or Ada isn't included?
>> … benchmark projects that include D and don't appear to be struggling under any kind of massive burden.
Check when those kostya/benchmarks language implementation versions were released.
>> What are your actual reasons for not keeping…
I don't want to do the work.
It’s a less serious question to ask, but was the name in any way related to Run-DMC, the 80s hip hop group?
The community is awesome, super friendly and super helpful even in the face of the most stupid newbie-questions I could come up with. ;-)
In the end, I backed out, because the language was too big and complex for my taste and needs. But if you kind-of-like C++, you are likely to fall in love with D.
I will probably give it another try one day, if only for the super-friendly and welcoming community.
Did you take up Sirius's offer to act as broker for 30% cut on deal?
What is your opinion of Rust?
It seems to me like Rust and D have the same target audience and both want to be the next C++.
D, for example, achieves memory safety in a very different way than D's borrow checker.
https://github.com/DigitalMars/Compiler/blob/7a33233c/dm/src... https://github.com/DigitalMars/Compiler/blob/7a33233c/dm/src...
DMC++'s EH is not setjmp/longjmp, it is Windows SEH (Structured Exception Handling) based. The 32 bit DOS extender EH is based on my own table-based design.
I'm a fan of D as a language, but how do you respond to the question of "We should just go with C# or C++, why go with a middle-ground language like D that few programmers know?"
Given that D --> C++ migrations have happened in the past, this strikes me as D's big challenge: finding its place.
I'm thinking in particular of OpenMW: https://forum.openmw.org/viewtopic.php?t=2189
Heartbleed. A C buffer overflow bug so disastrously expensive it has it's own logo!
(I know C# doesn't have buffer overflows either, but C# is a much bigger change than going to D, and is less portable.)
This 'unsafe by default' philosophy isn't universal in C++ though. The smart-pointer class-templates are always thread-safe, despite the possible performance consequences.
For OpenMW though, it's not a concern. They're building a single-player video game; the input is always trusted. As far as I can see, they made the right call by moving to C++.
Again, I really like the D language - but I can see why a project manager would go with C++.
This is a question I'm sure many have: how feasible would it be to get DMC++ emitting Linux code?
D uses the same backend, in theory, but I'm not sure how much divergence there is between the D backend and this code.
Perhaps you could get Hello World to build, make a gigantic list of everything that's broken, and see how much traction (hopefully) appears out of thin air :)
I love tcc for its "oh it already finished" compilation time (300ms :D), even (and especially) on older hardware. I wish there was something out there to compare it with; it's quite unfortunate it sits on its own, because it hasn't had an official release in in 4 or so years.
I've wanted to play with C++ for quite some time, but (as crazy as it might sound) gcc/clang's multi-second compile times for the smallest of things (eg, messing around with FLTK, which doesn't go that crazy with C++ features) makes tinkering a bit distracting. Multiple seconds for medium+ things, maybe, but tcc has spoilt me when it comes to rapid iteration, heh.
A C++ compiler for Linux with speed reminiscent of (or maybe even similar to!) tcc would have a very big (if quiet) impact for a few people, I reckon. Something nice to have one day.
(Unrelated, but fiddling around to get `dmd -c' + external libphobos.so + ld.lld makes for especially nice compilation speed :) )
Is there any specific reason the last step you speak of hasn't been taken? Just time/inertia, or reluctance for technical reasons? I think many people would be interested to hear what effort you'd want to fold into a Linux release, if there are any technical issues.
It's apparently really nice to use (I'm on Linux :( )
It was written by one person
It targets DOS, Windows and OS/2 (mostly Win32)
https://forum.dlang.org/group/c++.announce
I can't find anything about what changed between 8.52 and 8.57. But in 8.52, there wasn't even support for C++03, which the DMC site self-reports:
https://digitalmars.com/ctg/CPP0x-Language-Implementation.ht...
cppreference maintains a compiler support grid for features in the C++11 and later standards, which includes DMC 8.52 and mentions a small number of supported features.
http://en.cppreference.com/w/cpp/compiler_support
Here's a post on the DMC forum from 2015 which seems to agree that C++11 support is generally not there.
https://forum.dlang.org/thread/dbmsmttdbpfjsuhrhpkd@forum.dl...