Hacker News Reader

> And that’s why C is still the best language for systems programming

I think the only reason C is very popular for systems programming is that it allows you to do most of what you could do in assembler, but in an easier-to-work way. So basically C is an "acceptable, easier-to-use assembler", but far from ideal, because of the reasons you point out.

Honestly, the popularity of C itself is due to the success of UNIX, not due to any particular quality. On the other hand, the K&R "The C Programming Language" book is a classic, a well written book which surely does a great job of introducing the language.

But back then in the late 1960s, a new language called ALGOL-68 was specified, back then a very advanced language which had powerful features even for 2017. The problem is that nobody dared to implement the full language. CPL was a stripped down implementation of Algol-68, this was then stripped down even more to BCPL; then it was extremely stripped down to B (only one datatype), then Dennis Ritchie kept most of the syntax and added other data types and C was born. C was never designed from the ground up to be the best systems programming language possible; only to be able to work for Dennis Ritchie & friends' "toy" operating system, UNIX, a very stripped-down MULTICS.

I think the "UNIX-Haters Handbook" has a section devoted to C criticism.

> Honestly, the popularity of C itself is due to the success of UNIX,

I beg to differ. C was a fairly obscure language until MS-DOS came out. C turned out to be ideal for programming on DOS, and DOS programming was far and away the most programmed system in the world for a decade and a half.

MS-DOS also made C++ into a major language (via Zortech C++). When ZTC++ came out the penetration and popularity of C++ went through the roof.

(Yes, I'm tooting my own horn a bit here. But I honestly believe it is ZTC++ that got C++ its critical mass.)

MS-DOS made C and C++ into the juggernauts they became.

> I beg to differ. C was a fairly obscure language until MS-DOS came out.

On the early days of DOS, what was used the most was IBM-BASIC (and GW-BASIC) for users, assembler for games like Alley Cat, IBM LOGO for teaching, some games were made on FIG-FORTH even

Then borland brought Turbo Pascal which was hugely popular together with MS QuickBasic.

Meanwhile, and we're at 1986-7, a good amount of software was already made for the Unices, in C. I don't think anyone was doing C for DOS back then. My grandpa -a PC nerd- had a huge (200+) collection of diskettes around that era, which now I own, there are all sorts of compilers for the above languages, plus Fortran, Clipper, etc; but no C compiler. Even though he owned the K&R book.

First time i saw use of C on the DOS world was with Borland Turbo C++.

There were probably 30 C compilers for the IBM PC. Turbo C didn't appear until 1987, it was quite a latecomer. C was immensely popular on the C before then.

Borland produced Turbo C not to introduce C to the PC, but because C was so dominant on that platform. After all, why would they have done that after being so successful with Turbo Pascal?

The Aug 1986 issue of Dr. Dobb's lists:

* Aztec C

* C86

* Datalight C (mine!)

* DeSmet C

* Eco-C88

* Hot C

* IBM C

* Lattice C

* C Prog Sys

* Let's C

* High C

* Microsoft C

* Mix X

* Toolworks C

* Whitesmiths C

* Wizard C

PC Tech Journal reviewed 8 C compilers for DOS in their Nov-Dec 1983 issue, though I don't have a copy of it.

The majority of them unknown or impossible to buy in the 1983's reality of Portugal and Spain, as I mention in another comment.

Sorry but on my little corner of the planet, in the Iberian Peninsula, everyone was using Turbo Basic and Turbo Pascal, when not coding in Assembly for MS-DOS.

On the PC, C and C++ only started to be widely adopted as we started to code for Windows 3.x, due to the APIs being available as C instead of interrupts.

At the technical school I was attending, I was the outlier by having had the opportunity to get hold of Borland's Turbo C++ 1.0 for MS-DOS, everyone else couldn't care less.

Most Portuguese and Spanish magazines of the 80's and early 90's were full all imaginable programming languages for C64, ZX Spectrum, Amiga, Atari and MS-DOS. C didn't had a better spot on those articles than the other languages, quite the contrary.

Fellow Iberian HNers are free to correct my experience of those years.

In my part of the world, we used the Borland compilers for MS-DOS, Turbo C and Turbo C++. Turbo Assembler and Turbo Pascal was also popular. Never heard of Zortech C++ before

Borland decided to develop TC++ because of the success of ZTC++. (I know some of the people involved.) Before ZTC++, C++ was a niche language, and Borland was having great success with Turbo Pascal. ZTC++ came out in 1987, and TC++ in 1990.

After the success of ZTC++ and TC++, Microsoft changed direction and decided to develop a C++ compiler, too. I heard (but was never able to confirm) that Microsoft had earlier been developing their own object oriented extensions to C called C*.

Coming from Turbo Pascal TC++ was a bit underwhelming. When I tried it didn't colour syntax, there was those weird #include and compin=ling felt so slow. So my first contact with C was pretty negative :\

ZTC was a fantastic compiler, that is until it was purchased by Symantec, renamed, and become slow and bloated.

I could even say something similar about turbo c vs turbo c++, but that was more due to the language issues with c++ vs c.

Turbo Pascal was much better for programming on DOS IMHO.

TP was indeed a fine project. But it wasn't OOP (which was very hot at the time) and TP was very customized to the PC, meaning it had no penetration outside of DOS. TP died when DOS died.

TP 5.5 (1989) had OOP. https://en.wikipedia.org/wiki/Turbo_Pascal#Object-oriented_p...

I only started using TP with version 6, so all I know is OOP TP...

TP's OOP was modelled after Apple's Object Pascal. It was an awkward ugly implementation that had e.g. slicing problems and weird initialization syntax, and was fairly quickly deprecated when Delphi came around.

I loved Object Pascal, and it was my introduction to OOP, literally.

As I had to give a class on OOP to fellow students at the technical school as exchange for having access to Turbo Pascal 5.5.

Never was a big fan of some of the Object Pascal changes made by Delphi's class model, specially the fact that it was a kind of lost the opportunity to introduce RC alongside those changes.

Which eventually lead to the schizophrenic model that RC would only exist for COM based classes, but not others.

RC only for strings originally, later dynamic arrays, variants and interfaces; I no longer recall if the legacy OLE automation classes did automatic RC, but I don't think they did (beyond explicit RC in constructor / destructor implementations).

I maintained the Delphi compiler front end for 6 years or so, adding closures, enhanced RTTI, a bunch of work on generics and 64-bit porting, and some other things that ultimately didn't see the light of day. I have a long list of things I don't like about the Delphi language, corner cases you only really become fully aware of when living with the workarounds they force on the codebase.

Overload resolution is an almost wholly unspecified mess, for example. It started out Java-style, but the definition of more specific isn't locked down, and everything from strong typedefs (type TFoo = type TBar;) to closures (MyFunc(methodRef) - do you mean to pass method ref or result of calling method ref), to ranges (are smaller ranges more specific? what about overlapping ranges?)... and don't get me started on all the different string types.

I followed some of your blog entries, hence my earlier comment. :)

The first time I went through the Turbo Pascal for Windows 1.5 manual I wasn't that happy to see PChar, let alone the other variants that came later.

Although it seems that in today's world, most languages end up with a jungle of string types, for every possible variation of Unicode, ANSI and C ABIs.

I eventually did the full transition from Object Pascal to C++, so I only used the very first versions of Delphi.

So in what concerned Windows development, I ended up moving from Borland to Microsoft compilers when Visual C++ 6.0 was released, which means I lack the experience how later Delphi versions evolved.

I was VERY surprised to read about the linguistic traits of [B]CPL .. C was a pragmatic regression in that regard.

For big computers, C is good for systems programming because everyone else uses it and because of the titanic effort that has been put in to building really good optimizing C compilers. UNIX came along but when C killed pascal, IBM, Microsoft, everybody was on C's dick.

On small embedded computers, I think C is popular because it's so easy to build something that resembles a C compiler. The fact is, undergrad compiler classes almost build C compilers (some universities might actually go all the way.) You can almost build a yacc grammar that will read C and emit assembly. The small computer embedded vendors have a difficult problem, they need to provide tooling and they don't usually have giant piles of developers to sell to. So they dust off some abandoned "C compiler" they found in the gutter, add their hardware to it and call it good. At least that's my theory and I've seen some rough "compiler work" from some pretty substantial hardware vendors. In fact, if you're running a 16bit or smaller part or a weird part, I have yet to see a decent C compiler from a hardware vendor.

> You can almost build a yacc grammar that will read C and emit assembly.

I have looked at a lot of C parsers and compilers and have written a C to Lisp transpiler and this statement is ludicrously wrong. You cannot even parse C code with yacc because of typedefs. The grammar of C is context-sensitive. And this is after the C code has been preprocessed, something that also cannot be done with yacc.

C compilers are easy to port to different machines, but that is because C is a very poor language in terms of features and control flow constructs. For the limited amount of things that C gives you as a language it also comes with a huge amount of complicated baggage when it comes to implementation and corner cases.

True about the preprocessor. But the typedef issue is usually (trivially) solved with a link between the symbol table and the scanner.

This is a very important point. The hardware world itself suffers from this effect even more than software with synthesis and PNR for verilog/VHDL.

I'm hoping the compilation world progresses enough with efforts such as LLVM to break our chains to crufty languages.

"do what I want" usually means "simple ABI that is trivial to interface with" which means you "can layout structures the way you want" and "calling conventions are simple so you can call C from assembly and any HLL FFIs and vice-versa".

Really, that's all. If it was possible to build C++ classes in C or assembly, it would be done. But it's not because C++ doesn't have the kind of ABI (or any) that would allow one to do that.

Of course, in practice C only gets you 95% of the structure layout functionality you need, as the rules for packing structures and bitfields(!) aren't sufficiently nailed down. Still, 95% out of the box is pretty good, and with some care you can get 100%. This matters when writing, e.g., drivers, but also codecs for another example.

Any systems programming language aiming to replace C has to provide support for a C-like ABI.

I'm not praising C here, mind you. It can't be replaced (with Rust or similar) quickly enough.

> If it was possible to build C++ classes in C or assembly

You would be surprised, but Object-oriented programming has been done in C many times... and even before C++ existed.

I'm not surprised. But you'll note I said "C++ classes", and I did that for a reason: it's not possible to write C++ classes in C because there's no stable, public ABI for one to write to. (Well, for some compilers there may be a public ABI, and some might even be stable, but even so, it's a mess.)

On particular platforms they have nailed down the C++ layout enough to make this possible. I think windows did it for COM (or DCOM?). But in general you are right.

On of the most infuriating things about fancy languages is the way they all have their own ABIs. I mean it's nice that they support special features through those ABIs, but standardized vtables and dictionaries would take us a long way.

And Kudos to Microsoft for having a second crack at getting that right with .Net.

Yes, that's right, and multiple compilers support the Windows C++ ABI. You still couldn't write portable C++ in C or assembly (or anything else not C++). Does anyone write C or assembly to that ABI? Probably not, though to be fair that's probably in large part because if you're going to do that you might as well write in C++ :^)

Still, I think a public and stable ABI with fine-grained control of binary elements is a critical aspect of a systems programming language.

Also, the linker contributes a lot to making a systems language. E.g., being able to map globals to specific physical addresses. And, of course, either an ABI has to be dead trivial from a linker's perspective, of the linker has to be very intimate with it (think of symbol mangling). The linkers we use all evolved in a C world, and it shows. C w/ a really good link-editor and RTLD (I'm thinking of Illumos') is a much more advanced beast than C with plain old static linking. Even the ABI aspects of dynamic linking are simple, public, and stable.

A great systems programming language has to have a great linker story as well.

I would like a declarative language for laying out the heap, stack, objects, etc. Why have one ABI and not an ABI configuration protocol? Would love to share data structures between languages w/o having to write serializers.

You can't just wake up and decide to be Don Box like that!

I am going to out Box, Don Box!

As a counterpoint, PowerPC has at least 3 mutually incompatible ABIs for C.

But generally there's only one per-OS, so it's not that bad. C++ ABIs are per-compiler, or worse, per-{compiler, version}, with few exceptions (Windows).

> And Kudos to Microsoft for having a second crack at getting that right with .Net.

IBM did it first with TIMI for AS/400.

Also Microsoft is now in their third crack at getting it right with UWP (based on COM).

Objective-C used to be a precompiler to C. Hence the weird syntax, they needed some character patterns that weren't used yet in C to be able to preprocess the Objective-C parts.

> Really, that's all. If it was possible to build C++ classes in C or assembly, it would be done.

IBM's mainframe OS/400, nowadays known as IBM i supports OO Assembly.

MASM also used to have OO Macros.

Just a tiny nitpick: Only signed integer overflow is undefined, unsigned integer overflow follows modulo arithmetics. It's pretty much impossible to remember all those details.

These things originate in hardware variations. Apparently all architectures use binary the same way so unsigned overflow just drops the MSB. But signed integers aren't always two's complement, so there is variation.

See, no tedious rules to remember, you just have to understand how computers work. But the C standards call such things "undefined behaviour" rather than "platform specific" behaviour, and then try to pretend that the compiler can abstract the underlying machine away. That is they pretend programmers can understand the standard, and don't need to know how computers work.

The result is a maze of arbitrary - but historically rational - rules about when the compiler has to do something sane, and when it is allowed to do whatever the hell it likes to squeeze some micro-improvement from a benchmark.

> The result is a maze of arbitrary - but historically rational - rules about when the compiler has to do something sane, and when it is allowed to do whatever the hell it likes to squeeze some micro-improvement from a benchmark.

Which is a nice way of saying that sometimes they'll decide to elide chunks of code because they would only be reachable because of undefined behavior, and if that elided code happened to specifically be checking for and handling that undefined case as an error, too bad.[1] :/

1: https://news.ycombinator.com/item?id=14163111

The C standard actually has many alternatives to undefined behavior: implementation-defined behavior like the propagation of the high-order bit when a signed integer is shifted right, unspecified behavior like the order in which the arguments to a function are evaluated. Related to those are the implementation-defined, unspecified and indeterminate (unspecified or trap) values.

Now unspecified values are tricky again. They can propagate their unspecifiedness and can be different each time you look at them. x == x can be both true and false and is actually unspecified again. If your compiler is using this, things can get insane. I think here the definitions of the C standard would need to be changed a bit.

Also I don't see why signed integer overflow cannot be implementation-defined behavior.

https://kristerw.blogspot.com/2016/02/how-undefined-signed-o...

Unless you are paid by the line that should either work as "expected" (i.e. wrap) or produce an error about a meaningless comparison.

Is it worth a few developer days worth of work to track down a hard to repro bug that only happens with hard to debug optimizations enabled?

No, which is one reason why I almost always use unsigned types in my C code, particularly in the context of data structure management where negative values are unnecessary and usually non-sensical.

GCC supports -fwrapv and -fno-strict-overflow; and I think clang supports both, too. I've never cared to use them because I only rarely use signed types. But some projects and programmers use those options habitually.

AFAIU, Rust panics by default on signed overflow. And even if it wraps, that's not unequivocally better. Unlike with enforced buffer boundary constraints, neither is clearly better than what C does. Arithmetic overflow is a common and serious issue in just about every language. Short of a compile-time constraint or diagnostic that triggers if the compiler cannot prove overflow is either explicitly checked or benign (that is, a negative number is no worse than a large positive number in the context of how the value is used), there's no obvious solution that really forecloses most exploit opportunities across the board.

Because so much code, regardless of language, has some unchecked signed integer overflow bug, if you panic you make it easy to DoS an application. And a DoS can sometimes turn into an exploit when you're dealing with cooperating processes. For example, you occasionally see bugs where an authentication routine fails open instead of failing closed when the authenticator is unreachable.

If you silently wrap signed overflow, all of a sudden the value is in a set (negative numbers) that might be completely unexpected. Even in so-called memory safe languages negative indices can leak sensitive information or erroneously select privileged state. For example, in some languages -1 selects the last element of an array. You can check for negative values explicitly, but multiplicative overflow can wrap around to positive numbers, which is no better than using an unsigned type; a check for a negative values is typically redundant work which adds unnecessary complexity--and unnecessary opportunity for mistakes--relative to sticking to unsigned types.

IMO, signed overflow is the worst option. I just don't see the point. The only three options I like for avoiding arithmetic overflow bugs, depending on language and context, are

1) Check for overflow explicitly (independently from array boundary constraints) and bubble up an error;

2) Carefully rely on unsigned modulo arithmetic;

3) Carefully rely on saturation arithmetic.

IMO the C standard's fault isn't in its refusal to make signed overflow defined or implementation-defined, but in providing neither a standard API for overflow detection, a construct for saturation semantics of integer types, nor a compilation mode to warn about unchecked signed overflow (e.g. something at least as useful as -Wno-strict-overflow in GCC).

Fortunately both GCC and clang have agreed on a standard API for overflow detection. That's something. But unfortunately it'll be years before you can consistently rely on those APIs without worrying about backward compatibility.

> AFAIU, Rust panics by default on signed overflow

Overflow of any integer type is considered a "program error", not undefined behavior. In debug builds, this is required to panic. In builds where it doesn't panic, it's well-defined as two's compliment wrapping.

You can also request explicit wrapping, saturating, etc behavior.

Thinking about hardware is definitely the move when writing C.

This is the major struggle with abstraction. We want to remove the burden of knowing the ins and outs of the target architecture. Inevitably, we create trouble and fall on our faces when it turns out that the hardware is still in fact there and doesn't like when we ignore it.

It's really an impossible problem. One can't account for every architecture when designing a language. Likewise, one can't feasibly remember the details of every architecture while programming. Honestly I'd be interested to see some tools that approach the problem from a direction other than maximum portability. Not that I think they'd be popular or "good".

Easy check the family of Algol, Xerox PARC, Wirth languages.

Where safety is more relevant than maximum portability.

Everything that isn't really portable is marked as explicit language extension or unsafe construct.

One might complain that it leads to language dialects, but the same is true for C, where certain semantics depend on the compiler and even change between versions.

At least one extant (or recently extant) system has to emulate unsigned, modulo arithmetic. This can be handled by the C compiler transparently, however. From the C compiler documentation:

  | Type          | Bits | sizeof | Range                                  |
  +---------------+------+--------+----------------------------------------+
  | ...                                                                    |
  | unsigned long | 36   | 4      | 0 to (2^36)-2 (see the following note) |

  ...
  Note: If the CONFORMANCE/TWOSARITH or CONFORMANCE/FULL
  compiler keywords are used, the range will be 0 to (2^36)-1.
  See the C Compiler Programming Reference Manual Volume 2 for
  more information.

  -- Section 4.5. Size and Range of C Variables of the Unisys
  C Compiler Programming Reference Manual Volume 1.
  https://public.support.unisys.com/2200/docs/cp16.0/pdf/78310422-012.pdf

A range of 0 to (2^36)-2 implies that there's one bit combination not mentioned here (that range has only 2 ^ 36 - 1 values; 36 bits can store 2 ^ 36). What's the last combination used for?

I don't know off-hand. AFAIU the Unisys machines use ones' complement representation. My guess is that the native unsigned set of values includes the representation for both positive and negative 0. Or there could be a trap representation that is hidden in unsigned mode, which presumably would also make these machines examples of hardware that traps on signed overflow.

Wow. 9-bit words with 1's-complement arithmetic.

I wonder if it still does end-around carry...

I wonder how many processors there are in use these days that don't use 2s complement. I don't think I've ever seen one.

Certainly not general CPUs, but there are probably domain specific processors out there that use something else. Why would you want to design a domain specific processor and still use C? Beats me.

Even if you don't really want C on such a process there will be an emergent and unholy aliance between (1) a pointy-hair impulse within the manufucturer to have "programmable in C" on the feature list and (2) an empire-building impulse within the C standards writing ecosystem that wants to encompass every chip under the sun.

Those forces are so strong people are still pushing for C on FPGAs.

The bigger reason it's still undefined is to enable this type of optimization: https://news.ycombinator.com/item?id=14857316

The comment thread seems to suggest that even if you define the behavior you can still optimize that case, and in fact Clang does.

> you just have to understand how computers work.

rude.

Yup. You'd probably like the first footnote if you haven't already read it.

> C doesn't "let you do what you mean", it has no knowledge of special registers, interrupts, timers, DMA, etc.

Special registers are just slots at specific memory addresses.

As far as the rest is concerned, not a single language in the world will provide you with primitives for that and make them portable across a thousand architectures with a million different peripherals. That's the role of libraries and OSes with their drivers.

> Special registers are just slots at specific memory addresses.

> As far as the rest is concerned, not a single language in the world will provide you with primitives for that and make them portable across a thousand architectures with a million different peripherals.

I'm well aware, that's why I said.

> When you're dealing with the hardware architecture level you need more detail than C (or any PL) can reasonably provide.

A language does not need to provide every hardware primitive, just realize that it can't and implement a sane way to support hardware features that don't require abusing a cumbersome macro system and heavy compiler modification.

> That's the role of libraries and OSes with their drivers.

A ridiculous amount of vulnerabilities and wasted effort are sunk into operating systems and their drivers because of the garbage state of tooling for low level work. It's easy to make statements such as yours until it comes time to write/maintain a multi million line kernel and a only the lord knows how many line driver/subsystem fleet.

Kernel/driver devs do their best to make due, but the situation is far from ideal.

> A language does not need to provide every hardware primitive, just realize that it can't and implement a sane way to support hardware features that don't require abusing a cumbersome macro system and heavy compiler modification.

The information has to somehow be passed to the compiler, your options are: compiler specific features (see gcc attributes, etc), language level (see the fun that becomes with explicit_memset vs. memset) or a really fancy IR that can hold these types of requirements.

You're right, most of these problems stem from companies extending the Garbage C Compiler that encourages abuse of C's awful macro system to cover up its poor decisions.

> hardware features that don't require abusing a cumbersome macro system and heavy compiler modification.

I'm confused in that, it's true, I've seen the macro system abused. But for working with things like memory mapped registers and interrupts you don't _need_ to abuse it. It actually looks really nice with with memory-mapped structures that ply on the register structure. Maybe I'm just missing something.

> but if it makes an honest effort to be ergonomic for embedded

There are constant improvements but I wouldn't say we're there yet. I think we will be though, embedded is something a lot of folks care about.

Good to hear. I'm sure there is a large group of developers sitting on the sidelines that check in from time to time hoping to be able to leave C behind.

Has anyone on the rust team tried reaching out to get input/contributions from companies who are heavily invested in the embedded world? I know xilinx pours a lot of effort into language research, they do seem like an very proprietary company though.

We reach out and talk with companies regularly. We are both happy to organize a meeting with companies who approach us, and often (~yearly) organize calls with multiple companies using or evaluating Rust at once whom we reach out to. I believe there have been some of these focused on embedded stuff, but I wasn't on these calls.

The latest on this front: http://blog.japaric.io/rtfm-v2/

Found this paper useful: "The Case for Writing a Kernel in Rust" (https://www.tockos.org/assets/papers/rust-kernel-apsys2017.p...) linked from the "tock os" blog post: https://www.tockos.org/blog/2017/apsys-paper/

Yes, this is a good paper, but was posted after my comment :)

It sounds like you're looking for Forth. It doesn't have everything on your wish list, but it's a good bit more predictable when you have to get down into the weeds.

Thumbs up for driving more nails into C's coffin. What any pretender to the throne needs most is C interoperability because we're stick with legacy code. You need to be able to just code in C when necessary including the datatype zoo, the unsafety and the ugly macros. If it takes even 5 minutes to wrap a C function for your new language, that's years of work for some codebases.