The long goodbye to C
esr.ibiblio.org
esr.ibiblio.org
Raymond is not in fact in a good vantage point to judge the suitability of Go or Rust as "systems languages". His claims to authority in this post are "NTPsec" and GPSD†, but Raymond neither designed nor implemented either of those programs: NTPsec is a hostile fork of ntpd, and GPSD --- which is probably not a good example of a program that makes particularly strenuous demands on its programming language runtime --- was maintained by Russell Nelson, after being written by a different team --- before Raymond claimed it for his resume.
In this middle of this article, Raymond claims that Python is suitable largely for single-user applications operating "at human speed". This is so far outside the reality of how Python is used in industry as to call pretty much everything else Raymond writes in the article into question.
Surely, somebody else has to have written something better on the trend towards next-generation systems languages.
† These are the examples he provides, in this article, by the way.
Could you elaborate? Are you talking about e.g. numpy and scipy? Python is good at calling C libraries but that doesn't make it good for writing an IP stack or a device driver or an OS. Are you arguing against the "single-user" point or the "human speed" point?
It's hard to find programming language that will cover as much low level programming needs as C. If some aspiring hacker turns 14 today and wants to get into the low level systems programming/embedded programmer career learning C well is a must. They may use some other language more, but they need to know C unless they want to limit themselves.
Some processors have stack size just 2-8 and memory available can be something like 128B - 8KB. Any replacement must have programing model that is very low level. Any replacement for C must have very low level and simple programming model.
C++ may lose some ground to Rust but it may just end up saturated like Ruby, PHP did. Ruby is basically in steady state now. NodeJS jump in on a crowded market and got no where let alone it's crazy threadless model.
C++ btw have been pumping out new versions, C++17 was just completed by GCC and Clang is almost done. C++ is really trying to modernize.
Google whole backend is most likely C++... it ain't going no where.
Rust is a beautiful language but it still have a long way to go. And also resources such as books to help people jump on board not just general books either but books in different domain (scientific computing, data structure, web dev, etc.. whatever).
I haven't used it for very low level work, more for things like command line utilities (and with Lazarus for some light GUI app work). But the EXE/binary sizes are fairly small (without any optimization or stripping or compression). Also had used Turbo Pascal heavily much earlier. And I've read that Pascal at one time was used for systems programming; Apple may have been one company that used it (much earlier), and that their OS APIs were in Pascal. Windows API's (in Win32 at least) also may have had (not sure) some support for Pascal, viz. keywords like "long far pascal" in Win32 API code.
Side note: I wonder how many people use those abbreviations like viz. these days.
And there are still some companies selling Pascal compilers for the embedded space, like this one https://www.mikroe.com/
They also sell Basic compilers.
Of course C++ can just revert to being mostly C in these cases, and C is lightyears ahead in tooling and documentation here, but as a language I think Rust has a decent shot at that space.
One of the benefits of being a truly actively developed language, as opposed to fossilized Java.
Who knows, maybe it even gets value types before the C++ community can agree what modules are supposed to be.
And even then none of that solves the problems I mentioned in the comment you replied to.
While it may be true that ESR is not a superstar programmer in the image of a John Carmack or a Bill Gates, he has been a working C developer for over 30 years, and has contributed to numerous open-source projects, including systems projects, over that period of time. The examples he lists in the article are some pieces of software which he is currently a maintainer for.
Actually I think the fact that he is not a superstar programmer with a signature project possibly makes him a bit more qualified to comment on language trends. After all, most of the developers in the industry are by definition about average, and if average (or even somewhat above average) developers are finding their productivity much higher in Python and Go than they do in C, then they will start doing more work in those languages and less in C. In fact, that transition is exactly what ESR is describing in the article.
Certainly it seems that Go is usable for certain classes of problems that would formerly have been C's domain.
I'd by productivity you count time to solution, a lot of applications written in python do not have performance requirements, which makes the comparison very unfair.
I cannot say for Golang.
Edit/Continuation: I feel like the freedom offered by C works because of it's simplicity, and as a language for embedded systems I still enjoy developing in C. As someone who recently started working primarily in C++ though I am frequently astounded by the choices that were made. And to make things worse, offering C style controllability in a complex OO language doesn't work well. I shouldn't have to feel like I need to read the designers minds to predict how a language will act.
I think what replaces C should be something like Nim, which compiles to C and also has great C interop. C++ is definitely not it.
C has no standard language provided range checks, and pointer security, but the amount of compiler macroses that do the same/coverage tools/runtime security tools/checks provided by compilers is huge.
At least part of "inherent issues" of C that Rust advocates often cite was long solved by these tools long time ago.
They are not a bullet proof solutions to lack of security features in C, but neither are the built in checks of novel languages. The amount of NX breakouts, syscall hijackings, clever tricks around range checks in so called secure languages being demonstrated every year confirms that.
In the safety critical programming we use subset of C or (C++) with static code analysis. It's possible to prove that there is no dangling pointers, divisions by zero, floating point exceptions (with assertions), out of bounds array access. It's called verification.
Programming is limited into MISRA C and then it's run trough static code analysis tool like Astree to prove that there is no runtime errors. There are still bugs, but they are not "typical C errors". Waiting for new systems programming candidates to get their toolchain to match the task.
Ada, doing it since 1983.
The security track records of nearly all large scale network facing C projects disagree with you.
Nothing prevents quitting structural code in C++. Or even functional for that matter. (Though function objects are not quite light yet.)
There are even safe subsets where at most you will get an exception when you try to do something dirty. Easy to enforce.
Certain things like compile time calculation (being enhanced by the moment) do not even reasonably exist in C or for that matter Nim.
The about one gripe with C++ is ABI or rather lack thereof. Three other is the amount of legacy code written in C or outdated dialects of C++.
For instance, I had a ETL program I was writing to process 100+ GB files. I pre-allocated all the memory I needed for exactly one row of data and immediately printed out the data when it finished that row (stream style).
The entire program processed 100 GB with lightning speed in less than 1K of memory. Maxing out the performance of the disk.
Best of all, I only had to free my memory at the end. Completely safe. No danger of double freeing and no extra memory allocation at all. Literally a dozen mallocs at the start and a dozen frees at the end.
Could I do that in Go? Sure. Would it have as optimized memory access? Maybe... but probably not. And C my entire program was 200 lines of code and absurdly fast.
Don't get me wrong, I love Go. I write programs in Go often. But for simple apps where the code can fit in a single file nothing can beat C. No bloat, no extra memory operations, and a code optimizer that has been being optimized for 30+ years. Go's code optimizer is not even close yet from what I can tell.
Sorry if it's a naive question. My experience with C is limited.
90% of your program performance is going to be dominated by cache usage(unless you're doing something really dumb). With C you can place things next to each other explicitly so that your prefetcher/etc is automatically pulling in the data you need to the L1 cache.
Any ref'd/GC'd language is going to make that really hard(not impossible, but you have to jump through a million hoops like sun.misc.unsafe in Java).
Look up Data Oriented Design if you're interested in this stuff, Mike Acton does a great job covering it.
All it takes is 4-5 of these and now you're incurring performance hit of ~1ms which gets you into SSD territory[1].
As always, profile before making these optimizations but the OP was asking why something like this isn't possible in managed languages.
[1] https://arstechnica.com/information-technology/2012/06/insid...
Not true at all.
Oberon, Active Oberon, Component Pascal, Modula-2+, Modula-3, D, System C#, Nim all offer ways to manually control memory outside GC control, just like C and C++.
Glad to provide Modula-3 examples to any C tricks you might want to do, or in D if you prefer something more modern.
Note that I didn't say that it was impossible, just that the ergonomics of GC'd languages don't tend to focus on performance in the ways that languages with explicit memory control do.
GC systems programming languages offer exactly the same explicit control as C does, in unsafe memory blocks.
Do you want to shot yourself like C in Modula-3? Declare a variable as UNTRACED REF, 100% like a C pointer.
Do you want a C union? Declare a tagged RECORD type.
Do you want a packed record to fill you cache line? Compiler alignment pragmas.
Allocation of value types on the stack or global memory section? Great, it is the default allocation type.
Bit fiddling to prepare a statically allocated buffer for a DMA transfer? UNSAFE code section.
Yet you just talked past my point to give some sort of soliloquy about Modula-3.
It doesn't matter how those languages died, it just matters that they did. The ecosystem and number of people who know how to write a language is almost, if not more important than the language when it comes to shipping software.
We should not think of Java when comparing system programming languages with GC against C.
Yes, the languages you mention meet all the technical requirements(congrats, here's your 10 internet points) it doesn't matter if I can't find people who know how to write it.
Since you seem to want to avoid talking about anything I raise I don't think it's productive to continue this conversation.
For example, all the work with System C# done in Midori has been being applied to C#, with .NET Native, ref returns, local ref variables, value tuples, regions with disabled GC, memory slices, with a few more down the roadmap, like effectively manual memory management presented at OOPSLA 2017.
Or the ongoing @nogc and BetterC improvements in D, Wallmart, EBay, Netflix, Remedy Games among many others, seem to be able to find people that know how to write it.
I don't know how you concluded that nothing can beat C because Go is most of the times slower.
You can see it has pretty similar performance for a problem [0] pretty similar to yours.
[0] http://benchmarksgame.alioth.debian.org/u64q/revcomp.html
Looking at that benchmark that tiny difference in performance would have have been very magnified in my use case.
Though I will concede that it is possible that while slower it might still max out disk I/O and once the bottleneck isn't RAM or CPU it doesn't much matter what language you use.
And based on that, Rust does beat C.
That is something I have not done in Go and is good to know. But I could be wrong on this one, I have no idea how Go buffers are performance optimized but reading a character from a C array is one CPU instruction... it feels intuitive that in Go just the fact the extra layer of abstraction is added on top means it will be more instructions on the CPU.
But that is just a gut, I have not tested it.
Which in my use case would make a big difference over time.
But my intuition also tells me that it would still max out disk access in Go with modern CPUs.
Edit:
I should mention that my use case was to processes as many gigs of files on as cheap an AWS server as possible while maintaining a very strict SLA on how fast processing should be (the data became obsolete in hours and when we tried higher level languages the data was obsolete before a single file could finish). So I was trying to squeeze out as many CPU cycles as possible.
I should also mention that Go didn't even exist yet in a stable state when I did this project.
"Re-write Linux in Go" I think would be a massive but classic folly
It happens to be a trope that C would have died if not for Unix[2], but to the best of my memory of reading programming books of the eighties, C's un-safety was a _feature_, not a bug (I remember Java books were _so_ apologetic that Java didn't have points arithmetic).
In the days before optimizing compilers and slow computers, the ability to write unsafely manipulate arrays was actually useful when one needed speed (and if one wanted to be a "real hacker", one needed to optimize every byte and instruction.) Now it's true that not every program needed that speed, and quite a few would be willing to trade speed for safety, but no "self respecting real programmer" would allow himself to write in _anything_ that wouldn't let him be that "uber-hacker" (sounds familiar? I guess Resume oriented programming is an old problem).
[1]. Before I get the pedants that C++ isn't C, I'd like to remind them that the Windows NT codebase started in the late eighties and early nineties, when C++ _was_ almost like C with classes.
[2]. In reality, quite a few languages took off which were not the same language as their OS (for example, Java, Python, C#). If anything, I'd say it's more important that the compiler vendor be the same company as the OS vendor (see Borland's history), but even that seems like a more modern problem (again, see Borland's history)
I never did manually resource management with them. RAII was already a thing in MS-DOS.
Java and C# were pushed by OS vendors.
C will be around for as long as everything under the sun has a C compiler. If you ask yourself when was the last time you started a C project, ask yourself when was the last time you wanted to write something truly (truly, including embedded) portable instead and there is your answer.
So until we get to replace POSIX across the board, C will be around.
It will take generations.
"There are 8 standards and they all suck. We should make ONE good one to replace them all."
...
"There are 9 standards and they all suck. We ..."
POSIX is clearly a standard for C unixes, but you could imagine a restricted subset of POSIX that didn't govern libc being applicable to a non-C Unix.
Any operating system written in a non-C language with no C support is going to break all existing C applications. This is inherent to OP's desire to remove all C. For the record: I'm opposed to removing all C.
So why not preserve unix-like shells, filesystems, devices, commands, etc? None of those are tied to language of implementation. Potentially existing unix shell scripts could run safely on a purely non-C system. Sysadmins would carry over some familiarity from existing Unix-alike systems. That has some value.
Unix servers make up 68-98% of the internet, depending on who is measuring. There's a huge body of sysadmins familiar with shell who are not familiar with Powershell.
So that's exactly my point: I don't think we should start from scratch on things like the Unix shell, filesystem hierarchy, devices, etc. People know them. Throwing out systems people already understand for no reason is a huge anti-pattern.
If you think Powershell is superior, port it to Unix and convince Unix sysadmins to use it. I don't think it's going to be very popular.
Once you've learned how it works having an object oriented shell is game changing. Powershell isn't necessarily the ideal implementation but I'm just calling attention to it as an existing, popular example of something better
Bell Labs tried to kill their child and supplant it with plan-9 and inferno.
It failed.
Not because they were bad OSs, but because once you have market domination and code, things _must_ be backwards compatible.
If Linux/GNU didn't imitate POSIX, it wouldn't be any more popular than Haiku.
Maybe, but there are projects like D's betterC[1] that are designed to remove that reason to use C as well.
If D became available in all of those embedded platforms, I am sure lots of people would rejoice. Or maybe they would just shrug because they would not know what hit them. But it would be their loss, because D is, in many ways, C++ "done right". Which IMHO is a remarkable achievement.
Either way, I would not hold my breath for C becoming obsolete in the sense of "no one is using it any more". It is not happy, but for now it is all we've got.
I'm one of those people and I spent ~6 years in embedded and I still write C today (though it's no longer my primary language). Of my former co-workers, probably 50 people in total across 4 teams, exactly 0 of them were at all interested in any sort of C alternative. I'd wager less than 6 of them even knew what C alternatives were even picking up steam. If it wasn't talked about by folks like Michael Barr or Embedded magazine, they simply wouldn't have any way of finding out.
> Either way, I would not hold my breath for C becoming obsolete
I wouldn't either. Call it the old guard, call them greybeards, it really doesn't matter. They're the ones buying chips from vendors that supply C toolchains and compilers. Until that is addressed, none of this talk matters.
As someone explained to me, it's the reason Rust doesn't have a nice initializer syntax like C++11 for example[1].
1. http://en.cppreference.com/w/cpp/language/list_initializatio...
it's not really how "everyone" uses it. it's a narrowing that a subset of vocal people have been trying to push, i can only presume for ideological purposes.
These aren't uncommon targets, the PSP traditionally had 8mb of system memory. Lots of embedded routers and "IoT" things, etc.
Part of this is confused by everyone having a different definition of "System Programming". In my view if there's a target where hardware constrains prevent suitable performance from a language then that language isn't fit to be considered a "System Programming" language.
The only thing that Go lacks in regards to Oberon are a few intrisics in the unsafe package, easily doable in Assembly, just like ANSI C requires on its libc for bare metal programming.
C gets put into a pedestal, yet it is impossible to write an OS in pure ANSI C, without making use of language extensions or Assembly.
I've done it and it's a pain in the ass. Why? Because the GC is _non-deterministic_. On many languages it can fire up at any time, freeze your threads at any time, and take however long it wants to. "But it usually doesn't take that long!" And? AND? The problem is, you've immediately added a blackbox thread attached to all your data. Enjoy having your program segfault in areas that are completely unrelated because the GC didn't run the finalizer till later. Enjoy having to fix a bug that doesn't have a repeatable effect. You're automatically writing multi-threaded code (and all the complexity that entails) even when you have a single-thread.
A GC adds an entire dimension to bug fixing and potential failure points. Every bug, multiplied by every possible interaction with the GC. This is something that systems programming usually cannot afford. You need to be able to prove your code won't crash.
And that's one more thing. Tons of systems code pre-allocates everything for that crash reason. So what problem is GC actually going to solve when everything is already carefully pre-allocated?
It really sounds like people who aren't used to unmanaged languages are simply projecting their fear of the unknown. I use both types on a daily basis, and to me they're just "the right tool for the right job." I would never try to shoehorn the wrong tool into an ill-fitted application just because I like that tool.
You're adding a ton of incidental complexity for a benefit that you can't take advantage of(programmer productivity) because you don't have the necessary resources to make it happen(large swaths of memory and > 50ms response times).
I was a Native Oberon user during the 90s.
Educate yourself on Oberon and Modula-3, read about Midori and then feel free to carry on the anti-GC campaign.
If you're interested in Rob Pike's take on Go, he expressed regret over initially calling Go a systems programming language, because it gave people the impression that it was appropriate for writing operating systems. He prefers to call it a "server writing language" or a "cloud infrastructure language." @6:45 in this panel discussion: https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Pa...
As for the links they are easy to find on my comments history.
I don't have to repeat them all the time the anti-GC crowd comes to town.
Note that the set of people who read your post today probably includes people who did not read your previous posts on the topic.
Rust is well suited for OS code if not for the inertia that most OS devs are experts at C and not so in Rust and compiler support. Those two factors are going to become less important going forward, so Rust (with generous 'unsafe') could replace C in its last stronghold too.
Additionally, alignment constraints are also really important at that level, but rust doesn’t have a mature way to align structures or arrays.
Rust isn’t ready for this space yet.
You're certainly right about "mature way" wrt alignment, but it's something that can be worked around until then.
[1] https://github.com/spotify/linux/blob/master/include/linux/g...
Hail to Erlang and even more, Elixir.
Otherwise you pay the cost when you really do not need to. Just ask people involved in actual places where microseconds matter, such as real time control or high frequency trading.
Erlang does not provide these primitives enforcing unnecessary copies in a bunch of places.
There will probably always be a need in computering for 'Portable Assembly' too, which is another reason C (or C like languages) will never disappear.
And I think it is even more true for FORTRAN
I was confused there for a second because both "CE" and "CS" literally start with C.
By the way, to add another datapoint, I completed a bachelor degree in CS from 2012-2017 (part-time) at a German university, and we used:
- C in the first-semester "algorithms and datastructures" course
- Haskell in the second-semester "programming" course
- Java in the third-semester "software design" course and accompanying fourth-semester lab course
- Python (very briefly) in the fifth-semester "information retrieval" course
Those picking up the optional compiler related lectures, also got an overview around Lisp, Algol, Fortran, Concurrent C, Objective-C, ML, Cobol, Oberon, Modula-2.
Nowhere C was taught as such, we were expected to know about it from the C++ lectures.
It’s less stringent compared to Rust’s borrow checker, but also has meta programming capabilities of similar power to macroses. The default is GC manged memory like in Go.
However it’s super flexible in that you can go from 100% simple GC-based programming with dynamic arrays and maps to hardcore low-level stuff with manual memory or anything in between (e.g. Ref-Counting as in C++).