My not-so-humble-opinion is that if you find yourself doing metaprogramming with the C preprocessor, it's time to upgrade to a more powerful language.
Like kernel drivers, embedded (microcontroller) development.
Or perhaps you just need to develop a small loadable library without any runtime (no malloc, etc.). Or perhaps writing Python, Lua etc. C-module for better performance.
You just need to treat C with proper respect, not be too arrogant about your skills.
I hope over time we can move more and more to Rust or some other language which gives as many errors at compile time as possible and helps with not shooting at our feet.
Shouldn't have too many C-macros, of course. Some macro messes I've seen can be nearly impossible to debug...
We've been hearing "C isn't the problem, bad programmers are the problem" for 40 years. This has continued to perpetuate the problems with the language.
I mean, I certainly agree that there are no alternatives to C now for certain domains, but there are real problems with it that aren't solved by just "treating it with proper respect".
But often you just need to get the job done, and have limited options available.
Perhaps in 50-100 years, people looking back at software from this era won't be able to understand how we managed to make it work at all.
I haven't met or know anyone mastering C/C++ (especially C++), and doubt I'll ever hear about one.
C can definitively be mastered. C at it's core is pragmatic minimalism. The basic ideas are very simple. If you want to be a language lawyer and know the standard completely (including all historical accidents), it can get complex, but still manageable. But anyways, you shouldn't be a language lawyer. That's not C. Make maintainable programs instead, stay in the middle of the road.
C gets out of the programmer's way. It's not about mastering C, but about mastering programming. And while I agree there's much bad C software out there: I haven't seen one maintainable non-bloated Java project. Here is one master of C that I'm sure you know: Linus Torvalds. Of course there are many, many more. Just remember it's not about C, but programming machines in general.
If that's true, how come there are almost no widely used network facing C programs that didn't have plenty of security vulnerabilities?
(Anything written by Daniel J. Bernstein doesn't count, because there's no way he's a mere mortal. :))
Postfix is also much more widely used than qmail.
Goto, you just can't do without it.
Also not saying that in some cases even C is too much abstraction. But in general, its memory abstractions and its function call abstraction have proved to be very effective helping humans develop in a modular fashion.
Strange, the last time I used it I was still doing Basic in the early 90's, or when I do some Assembly programming.
So yeah, we can do without it, as long as the languages are powerful enough.
Not saying there's no use for backward gotos, just don't remember ever needing them. Maybe they could be useful for some C code generation cases or perhaps for retrying some operation?
What really annoys me are C++ exceptions used for flow control. Surprise gotos. I loved exceptions long ago at first, but over time the feeling has shifted more and more towards hate. Exceptions make code bases so much harder to understand. Especially when combined with inheritance.
There are indeed C compilers that don't support recursion (or function re-entrancy). What does the standard say about stacks?
Recursive function calls shall be permitted, both directly and indirectly through any chain of other functions.
[1] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
> For such an object that does not have a variable length array type, its lifetime extends from entry into the block with which it is associated until execution of that block ends in any way. (Entering an enclosed block or calling a function suspends, but does not end, execution of the current block.) If the block is entered recursively, a new instance of the object is created each time. The initial value of the object is indeterminate.
I also haven't seen one large-scale network-facing C project written using reasonable development practices (i.e. not absurdly expensive formal verification) that hasn't had some sort of terrible remote code execution vulnerability. By contrast, Java programs tend to be far less vulnerable to this kind of problem. (The closest thing is probably bugs with attacker controlled java.io.Serializable, which while severe are much less frequent than RCE in C programs.)
> Here is one master of C that I'm sure you know: Linus Torvalds.
I agree that Linus is a master of C. Linux is also an excellent example of the above problems with C.
Use your tools wisely. Architect programs for clear data flow. Validate inputs as soon as possible. Don't do (de)serialization by hand.
By all means use a managed "memory safe" language if you have many data accesses that are not behind a validation gateway. But for large amounts of data or complex software, that's still not an option. C is still the only language in which I can write software of advanced complexity - because it does not get in my way.
Mesa/Cedar at Xerox PARC, Topaz at DEC/Olivetti, Oberon at ETHZ, JavaOS at Sun, Singularity/Midori at Microsoft Research
> Game Engines
Unity is slowly rewriting parts of their C++ pipeline with C#, after their IL2CPP compiler got mature.
Then there are game engines like Xenko being done.
Also being with one foot into games since the 80's, I remember the transitions "Pascal/C are too slow, we stick Assembly", "C++ is too slow, we stick with C", now we are in "JavaScript/C#/Java are too slow, we stick with C++" phase.
This has gotten a lot better with static code analysis (clang-analyzer, Coverity, ...) and whitebox-fuzzing (libfuzzer, afl, ...). Both simply point you to the bugs and don't require a formal spec.
Together with Valgrind, these tools have revolutionized C programming for me.
Pragmatic minimalism as it was for the hardware at the beginning of the 1970s, to be exact.
The C machine model says nothing about vector instructions, instruction re-ordering, NUMA, GPGPU, IO ports, Harvard architecture, multi-core.
Modern compilers are still rather bad at vectorization. What's worse, those optimizations are often unstable. A small change might make code several times slower. Performance regression tests and when that fails, vector intrinsics are a must.
I’d say I had a working knowledge of C++ after about a year ramping up on a codebase based on a mature C++ framework.
That was after years of writing reliable, performant C code. My C++ is now significantly better than the C I can produce, on both fronts, and at the same time.
I’m also much more productive as a side effect of that.
template<template<class>, template<class, template<uint>>, class>
There are equally important reasons why C's usage has been in steady decline from the '90s to today.
More important was it was the only language that really worked on DOS, which ruled the computer biz for 10 years. I would guess 80% of programming was done for DOS, and C was a very good fit for the 640K systems.
C and C++ had no special place on the spectrum of programming languages usage, unless we were porting code between MS-DOS and UNIX.
Until it was time to move into Windows, and anyone not using C or C++ started to be left behind, manually writing FFI wrappers (e.g. Turbo Pascal/Delphi) and we eventually migrated to one of them.
You always have other options if better languages can compile to C or integrate it via FFI. There's been plenty to do that, too. For simple and safe, Modula-2 was early one compiled to C that couldve had syntax modified to be more like it. If aiming for power, PreScheme was a systems dialect of Scheme that compiled to C at one point. The OcaPIC (Ocaml) project and use of ATS language for 8-bit show stuff like that could probably be made to wofk in areas C is used in, too.
So, I'd say it's a myth you need to develop in C just because a target has only C libraries or compilers that safer languages can use. Instead, we get a new claim that people just didnt do it for reasons that varied per project or person but wasn't technical capability.
How does that sound?
The majority of them also enjoy Pascal and Basic compilers.
So sometimes, there is indeed an option.
Also many languages have compilers that are able to generate C code.
As to ensuring, well that is what testing is for.
Use Common Lisp to generate C code. There are three libs that are specific for this, see C-mera for example.
Is this a joke, or do they do some sort of stack-bounding static analysis?
This is true. With all the new and fancy languages around, it's strange that there are so few attempts at making a better C.
I would say that the characteristics of C is: dead simple, manual memory management, "fancy assembly".
Rust is close, but fails on being dead simple.
The best attempt I know of, is Zig: http://ziglang.org/
But it's still very early in development. I am using it for microcontroller code right now though, and it's much nicer than C already, despite having to deal with bugs.
My guess is that in most (not all) cases C++ is usable instead of C, but the culture of such development is to prefer C. It's a real pity, because for someone with a moderate amount of judgement (i.e. not going off the deep end on unnecessarily complicated C++), it only takes a moderate amount of C++ knowledge to be able to write more maintainable and correct code that performs equally well.
For example, to produce generic SIMD-accelerated code over both Intel-specific SIMD operations and the Sleef vector math library I needed for FHT-based JL-transforms and kernel projections, I used template specialization to abstract to operations on physical memory. [0]
The generic methods themselves were gross, but in application, it was extremely simple in downstream code to specify an operation and a container and have the compiler pick the fastest method for the job. I’ve also checked my generated assembly and the smaller functions are always inlined, even though they’re called from static constexpr function pointers.
This is also a great use case for macros, as they saved me huge amounts of typing.
I’m sure many will say C++ is the wrong direction to go for metaprogramming, but it’s worked great for my purposes and lets me get straight to the hardware.
[0]: https://github.com/dnbaker/frp/blob/master/include/frp/vec.h
Python has incredible productivity and versatility with ease of learning. It's readable. It's easier to prototype things like verification tools in it than C. There's also tools like Hypothesis to test that. Then, there's tools to speed up the code like Cython.
If anything, Python would be a safer, faster way to write tooling for modifying or testing C code than C itself. One could also convert Python generated tests to a C library of tests with automatic validation of data types, coverage, etc. So, I think it would make sense if someone did anything from prototyping C in Python to using it to test C code.
And do note Ive enjoyed reading your experimentation with macros in C. I learned about defer through it. Just commenting on that one point.
TLDR: because you need to test the real thing, and using Python to test C code causes too many important differences between tests and production.
To test C code you need to compile it, and there’s no C compiler inside Python. Also if you have C++, sometimes you can abuse templates for similar effects, and Python doesn’t have C++ AST either.
C toolchains largely depend on environment: included & linked files, environment variables, compiler & linker switches, they all make a huge difference. You’ll spend much time replicating that in Python, and it’ll break when you’ll upgrade any part of toolchain (compiler/IDE/build automation/etc).
To a lesser extent, same applies to native binaries as well. If you’ll manage to use Python to produce tests, C compiler to build them, then Python to run them — the runtime environment will be different from the real one.
I think nickpsecurity has introduced two ideas, and you are only countering the more complex one. The ideas were,
* Use python as a preprocessor
* Use python to call into C.
To the first idea: the language you choose to use as your preprocessor. Instead of using C macros, you could have an alternate DSL that you transform into C. Then you compile this. Given what an awkward thing the C preprocessor is, I am surprised that it continues to hold mindshare against options like this. Awk is a better powerful transform tool, and easily compiled for any platform it is not already on.
Line numbers is a complication if you do your own preprocessor. This post on PHP suggests how to deal with this, https://stackoverflow.com/questions/396644/replacements-for-...
To the other point. The complications you raise are real, but you can manage them away by setting your C project up to create a shared-object build. For example, it is trivial to use Racket to write tests against shared object files (once you know Racket). This still doesn't address the runtime issue you raise.
Separate issue. There is a set of debugging C-preprocessor macros with similar intent to the ones posted in OP published in "Learn C the Hard Way".
It’ll become harder to find developers, and longer for new ones to start being productive. Also it’ll become harder to do cross platform development because you’ll have to port, and then support, that custom pre-processing tools/DSL for every platform.
> Use python to call into C
Last time I’ve checked Python can’t import C headers. To call into C, you somehow need to negotiate function prototypes & calling conventions between the two languages. Regardless on how you do it manually or automatically, it’s very expensive to support. People do it when the have to (e.g. to interop Python with C libraries), but for tests, the approach is way too expensive for most project.
> it is trivial to use Racket to write tests against shared object files (once you know Racket)
I don’t know Racket but I doubt it’s trivial. SO libraries export C functions, they use C structures for arguments and return values, and these can be very complex. Pointers graph, function pointers, pointers to pointers, variable length structures, other advanced features are pain to use from any other language (besides languages specifically designed to be backward compatible, like C++, objective C, and maybe D). And if you need to test C++ code it’s even harder, unlike C it doesn’t have standardized ABI, i.e. the ABI changes between compilers and their versions.
Agree regarding C++ ABI also. I don't have much experience with C++. My usual approach is to use C for system calls and bare-minimum surrounds, and then use the wrapping approaches described above to get to a high-level language. This is why the struct issue didn't come to mind for me.
* Use python to call into C."
You nailed it. Also note that Python isn't my recommendation so much as what mort brought up that I'm countering with examples showing even it can help. If it was my choice, I'd pick a better HLL for these goals. Let's keep looking at this a bit, though, where I'll introduce those where they're useful.
PHP was a great example I didn't think of on preprocessor. It versus C's is either the 1st or 2nd most used one out there. On the high end, some "term-rewriting languages" can easily handle jobs like refactoring: TXL, Rascal, OMeta, Ohm. Alan Kay et al in STEPS project did a whole OS with graphics stack and all in a mere tens of thousands of lines of code using such a language. One can, as people do with Prolog and STEPS did, pick a general language that can be extended with meta facilities for DSL's or rewriting where opportunistic. Then, where that doesn't work out, you at least have general-purpose language to fall back on.
(Use Control-F to go to anything that says "STEPS" to find the reports. Work from the bottom up since it's chronological series.)
On your other point, your Racket example is one way it could work. I was also advocating in this thread just using Python itself to build tooling for its own benefits and ecosystem. A lot is already built. Use C for low-level software that strictly needs it if nothing else is available with HLL's like Python for stuff that doesn't need it. The language I've been telling C programmers to check out for scripting-like use is Nim. It looks like Python, has real macros, can call C, and compiles to C. Here's a comparison I just dug up:
https://github.com/nim-lang/Nim/wiki/Nim-for-C-programmers
I also doubt it will be harder to find C developers if tooling is written in HLL's. For one, they just have to use the tooling rather than write it. I doubt most C# developers extend Visual Studio, most Java developers extend NetBeans, and so on. If they do have to learn, using a language like Python or Nim should make low barrier to entry since even folks with no experience pick Python up quickly. A C-like subset of Nim or something similar will be just a lot like C with easier syntax and less stuff to worry about. If anything, productivity will go up over C like shown in about every language study ever done.
I have encouraged people to do C-like languages with safe defaults, cleaner semantics, a REPL, better macros, easy integration of C, and outputting C. That way, one can program in a cleaner language avoiding most headaches and accidents that come with C being designed for 1970's hardware. Aside from Wirth's languages with C syntax, a good picture of what this might look like is Julia language which was femtolisp on the inside. Especially its seemless interoperation with C and Python.
I think this is because writing python code is like writing C++ without templates, but all the integer types declared “auto” and all the pointers void.
The main difference is that the C++ compiler still* does more type checking than python even after you abuse the warning flags enough to get it to stop gently telling you to change professions!)
Reading it reminds me of when I accidentally got my emacs syntax highlighting stuck at “white on white” for a bunch of stuff, but I digress.
I guess I find it to be a completely unproductive, unreadable, and unmaintainable language. It’s OK for throw away prototypes, but I’ve never seen a team throw away a python prototype and rewrite it in another language. I know of python projects that failed and were [repeatedly] rewritten by the same team in python, and I know of python teams that failed and got swapped out for another language, but that doesn’t count. Part of the problem with the python prototyping story is that python wants to own the event loop and asynchrony / thread model, and changing those are often the main reason to swap languages — the next step is always a rewrite in my experience.
Wow. Long rant. One too many ‘assert “” != None’ errors this week. :-)
Regarding the event loop, I don't think that's true; you can embed libpython and call functions from your own loop: https://docs.python.org/3/extending/embedding.html
Its basic operations don't often crash a system or lead to hacks. It also promotes readable, concise code with a lot of FOSS utilities for things like testing (esp see Hypothesis). This both increases development pace and reduces defect according to about every study that's ever pitted a HLL against C. The best one I've seen in terms of apples to apples was this Ada and C comparison:
http://archive.adaic.com/intro/ada-vs-c/cada_art.pdf
Note that Python isn't the main point or even what I brought up, though. mort's point read like you shouldn't develop C-related tooling in high-level languages like Python. I'm countering saying you should do as much work out of C as you can with even Python being beneficial. I give much better examples in my other response here:
I have been working on Snow, a unit testing library for C.
These days I mostly use C, and occasionally resort to python as an accompanying scripting language to generate data definitions (either in plain text formats or as C structures).
Macro metaprogramming is still needed in C++. Other powerful languages might lack some nice properties of C. Like easy interfacing with other languages, and a wide availability.