Hacker News Reader

People love ceremony. All the source files are written in lists in cmake files. [...] I love writing C because I do none of that overhead. The files to compile are whatever is on disk.

I couldn't agree more. I can't stand build tools that don't take globs! I shouldn't need to list the same thing in multiple places just to make it work. Almost all the time, the files on disk should be the only list needed.

The reason why people write all the source files in a list in a cmake file is that it means that when one developer adds a new file, everyone else's build system automatically regenerates the next time they try to build the project. When you use globs (which cmake does support), it means that the CMakeLists.txt file doesn't get modified when adding a new file, so the cmake generated makefiles/ninjafiles/whatever have no way of knowing that the build was modified. Instead of having the build system regenerate and then the build go through, they get a linker error and have to manually run "cmake -B build" or whatever to fix it.

CMake can detect when new source files are added and rerun itself. You need to know how to spell that option when globbing though.

From the cmake documentation:

> The CONFIGURE_DEPENDS flag may not work reliably on all generators, or if a new generator is added in the future that cannot support it, projects using it will be stuck. Even if CONFIGURE_DEPENDS works reliably, there is still a cost to perform the check on every rebuild.

Somehow this isn't a problem with Cargo.toml, however. What's cmake missing?

> What's cmake missing?

Where to begin... Maybe let's start with "a build system".

(I'm serious, CMake isn't a build system and doesn't have one. It supports a few...)

What are the alternatives?

I wish I new any good ones. Using CMake myself.

Is this much of a problem if you just assume the mentality of having to run cmake every time you pull code?

CMake supports globs. It just has notes in the reference docs about why that can lead to slower build times or even incorrect builds (like undetected new files) in some cases.

> embrace code generators.

You also don't need to limit yourself to the C pre-processor. There are other dedicated pre-processing languages, like M4, and templating languages you can use.

Definitely don't limit yourself to the C preprocessor. That way lies a tarpit. M4 took several months off me, it can definitely be used to do things but maybe shouldn't be. A really bad play was generating C++ from cmake as a compromise that delighted noone.

Python is good at generating C source. I changed to mostly using lua five years ago as python climbed the complexity curve. Sometimes the right thing is C itself, e.g. laying out data based on sizes of various C types. Currently I'm trying to use xslt to emit syntax trees that pretty print to get the generated source in what is probably a dead end.

Sort of happy to see m4 mentioned, but for the uninitiated: m4 is the receiver of an awful lot of hate, because things can get unwieldy quickly. Go in with your eyes open.

I'm curious how your build process works with the codegen approach. Do you have the codegen run a pass everytime you build the project? Or is it a manual process?

I've always wanted to use Lua to generate boilerplate C code (like typesafe vectors) instead of relying on macros, so might actually start thinking in this direction going forward.

Not the person you asked, but this sort of thing is pretty easy with templating languages, such as Perl's Template::Toolkit[1] or Python's Jinja2[2].

The advantage of a system like that over a plain programming language (such as Lua you mentioned) is that there's less escaping and string.format(xxx) sorts of things. The output text is the primary thing and the templating language has its own special syntax to "activate" it when you need it, rather than the other way around.

That being said, Lua's square-bracket strings are pretty good for bulk text, too (:

[1] http://template-toolkit.org/about.html

[2] https://jinja.palletsprojects.com/en/3.1.x/templates/#synops...

Template-Toolkit seems like the right tool for the job. Thanks for sharing.

Instead of code generation as a special case, let it be the default. That kills most of the complexity.

src/ contains lots of code in lots of languages, all describing code generators of varying complexity. src/foo.c uses 'cp' as the code generator. src/bar.c.py uses 'python'.

gen/ contains the result of the ad hoc code generators. Same tree layout as src, one to one map from those files. All of that is C. Files that get #included and don't make sense standalone I tend to suffix .data, e.g. src/table.data.lua runs to create gen/table.data.

obj/ contains all the C compiled to object code or llvm bitcode, depending on the toolchain.

I originally did that with the idea that I could distribute gen/ as the source code for people who wanted to build the project without dealing with the complexity of the build system, strongly inspired by sqlite's amalgamation. Sqlite is built from a chaotic collection of TCL and C code generators but you don't see that as a user of sqlite.c.

It turns out that debugging the stuff under gen/ when things go wrong is really easy. Valgrind / gdb show you the boring code that the generators stamped out. So in practice I keep that separation because it makes it easier to trace through the system when it behaves unexpectedly.

Lua does that really well. Good multiline string literal support. A template string that you call gsub() on a few times then dump to stdout is very quick to put together. You've got string format for more complicated things, maybe using one of the string interpolation implementations found on their wiki.

In procedural C land, unit testing is nowhere near as relevant as it is in memory-safe OO lands. Assertions are far more relevant. You want to detect illegal program states in development as early as possible. In C11, you also have static_assert for things like "is this static lookup table as big as needed".

I get why assertions would be more important than in memory safe languages, but why are unit tests less important in C than in higher-level languages? Or you just mean relative to assertions? (which to me don't really influence each other in that way)

Because the coding style is different. The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager. Procedures are sensitive to the context where they are called. A procedure calling another procedure usually takes care to provide valid input data. Robust error-checking must be in place, or your program is unsafe.

A unit test wouldn't know the calling context of procedures deep in the call graph, and cannot detect unsafe but seemingly working code.

This is more of a culture surrounding the language than anything else. Modern C avoids global state, prefers directly specified context and even allocation is external to the function's logic.

And testing functions is easy.

This is as opposed to procedures, I.e. functions that operate on global state with fragile conext assumptions, which almost impossible to replicate in test environments.

> The basic unit of code is the procedure, not abstract entities like DateTimeCalculator or EmployeePayrollManager.

I agree, but in almost any non-trivial well-run C project the basic unit is going to be datatypes, opaque pointers, modules, etc.

In that case I generally throw in a `bool MyObjectType_test(void)` into the module so, in some sense, I'll have some basic sort of testing. Not as nice as the OO languages provide, but enough to give myself some confidence that changes don't introduce blatant bugs.

I work mostly with functional programming languages where the function is the basic unit of code, what you say is true but is also true for "abstract entities". Not all classes need robust error checking if you control how they are being called.

> The basic unit of code is the procedure

I think ultimately this is where we think differently. If you are okay testing a whole class in an OOP context and calling that a unit test, then a test that calls multiple procedures (because one of them creates the input for the main procedure I want to test, for example) can also be considered a unit test.

I also think because it's lower level what functions are built on tend to be long term stable. Function was tested at one point and then used in production for a couple years. It works and will continue to work as long as no one messes with it.

> In procedural C land, unit testing is nowhere near as relevant as it is in memory-safe OO lands.

I'm not sure why you'd say this. Testing has more to do with the developer than the language. The SQLite project is written in C and has 590 times more test code than source code [1].

[1] https://www.sqlite.org/testing.html

> I love writing C because I do none of that overhead. The files to compile are whatever is on disk. The tests to run are all the tests in the source, they aren't listed somewhere else as well. Any file called .lua.c is a lua program that writes the intended contents of .c to stdout. *.c.h will be compiled as C and then run to write a C header on stdout.

Do you have any sample projects that showcase both the tests and the code generation? I'd love to take a look.

The tests look much like https://github.com/JonChesterfield/EvilUnit/blob/master/evil.... That's from the self tests for the test framework. It's syntactically modelled on catch2. The name is because I built the thing out of preprocessor macros to run on freestanding c89.

The projects using code generation in that fashion are proprietary. There's a makefile at https://github.com/JonChesterfield/boring-makefile/blob/mast... set up to do the src/gen/obj codegen by default structure.

Just made the ad hoc xml codegen hackery public, some examples of tests written a couple of days ago at https://github.com/JonChesterfield/xml/blob/main/tools/intse...

Wild, I was just thinking about how I wanted to do reflection on C structs using LuaJIT FFI for serialization.

It's worth considering the other direction - write the struct definition in json or xml, or dicts in python source or whatever, and generate the C structs and the associated functions and data from that.

Go slightly further and write out function types like that and you get something equivalent to swig, where instead of parsing C and trying to emit wrappers for other languages from that framework, you have the data in native lua tables that you emit code from.