Show HN: C programming language extension: Cedro pre-processor | Hacker News Reader

Show HN: C programming language extension: Cedro pre-processor

sentido-labs.com

97 pointsAlbertoGP33 comments

>Looking for prior implementations of this idea I’ve found magma (2014), where it is called doto. It is a macro for the cmacro[0] pre-processor

Hey, that's me! I wrote cmacro. I'm glad someone has built a newer and better alternative (cmacro is a big mess inside and has some significant parser errors :P).

Related to the backstitch operator, I've wondered about implementing a transform to turn code like:

  type_t value = allocate();
  operate(value);
  operate(value);
  destroy(value);

Into:

  type_t value = allocate();
  type_t tmp1 = operate(value);
  type_t tmp2 = operate(tmp1);
  destroy(tmp2);

To implement something like linear types[1] in C, such that each instance of type_t must be used once and only once. But I imagine this would require too much integration with the type system and code analysis.

I wish binary inclusion was provided by more languages or build systems. For small files it's a very suitable alternative to the difficulties of packaging resources when distributing an application or library.

Great project! C is a very useful language and this type of source-to-source compilers are huge force multipliers.

[0]: https://github.com/eudoxia0/cmacro

[1]: https://en.wikipedia.org/wiki/Substructural_type_system#Line...

> I wish binary inclusion was provided by more languages or build systems. For small files it's a very suitable alternative to the difficulties of packaging resources when distributing an application or library.

When it occasionally comes up as a need, I use the bintosrc command available at https://github.com/jochenleidner/ltools

Nice to meet you!

Thanks for the nice words, my program is much more modest than cmacro :-)

Yes, I think that the transform you describe would require the processor to know about types, which is something I avoided here. I experimented early with writing a function that would track the types of things and enable smarter macros, but it became obvious that it was going to be a choice between writing a full compiler frontend and a hacky thing that worked only in trivial cases, so I discarded it.

The decision was to keep it as a naïve macro processor, just with a minimum understanding of C syntax, and leave out anything that requires taking into account the semantics. I have other macros that do things like expanding «obj[a..b]» to «(Slice){.start_p=&obj[a], .end_p=&obj[b] }» but they are not worth it, because to work smoothly they would need to know the type of «obj». If absolutely needed, I think that «_Generic» could be used here but it starts complicating things beyond the point of diminishing returns.

You can get quite far with a preprocessor that's completely unaware of the semantics of the code. Or that trusts the programmer to use the macros on constructions of the right type.

Even parsing C "correctly" is a huge problem, since the syntax is ambiguous and the ambiguity can only be resolved by keeping a symbol table that tracks what kind of thing a symbol is.

So there's a huge pragmatic force to keep a strict separation between a type-unaware preprocessor and a full-blown compiler frontend.

(I considered using clang to dump the C AST when building cmacro, but concluded it would be too unwieldy.)

Formally this is called a pipeline operator and can be used like this:

    type_t value = allocate();
    operate(value) |> operate |> destroy;

Functional languages like OCaml, F#, and Haskell have it built in.

You are right, and I list some of those in the “Related work“ section: https://sentido-labs.com/en/library/cedro/202106171400/#back...

I am a very weak C programmer, but one thing I find myself doing a fair bit is wrapping the intermediate data from sequences of operations like this into a single structure representing the whole task or unit of work or whatever. That structure then has a single 'valid' and/or 'error' field that each step checks to see if it should actually proceed, and if so, it will fill its field in the structure, and the next step will pick up from there.

That way you get to write something like:

    typedef struct cookie_batch {
      bool valid;
      bool baked;
      unsigned short ingredient_count;
      char **ingredients;
    } cookie_batch;

    .
    .
    .

    void bake(cookie_batch *batch) {
      if(!batch->valid) return;
      batch->baked = true;
    }

    .
    .
    .

    cookie_batch batch = { .valid = true };
    add_ingredients(&batch);
    bake(&batch);
    serve(&batch);
    cleanup(&batch);

So you don't need any complex error handling in client code.

Yes, that’s a very clean and robust approach. In the “Deferred resource release → Related work” section, one of the links is to a video from the ACCU 2021 conference by Luca Sas titled “Modern C and What We Can Learn From It” that explains modern C techniques, and your one is mentioned at 23′15″: https://youtu.be/QpAhX-gsHMs?t=23m15s

If you haven’t watched it, I can recommend it.

Not sure about the example semantics, but with MyDef (https://github.com/hzhou/MyDef), it is trivial --

    $call value_ops, val, operate, operate

    subcode: value_ops(v, @ops)
        type_t $(v) = allocate();
        $(for:op in $(ops))
            type_t tmp$(_i) = operate($(v));
            $(set-1:v=tmp$(_i))
        destroy($(v))

MyDef is a general purpose preprocessor.

EDIT: maybe this is more fit what was intended

    type_t var;
    $(for:op in allocate, operate, operate, destroy)
         var = $(op)(var);

This is the first public release, open source under the Apache 2.0 license.

From the linked document:

> Cedro is a C language extension that works as a pre-processor with four features:

> - The backstitch @ operator.

> - Deferred resource release.

> - Block macros.

> - Binary inclusion.

The source code archive and GitHub link can be found at: https://sentido-labs.com/en/library/

I have no intention of building this up to a new language: if you can switch to a new compiler, there are better alternatives. This is for cases where all you have is a C99 (or close to it) compiler or you need to produce plain C code for compilation on a different target machine which might only have C89.

This feels very niche. Typing the \ at the end of multi-line macros is something many editors help with (and forgotten \s are visible in syntax hilighting.) Binary inclusion is frequently just a small build script. Deferred resource release is already available as a GCC/clang extension, so it's only for less capable compilers as you say yourself.

That leaves the @ operator… which seems useful for specific cases (e.g. graphics rendering), but I can honestly say I don't really feel a need for that in the codebases I worked with.

But. Let me offer some suggestions for extensions! :)

An improved preprocessor would be massively useful if it had features like:

- enumerating struct members in preprocessor macros (something like:)

  #define MACRO2(typename, fieldname, arg1, arg2) ... arg1 arg2 ...
  #define MACRO(typename, arg1, arg2) something { #each-member(MACRO2, typename, arg1, arg2) }

  struct bla {
    int a, b;
  }

  MACRO(struct bla, x, y)
  /* => something { MACRO2(struct bla, a, x, y) MACRO2(struct bla, b, x, y) } */

- same, for function parameters

- and last but not least, macros consuming { } blocks, e.g.

  #define MACRO(...)#{name} something something #{name} something #{name}
  
  MACRO(1,2,3) {
    this block is consumed by the macro and placed in 2nd and 3rd #{name} (first one just signals the macro consumes a {} block
  }

...oh and, almost forgot, "extensible" macros:

  #define MACRO(...) abc

  #ifdef MACRO
  #extend MACRO(...) def MACRO(...) ghi
  #else
  #define MACRO(...) def stub ghi
  #endif

(also, macros that can define other macros. Like _Pragma() => #pragma, _Define() => #define)

+1 on the macro-time iteration. The hacks I found to do it are just too unwieldy to justify their use.

Yeah, everything I posted there is based on something I had to do at one point or another, and none of it was pretty. (Some ugly enough to have decided against it even though it was done and working.)

Pretty straightforward with m4.

Can't tell if this is a joke or not. Sure hope it is.

If it isn't — the number of people who understand m4 well enough to be able to maintain something like this can probably be counted on a few hands.

The struct enumeration thing is certainly useful for homogenous vectors being written as structs, but one of the issues is that it requires the preprocessor to have significantly more knowledge of the underlying language. Would it be a better idea to have something generate X Macros from struct defs instead?

Yeah, the enumerations require the preprocessor to understand the language and process the struct definitions... which may not even exist yet if they contain macros :D.

It's not an easy problem. Generating some kind of macro-usable list like you suggest is probably the way to go, essentially making the entire thing a 2-pass process. (That said, the C preprocessor doesn't have a concept of "lists", perhaps that's the thing to actually add here.)

> enumerating struct members in preprocessor macros (something like:)

I considered something along these lines: my main use case would be to produce string versions of enum values, like Rust’s «#[derive(Debug)]».

The problem is that, without type information, it would be too fragile, so I decided to leave it out.

Still, I’ll consider what you propose, maybe there is some form of purely syntactical macro that could cover these use cases.

> The problem is that, without type information, it would be too fragile, so I decided to leave it out.

typeof() is being considered for ISO C2x and has been around for ages in GCC/clang. _Generic is already in C11. With a struct member iterator, these 2 could/should be useful to branch out into type specific handling for each field.

Admittedly those are somewhat limited, particularly when working with typedefs, but that's a bit of a fundamental problem in C :(

Thanks for sharing.

While the original C preprocessor can be a pain (but is still useful to have), additional pre-processors can add language features without abandoning the C standard and one's tool chain. One issue is that other developers need to either learn the dialect accepted by the preprocessor or make sense of its output.

Here I think some good decisions have been made:

Pros:

+ safer code

+ shorter/more concise code

Cons:

- learning curve of the pre-processor

Objective C and C++ started out as pre-processors, of course... ;)

Impressive approach, you appear to have gone to great lengths to make the Cedro processor be plain C without any dependencies.

As an extra related work on the deferred resource release: Dlang uses scope guards https://tour.dlang.org/tour/en/gems/scope-guards

Nice modest improvements. I could see using this in my previous life as a C programmer of a large project. I am a bit surprised that the defer features uses the 'auto' keyword rather than introducing a new 'defer' keyword. Seems like 'defer' would be clearer.

But as I type this comment I realize that other syntax parsing tools like IDEs would be happier if the unprocessed code is still valid C.

> But as I type this comment I realize that other syntax parsing tools like IDEs would be happier if the unprocessed code is still valid C.

Yes, that’s the reason. The code is still close enough to standard C to work well with Emacs for instance. It would be a deal breaker for me to require a special editing mode.

I wonder if this also allows for grouping registers together as a macro, rather than having to allocate an array to track ports and registers like digitalWrite() in arduino.

e.g. Instead of writing...

```

ANSELBbits.ANSB3 = 0; // RB3 is digital (pinMode)

TRISBbits.TRISB3 = 0; // RB3 is output (pinMode)

LATBbits.LATB3 = 1; // RB3 is set to high (digitalWrite)

```

we would be able to write the macro like below, where `ANSEL[RB3] --> ANSELBbits.ANSB3`

```

pinMode(RB3, OUTPUT);

digitalWrite(RB3, HIGH);

```

I’m trying to see if this could be done without writing your own macro in C (put it inside «src/macros/», list it in «src/macros.h»), but I first need to understand it correctly.

Since «ANSELBbits.ANSB3 = 0» sets the write to digital, would it not belong in «digitalWrite()» instead?

Would it work if the generated code was like this?

    TRISBbits.TRISB3 = 0; // RB3 is output (pinMode)
    ANSELBbits.ANSB3 = 0; // RB3 is digital (digitalWrite)
    LATBbits.LATB3 = 1; // RB3 is set to high (digitalWrite)

EDIT: Assuming I got the meaning right, you can get quite close with standard C macros, but since I’m here demoing Cedro we could as well use it:

    #pragma Cedro 1.0

    enum { OUTPUT = 0, INPUT = 1 };

    #define { pinMode(REGISTER, DIRECTION)
    TRISBbits.TRIS##REGISTER = DIRECTION
    #define }

    enum { LOW = 0, HIGH = 1, DIGITAL = 0, ANALOG = 1 };

    #define { digitalWrite(REGISTER, LEVEL)
    ANSELBbits.ANS##REGISTER = DIGITAL;
    LATBbits.LAT##REGISTER = LEVEL
    #define }

    #define { analogWrite(REGISTER, LEVEL)
    ANSELBbits.ANS##REGISTER = ANALOG;
    LATBbits.LAT##REGISTER = LEVEL;
    #define }

    int main(void)
    {
      TRISBbits.TRISB3 = 0; // RB3 is output (pinMode)
      ANSELBbits.ANSB3 = 0; // RB3 is digital (digitalWrite)
      LATBbits.LATB3 = 1; // RB3 is set to high (digitalWrite)
    
      pinMode(B3, OUTPUT);
      digitalWrite(B3, HIGH);
    }

If you put that in a file («regtest.c») and run it through Cedro and the standard pre-processor, you get this:

    $ ../bin/cedrocc -E regtest.c
    # 1 "<stdin>"
    # 1 "<built-in>"
    # 1 "<command-line>"
    # 31 "<command-line>"
    # 1 "/usr/include/stdc-predef.h" 1 3 4
    # 32 "<command-line>" 2
    # 1 "<stdin>"
    # 3 "regtest.c"
    enum { OUTPUT = 0, INPUT = 0 };
    # 9 "regtest.c"
    enum { LOW = 0, HIGH = 1, DIGITAL = 0, ANALOG = 0 };
    # 16 "regtest.c"
    # 21 "regtest.c"
    int main(void)
    {
      TRISBbits.TRISB3 = 0;
      ANSELBbits.ANSB3 = 0;
      LATBbits.LATB3 = 1;
    
      TRISBbits.TRISB3 = OUTPUT;
      ANSELBbits.ANSB3 = DIGITAL; LATBbits.LATB3 = HIGH;
    }

The only part that uses Cedro is the block #defines for «pinMode()» and «digitalWrite()».

EDIT2: I suspect that what you really need is a «macro» that checks that the register has been last set to the correct direction (INPUT/OUTPUT) and mode (DIGITAL/ANALOG) before calling the «…Write()»/«…Read()» functions. For instance, if B3 was last set to ANALOG and you call «digitalWrite(B3, …)», it would insert «ANSELBbits.ANSB3 = DIGITAL», but not if it was last set to DIGITAL, and so on for all function variants: «digitalWrite()», «digitalRead()», «analogWrite()», «analogRead()».

You could write such a function in «src/macros/pic32.c», list it in «src/macros.h», and Cedro would run it.

The deferred resource cleanup is really the thing I wish the most was in C. I know that it's relatively un-"C-like", but having to make sure to clean up objects on failure is one of the more annoying things that make my code messy. I like the `defer` paper and proposal linked in this article.

Like the article mentioned, GCC, clang & icc all have the __cleanup__ attribute that can call a destructor for when a value moves out of scope. (And works pretty much how it would in C++). MVSC is really the only one missing it of the major compilers, but it doesn't implement half the actual standard anyways.

__cleanup__ might be a C extension, but it's pretty widely supported. If you really wanted to have it be a code block instead of a function, in plain C, then you could possibly also use clang or GCC's lambda-ish extensions.

Nice project! It makes me think how awesome it would have been if the C programming language had an actual community around it with package repositories and opinions and just actual gradual improvement of everything. Instead of the isolated projects each with their own build systems and styles, and the IRC channels with angry people telling noobs to read K&R.

Your comment reminded me of the early days when I learned C in the mid 1980s, and I complained about the lack of a string type (which I had been used to from Pascal, Modula-2 and even BASIC), people said one should have the freedom to implement them as 0-terminated strings or using length and a char array, or even in other ways.

To me that has always been a lame excuse for a poor standard library. You can always build your own, even if the standard library has a solution that works for 99% of cases - and that makes 99% of code more readable to 100% of people.

I love C (let's not get into a C++ rant here..) so this is just a comment about the culture, like the parent post. The fact that people who don't like to re-invent the wheel have not stuck with C, and maybe gone to join the Perl, Python, Java... communities made this issue worse over the years.

In theory, this could still be done now (e.g. a standard build system), but I think most "systems language" innovation action now firmly happens in the Rust camp (which may be a good thing, because the world needs more safe code).

To be fair, maybe it would have been a bit of a shit show if they standardised on a string format before the popularisation (or even invention) of UTF-16 (or UTF-8, whatever floats your boat).

UTF-16 makes only sense for backwards compatibility with something that was originally designed to use OCS-2. Annoyingly, it can be both big and little endian encoding.

UTF-8 saves space and is just as easy (and fast!) to process as UTF-16, as both are variable length encodings. UTF-8 is always unambiguous and self-synchronizing. The byte order stays always the same. The only complication is potentially multiple ways to encode a character, but that can be solved by just declaring all but the shortest encoding illegal.

Hopefully we'll only have UTF-8 in the long run, both externally and (by default) internally.

C string functions are mostly compatible with UTF-8, as long as individual non-7-bit characters are not processed or the string indexed, etc.

I have a slight urge to blame this lack of "community" to the institutionalized nature of C's standarization in the style of "Against SQL".[1] Although in this case the barrier is not between research and implementation, but between implementation and standarization.

  [1]: https://scattered-thoughts.net/writing/against-sql/#innovation-at-the-implementation-level-is-gated

(But hey, the defer proposal is here and might actually become a thing... right?)

Communities with opinions and build systems and style guides showed up, and weren't that hard to find.

https://www.gnu.org/prep/standards/html_node/index.html

Reply on news.ycombinator.com