Libmill: Go-Style Concurrency in C | Hacker News Reader

Libmill: Go-Style Concurrency in C

libmill.org

233 pointsCieplak76 comments

The macro should be called select, since that's what Go calls it. (Of course I know there is a POSIX function (not C) by that name; so what).

This "choose {" "end }" business is an awful way to design the macro. If you need nesting that involves a block, with prolog and epilog material on either or both ends, you just have to hide that into the macro:

  selbegin;

    out(ch, int, 42):
      foo();
    in(ch, int, i):
      bar(i);
    otherwise:
      baz();
    
  selend;

I don't understand the downvotes. I have 30 years of (continous!) C experience. This is the best practice for doing a macro like this.

This "end }" business has improper nesting, visually. The choose is outside of the braces, the end is within; it is scatter-brained. There is no error checking for a forgotten end.

I'm looking at the implementation and it's looking quite weird! choose simply expands into mill_choose_init__, and end expands into mill_choose_end__. The definitions of these seem as if they should pair together:

  #define mill_choose_init__ \
      {\
          mill_choose_init_(MILL_HERE_);\
          int mill_idx = -2;\
          while(1) {\
              if(mill_idx != -2) {\
                  if(0)


  #define mill_choose_end__ \
                      break;\
                  }\
              }\
              mill_idx = mill_choose_wait_();\
          }

But, oops, look at the obvious lack of balance in the braces.

   mill_choose_init__ {

   mill_choose_end__ }

we are putting a brace after the if (0) which balances the one after break, and the final missing brace that balances the one opened internally in mill_choose_init__.

There are best practices to design this sort of macrology that don't require the user put braces in weird places to balance the inconsistencies in macro expansions.

Speaking as a professional, I would not pass this in a code review.

That we have to use a flaky preprocessor in order to get a half decent syntax is already a significant regret. We must not compound the regret by allowing the macrology to be less than as good as it can be.

As someone unfamiliar with this library or concurrency in C for that matter, I do not understand your comment. The code examples on libmill.org look simple enough that I'd want to use it. What is it that you think is not right about it ?

I think their ‘should’ is just an idle opinion, not a technical comment as in saying it’s broken.

It is an opinion, and it was obviously written in spare moments that can be identified as idle. But it is far from an idle opinion. I consider that a personal attack.

Furthermore, my opinion here is, in fact a technical comment. Hidden open braces inside a macro expansion that are explicitly closed by ones written by the user, and vice versa, is actually a technical issue, not simply aesthetics. The two are not entirely separated.

Why ‘should’ it be written as ‘select’? How is the current name incorrect? It’s just an opinion.

The downvotes are probably because you’re expressing a naming bikeshed issue like your opinion is undeniably correct.

‘I would call the function ‘select’’ would be a reasonable comment.

It doesn't match the Go syntax that it's trying to quite faithfully imitate. This is for a psychological reason, which I'm dispeling as a needless concern.

It's fairly obvious to me that the author wanted that identifier; I'm just saying that the reasons against changing the name are weak.

I don’t know what you think ‘should’ means but it’s not ‘I think this way may be a better trade-off.’

And the Go syntax doesn't match the Alef syntax that it's trying to emulate. Go is clearly wrong here, and the correct name is 'alt'.

"Not breaking everything on posix systems when you include standard headers" isn't a psychological reason.

It's pretty ludicrous to alias select since it's the sort of thing you're likely to be calling in a concurrent program.

It's not really the sort of thing you'd use in a modern concurrent program. There is no reason to use select on a system where you have poll. As of 2001, POSIX moved select into its own header, so you shouldn't be getting a declaration of it any more from <sys/time.h>.

libmill provides some wrapping for polling mechanisms; you'd probably be going through that in libmill-based sources.

The clash, if it happens, has very easy, localized workarounds.

Note that the library has a macro called end, which is a very common identifier in programs. Why didn't the author consider that a problem, but switched Go's select to choose?

select is super well known in C, as I'm sure you're aware. I have over 20 years in it myself.

sure, workarounds are easy, but that's not the point. your suggestion that it should be named the same as Go is a bit ridiculous. why is that even important? what is important is potentially confusing developers, and adding a new "select" into C would do just that.

the only mistake is Go using the word select to begin with.

select isn't in C; it's in POSIX.

I worked in Windows development shops where nobody knew select; but they could write WaitForMultipleObjects code in their sleep.

A strictly conforming ISO C program can use the select identifier however it wants, including as an external name for an object or function.

> your suggestion that it should be named the same as Go is a bit ridiculous

It's not any more ridiculous than wanting to have a library which imitates Go concurrency down to macros that imitate some of the syntax.

>I worked in Windows development shops where nobody knew select; but they could write WaitForMultipleObjects code in their sleep.

That's hard to believe considering the fact that POSIX-compatible select() is a part of the winsock API.

> your suggestion that it should be named the same as Go is a bit ridiculous.

Are you aware that you're making that complaint in a discussion on Libmill: Go-Style Concurrency in C?

It's one thing to skip the submission to jump to the comment section. It's an entirely different thing to start mindlessly criticizing constructive criticism presented by fellow users while being entirely oblivious to both the subject and the context. That's just noise, and contributes zero to the discussion. In fact, it takes an awful lot from it.

> Are you aware that you're making that complaint in a discussion on Libmill: Go-Style Concurrency in C?

Just because it's Go-style, doesn't mean "Exactly like Go, down to the keywords used".

> Just because it's Go-style, doesn't mean "Exactly like Go, down to the keywords used".

That point was already discussed previously in the discussion, and it's pointless to continue repeating what has already been said, particularly when those arguments boil down to nothing more than a personal opinion.

I read the entire discussion and didn't see a single person mention it, but ok.

While I agree it's poor form to reuse the word 'select' for obvious reasons, there are few circumstances where it makes sense to use the select() interface in any new programs.

It's a very common source of hidden bugs thanks to FD_SETSIZE.

It also performs badly due to to all the bitmask twiddling, and copying of those bitmasks to the kernel and back.

If you want to select on file descriptor 1000, the bitmask has to include zeros for descriptors 0 through 999. Those get copied to the kernel and iterated over dutifully.

I have a hard time wrapping my head aroun dthe fact that you think the brace placement is a problem while entertaining the suggestion that calling a macro "select" with the obvious complete breakage under unix would be fine.

One is a minor esthetic problem, the other would be a nightmare to support and work around.

> suggestion that calling a macro "select" with the obvious complete breakage under unix would be fine.

There are lots of API's in the world; you can't ponder about all of them when naming a macro. How do you know the next system vendor doesn't have a choose function? Or if not the next system vendor, then some third party library or module you'd like to use or whatever.

POSIX doesn't reserve the select identifier for use as a macro, unless the header file is included which defines it. It is not "complete breakage", and there are workarounds in that situation, one possibility being:

  #undef select
  #define whatever mill_choose_init___

The breakage only happens for code that uses either the select macro or the select function, if the headers are included in the proper order: the system headers first, then libmill.h. I.e. this by itself doesn't cause a problem (except in the improble case that <sys/select.h> defines a select macro):

  #include <sys/select.h>
  // #undef select   at worst!
  #include "libmill.h" // with #define select macro

This does cause a problem, 100%:

  #include "libmill.h" // with #define select macro
  #include <sys/select.h>

But you should never do that: do not include local headers before system or third-party headers, unless you're sure they don't define any macros.

There is also no problem if the rest of the file uses the select macro. The only issue arises if the rest of the file wants to use the select function, and then we put in a workaround. You're not going to call the select function in hundreds of places, where you also need the libmill header.

TL;DR: it's not anywhere near a "nightmare".

Moreover, libmill defines a few macro with short names. Conceivably #define end could be a problem, as well as #define in(..) and #define out(...).

For instance if we have a "range.h" which has this:

   struct range {
     int beg;
     int end;
   };

and we do this:

   #include "libmill.h"
   #include "range.h"

then libmill's end macro will screw up the struct range declaration, not to mention any place where we use end as an identifier. It's not an uncommon local variable name, like for pointers pointing to the end of a memory range.

The stdlib namespace is golden. The POSIX namespace is silver. Only the foolhardy tread on either. One might even want to use libmill in conjunction with some socket code to build a non-blocking state machine using select().

I would say that such work would be a candidate for rolling into libmill itself, so the application wouldn't deal with select at all.

Note that the code already has some multiplexing gadgetry there, switchable between Linux's epoll, BSD's kqueue and the portable poll (see epoll.inc, kqueue.inc and poll.inc).

"Speaking as a professional", if you wrote something like this in my codebase with this justification, I'd fire you before any more carnage could be done.

Holy shit.

Standard practice in C is to prefix your functions with their 'namespace' to avoid collisions. In this case, I don't even like that they used "choose" instead of "mill_choose" or "mill_select".

But redefining select() is a whole other level of inconsiderate.

I'm working within the premise of this library, which is to provides un-hygienic syntactic extensions to C via preprocessor macros, which are presumably supposed to look reasonably good in comparison to Go.

Did you notice the #define end, which is a common name for struct members and local variables. Redefining select is certainly not a whole other level of inconsiderate in that light.

The caveat is that whoever uses this should include it last, after any other headers, and then deal with the macro clashes: don't use "choose" or "end" in their code and so on.

One way to mitigate this is to have these macros in a separate header, say "libmill-macros.h", so that the rest of the API is available without them.

You’re suggesting creating incredibly fragile code that will cause a lot of cognitive overload which is completely unnecessary in the first place.

Don’t do this, ever. Just because it’s technically possible doesn’t mean it’s a good idea.

I have no idea what you're talking about; we're discussing small, incremental changes in something whose design I have nothing to do with otherwise.

they are suggesting improvements. The code as a whole is problematic due to a number of reasons and they are offering improvements. If they had prefaced everything with add the namespace to the definitions, I doubt most people would have had as much of a problem with it. From what it seems, they didn't though because the developers don't seem to be worried about library namespace collisions.

> You’re suggesting creating incredibly fragile code

You're making ill-informed accusations.

Please check libmill and pay attention to how it has been implemented, particularly the design choices covered in this thread, and afterwards re-read OPs comments.

> Did you notice the #define end, which is a common name for struct members and local variables. Redefining select is certainly not a whole other level of inconsiderate in that light.

Local shadowing is completely different to global shadowing.

The PP has made numerous valid justifications for why there either isn't a conflict or an easily mitigated conflict. Here's another one: use separate compilation units. Include select(2) in one and include libmill in another; then wrap their interfaces and link them the way you want.

I don't know what's happened to HN these days. The sheer number of kids who have come out of the woodwork to flame this guy in different ways on different threads -- all with the same level of ignorance on a really simple topic -- is nothing short of deflating.

Undefining select, relying to the import order of files, are all things that are dubious at best, and create fragile code.

To claim everyone who doesn't share your opinion are kids that just came out of the woodwork and are ignorant is not helping the discussion either.

Grandparent made a controversial suggestion, and the discussion shows it.

Undefining the macro definition of a standard function is an approach described in the very ISO C standard itself.

I have the 1999 version of the standard in PDF form handy. Let's see,

7.1.4 Use of library functions

1 [... snip ...] Any macro definition of a function can be suppressed locally by enclosing the name of the function in parentheses, because the name is then not followed by the left parenthesis that indicates expansion of a macro function name. For the same syntactic reason, it is permitted to take the address of a library function even if it is also defined as a macro. The use of #undef to remove any macro definition will also ensure that an actual function is referred to.

Nothing dubious about using #undef to remove a macro definition of a library function.

There probably isn't a select macro; the #undef is just in case there is one, so that when we #define our own, we don't get a diagnostic about a redefinition with a different token replacement sequence.

The entire premise of "complete breakage" and "nightmare" is unrealistic. No competently architected project universally includes select(2). There is no reason to have random invocations of the call all over the place. Are you prepared to setup the timing, signals, the fully elaborated errno switch handling every error, including finding a bad file descriptor on EBADF to toss for a reinvocation, or how to continue after EINTR all to make this expensive (and de facto deprecated) system call which generally forms a single blocking point deep in a program's core event loop willy nilly like it's some kind of strlcpy()?

No, you're not. Frankly I've never seen select(2) properly invoked in more than one place in an entire project, and neither have you.

That's why a C professional isn't really phased. Sure, there are about a million english words in the OED and some more creativity would be fantastic. This still isn't a problem, on the facts. It's not an opinion.

Give me a break. OP led with a trivial and trivially bike-sheddable opinion on a name(clash), egged on by OP's low-effort question, "so what?" If you don't want to dance with trolls, don't flirt with trolls onto the dancefloor.

The point about the danger of mixing explicit brace opening with implicit macro brace closing is prudent, though.

> OP led with a trivial and trivially bike-sheddable opinion on a name(clash),

This discussion is on a library that advertises itself as offering "Go-style concurrency in C", and the OP presented a valid case about an aspect of the library that misses its whole design basis.

Quite obviously that's not bikeshedding, but valid criticism regarding the very basis of the whole project.

There are lots of APIs in the world, that's true. The one with "select" in it is a staple of nearly every C program and has existed for decades. I think it would be wise to name your stuff so as not to collide with it.

If someone someday makes a name that collides with yours, that's a problem for that "someday". Avoiding today's problem today seems pretty valuable.

Thanks, I am quite well informed in this sort of debate.

POSIX systems in fact go out of their way to support conforming ISO C programs, which are permitted to contain things like:

   void select(int x) // not in any ISO C reserved namespace!
   {
     selection = x;
   }

In the case of the GNU C library how it works is that select is a weak symbol aliasing for __select. If anything in the library needs to use select, it calls __select, so as not to be derailed by a user program that redefines the weak symbol.

Issues with macros are less severe compared to clashes in actual external linkage. No weak symbol support needed in shared libs; just something has to be #undef-ed here, or an order of headers rearranged there.

I agree that the “end }” syntax is bad. I disagree with your opinion that `choose` “should” be named `select`. To software written in C, the syntax or naming choices of other languages are really quite irrelevant. The code must be workable in C (and pertaining context, of which POSIX is a part), with minimum fuss and, as far as at all possible, without necessitating any “workarounds”, however minor you opine they would be.

It is not realistic to demand there be possibility of name clashes in the use of a library that uses non-hygienic macros with short names to simulate the syntax of another language.

> To software written in C, the syntax or naming choices of other languages are really quite irrelevant.

However, at least one person out there thinks that the syntax and naming choices of Go are relevant to software written in C, so they wrote this thing called "libmill".

Oh wait ... what were we talking about?

Experience doesn't always build upwards. Sometimes it's 30 years of learning to see the world a certain way that fits your view. Bad habits and unique domain expertise and everything in between.

Do you have evidence that my comment stems from years of developing a poor world view or entrenched bad coding habits?

You’ve judiciously used the length of your career to buttress various controversial claims throughout this thread. Don’t get me wrong; I love me a controversial opinion. Nothing quite like the sound of a crumbling institution. But your constant claim to authority is jarring, and to put the burden of proof on someone else here is a bit too much.

He merely questioned the ironclad value of your opinion as gospel “just because you have a lot of experience”. He didn’t even say it’s definitely wrong , but that it’s not necessarily always a good argument. That is completely fair. You can’t go and ask OP now to prove that you somehow hold a mistaken belief.

You want to make controversial claims? Great, please do. Love it. But you’ll have to back them up with more than just your say-so.

Experience accounts for nothing in iconoclasm. Thankfully.

(And I actually think the points you raise, themselves, are interesting, and great food for thought. For the record.)

I've carried all the burden of anything resembling proof in this entire thread, in the face of baseless claims that the world will fall apart if you have a #define select macro in a header file that simulates Go syntax!

I've exhaustively explored pretty much the entire space of this. You can whip out and environment with a compiler and try those things I wrote about.

There is literally nothing more to add.

I'm pretty sure it's possible to get it to look like "choose { cases... } end;" instead, which is what I did in a mostly joke library I wrote to see how far I could go with the C preprocessor:

  chan_select(2) {
  chan_case_send(c, &foo, 0):
      do_stuff();
  chan_case_recv(c, &bar, 1):
      do_other_stuff(bar);
  } chan_select_end;

But the implementation (https://github.com/iriri/libcee/blob/master/include/cee/chan...) is both horrifying and not as efficient as just doing the sane thing:

  chan_case cases[] = {
      chan_case(c, CHAN_SEND, &foo), /* unfortunately still a macro but it's basically just a struct literal */
      chan_case(c, CHAN_RECV, &bar),
  };
  switch (chan_alt(cases, 2)) {
  case 0:
      do_stuff();
      break;
  case 1:
      do_other_stuff(bar);
  }

Actually, I think you could pare it down to

  choose { cases... }

using https://www.chiark.greenend.org.uk/~sgtatham/mp/ techniques.

I'm not a fan of your suggested approach because automatic reformatting will remove the indentation, whether done in an editor or with a tool like clang-format, etc. To be fair, automatic reformatting does have its limitations, and there are some cases where it's best to just disable it, such as in the definitions of some macros. However, I prefer to avoid requiring users of my macros to turn it off, if possible.

In this case, I think you could make it work without any macro at the end, just a closing brace, without changing the rest of the syntax (including how the label-like constructs implicitly open a scope, which I think is a bad design but whatever). Something like this (for simplicity I've omitted backslashes and the concat calls needed to make label names unique):

  #define choose
    if (1)
      goto register_options;
    else
      while (1)
        if (1) {
          /* We get here once the registrations are done */
          void *target = mill_choose_wait_();
          goto *target;
        } else
          register_options:
          if (0) {

  #define in(ch, type, var)
    } else if (({goto register_N; did_register_N: 0;})) {
      type var;
      register_N:
      mill_choose_in_(ch, &var, &&picked_N); /* computed goto because why not */
      goto did_register_N;
      picked_N

>I have 30 years of (continous!) C experience.

In here, nobody knows (and more importantly nobody cares) you are a dog.

This is something that would be extremely useful on microcontrollers, where you do lots of event-driven programming. Any sane design ends up with a state machine (or a hierarchy of state machines) driven by events generated in interrupts.

Coming from Clojure, I wished for something like core.async on microcontrollers for a long time — a way to convert most of my state machine into sequential code, with the complexity hidden, all while keeping everything in C (converted/generated during compile).

> a way to convert most of my state machine into sequential code

Isn't that the norm for state machines in C?

    while (cont) {
      switch (v) {
      case S_STOP:
        cont = 0;
        break;
      ...
      }
    }

Or are you talking about inlining sequences of transitions into eachother? I've been avoiding core.async so I don't really know what it does that has to do with this.

With core.async (or go, for that matter), you can write code like:

* do one thing

* wait for the result of one thing (for example, an event from an interrupt)

* take the result and feed it into another thing

* wait for the result of another thing

So the code looks sequential. It's easy to write and easy to understand. But the entire 'go' block returns immediately: the whole thing gets converted into a state machine (which you don't normally see). That way you don't have to manage the complexity of the state machine yourself.

Note that this is not the same thing as threading. Also, it does not require threading at all, for example core.async works just as well in ClojureScript, even though Javascript has no threads.

You may be in for a treat if you haven't already come across protothreads [0].

They provide a form of stack-less, co-operative multi-threading and are aimed at microcontrollers.

I came across them by chance, and have used them on an ESP8266 – it's really made everything feel much more clear and far less confusing.

You do need to wrap your head around the trick, but that really shouldn't take long. You also need to put any state that you expect to be preserved between calls of the function into something more persistent – just like with state machines. The examples all use static variables, but I am happier using a struct pointer to maintain state.

[0] http://dunkels.com/adam/pt/

Have you considered C++ coroutines? Imho they struck a really nice tradeoff in making it a language feature and letting you swap out the underlying infrastructure.

C++ coroutines, being a feature of C++17/C++20, are not present in most if not all micro-controller toolchains, so I'm not sure if it would be relevant for the parent

What processors are you referring to? Everything ARM is well supported by LLVM/Clang which implements the coroutines TS.

AVR and PIC? Maybe things have improved since the last time I've explored compilers alternative to those supplied by manufacturers. But from what I understand avr-gcc is the standard for avr, and PIC is just targeted by the manufacturer's compilers.

If you are OK with stackful coroutines (that libmill provides) then an RTOS (like freertos) would mostly be the same thing. Context switching for those isn’t as expensive as for PC Threads and processes.

If you can’t afford the memory for a dedicated callstack per thread then there are the other event driven models that had been mentioned here. However one loses out one real-time capabilities and priority handling by using those compared to a preemptive RTOS.

Have you considered gnupth, a venerable, solid light-thread library ? I've never used it on a microcontroller and if you work from interrupts you might need to adapt its event loop, but that's still probably the first lib I would try, having successfully used it in large industrial projects in the past.

Actually, there were no release since 2006, when the last bug was fixed I guess :+)

https://www.gnu.org/software/pth/

There's was an episode of the This Week In Tech podcast from 2015 that interviewed LibMill's creator about its design concepts:

https://twit.tv/shows/floss-weekly/episodes/358

And HN discussion from 4 years ago: https://news.ycombinator.com/item?id=10585505

It seems like development stopped a few years ago. There were steady releases almost monthly for a few years and then it just stops in 2017. Did the author lose interest?

Are there any plans to support multicore? I noticed one of the examples addresses this by using fork(). Multicore is something goroutines > 1.5 support.

Isn't this something already provided by libtask? Libtask was written by Russ Cox.

Based on a very cursory check, it seems libtask doesn't do channels. Libmill does, in the style of Go. So libmill seems to be more a superset of libtask.

Go channels et al are actually a syntax sugar for a C library that later got subsumed and rewritten in Go, but can still be used - libthread

libtask does have channels[1].

libdill (also by the creator of libmill) has some features that libtask does not.

[edit]

Also one should note that libtask predates Go, and so the API definitely feels very different. Libmill seems to specifically have a goal of making the usage feel "go-like"

1: https://swtch.com/libtask/channel.c

I missed that, thanks. I did say it was a cursory check. I somehow overlooked it there at the top and bottom of the main libtask page.

It's amusing to note that the rewriting of a concurrent language as a C library has happened before in Bell Lab's history (by some of the same people who worked on Go and its precursors).

Specifically, the same thing we're talking about here already happened with Alef[0], a CSP-based language, which was scrapped in favor of a C library[1] (man 2 thread) in the third edition of the Plan 9 OS[2].

[0] http://doc.cat-v.org/plan_9/2nd_edition/papers/alef/

[1] http://man.cat-v.org/plan_9/2/thread

[2] http://doc.cat-v.org/plan_9/4th_edition/papers/release3

Is libmill safe with garbage collectors (like Boehm GC)?

How does this compare to boost:Fiber (other than the obvious "it's in C, not C++")?

I actually really like the syntax for how they're emulating sending and receiving from channels. I like it better than Go's way of doing this with the <- operator.

What would be nice would be a defer function for closing channels and things.

The out and in functions should be swapped to match the go example?

What is Go style in this context? Isn’t it CSP?

who is using libmill by the way? like other open source projects using this library?

Go is Go-style concurrency _almost_ in C.

How come neither libmill nor libdill sites have TLS. It's almost a chore to set up a site without it nowadays.

Reply on news.ycombinator.com