The lost language extensions of MetaWare's High C compiler (2023)
duriansoftware.com
duriansoftware.com
I am fortunate enough to own a copy of the High C/C++ Language Reference in English. (-:
* http://jdebp.uk./FGA/metaware-iterator-driven-for.html
* http://jdebp.uk./Proposals/metaware-iterator-driven-for.html
Another part of the High C/C++ Language Reference describes non-local labels and jumps to them. It doesn't go into great detail, but it does talk about stack unwinding, so expect something similar to how High C/C++ implemented throwing exceptions.
As Joe Groff said in the headlined post, MetaWare implemented it by turning the nested body of the for statement into an anonymous nested function, which is called back (through a "full function pointer") from the iterator function whenever there's a "yield()" in that latter.
So there's no "whenever the iterator function returns". It only returns when it has finished. The body of the for statement is called by and returns to the iterator function, which is in its turn called by and returns to the function that the for statement is in.
All of the "saving state to the stack" that happens is just the quite normal mechanics of function calling, with merely some special mechanics to pass around a pointer to the lexically outer function's activation record (which is why a "full function pointer" is not a plain "function pointer") as a hidden parameter so that the (anonymous) lexically inner function knows where the outer one's automatic storage duration variables are.
MetaWare also had non-local goto from within nested functions back out into lexically enclosing scopes, and since the for statement body is a nested function, it's just a question of employing that already available implementation mechanism (which in turn does the same sorts of things as throwing exceptions does, unwinding the stack through the iterator function) for break/continue/return (and of course goto) inside the for body.
1. underscores in literals:
int a = 1_234_567;
2. case ranges: case 5 .. case 6:
3. named arguments: void test(int a, int b);
void foo() { test(b:3, a:4); }
4. nested functions: int foo(int i) {
int plus(int a) { return a + i; }
return plus(3);
}
5. static nested functions: int foo(int i) {
static int plus(int a) { return a + i; }
return plus(3);
}
Error: `static` function `test.foo.plus` cannot access variable `i` in frame of function `test.foo`
6. a feature similar to generators https://dlang.org/spec/statement.html#foreach_over_struct_an...I occasionally look at statistics on the sources of bugs and security problems in released software. Array bounds overflows far and away are the top cause.
Why aren't people just sick of array overflows? In the latest C and C++ versions, all kinds of new features are trumpeted, but again no progress on array overflows.
I can confidently say that in the 2 decades of D in production use, the incidence of array overflows has dropped to essentially zero. (To trigger a runtime array overflow, you have to write @system code and throw a compiler switch.)
The solution for C I proposed is backwards compatible, and does not make existing code slower.
It would be the greatest feature added to C, singularly worth more than all the other stuff in C23.
Where can I read about it? The only way to make ptrs to array elements also safe that I can think of, is to replace them with triples: (base, element ptr, limit).
As for WG14, I have no hope, they ignored several proposals, and seem keen in having C being as safe as hand writing Assembly code, and even then, Assembly tends to be safer, as UB only happens when doing something the CPU did not expect, macro assemblers don't do clever optimizations.
maybe as the center of gravity moves from people writing game engines and kernels to people keeping legacy code running we will get more of a constituency for bounds checking
agreed about asm being safer
I almost always find myself struggling to name and namespace things correctly for long term durability. Almost all compiled languages get this wrong. They generally force you to consider this before you start writing code so you can explore the shape of the solution first.
I think lisp is the only language I've used where this wasn't a burden, but in reality, lisp then forces you to deeply ponder your data structures and access ideology first, so I didn't find it to be that rewarding in the long run.
I love that Go lets you bang simple "method like functions" straight onto the type. This solves the first layer of namespace problems. It does nothing for the second though, and in fact makes it worse, by applying "style guidelines" to the names of the containing types. I am constantly let down by this when writing Go code and I find it hard to write "good looking" code in the language which is all the more frustrating because this was what the guidelines were supposed to solve in the first place.
I really just want C and I want to namespace my functions that act on structs into the structs themselves. Then I can name stuff however I want and I don't have to prefix_every_single_function() just so the me and the assembler can fully agree on the unmangled symbol table name which I will almost certainly never care about in 99% of what I compile.
There's a real joy to the fast initial development and easy refactoring you can find in scripting languages. Too bad they all have wacky C interfaces and are slower than molasses.
--
You got any idea why people hate the concept of nested functions in C?
Why named named arguments test( a:4, b:3) instead of test(.a=4, b.=3);
Have any idea how you would handle first class types in C.
No, I was surprised to hear it.
> Why named named arguments test( a:4, b:3) instead of test(.a=4, b.=3);
Consistency with how static initializers work in D.
It breaks the model of c mapping to the hardware.
Functions in C are just labels you jump to. Assuming you want to capture values from the environment you're defining the function in, you have to implement some sort of function struct that is callable, and it probably won't work with any shared libraries on your system. Also it would be slightly slower, and C programmers tend to not be willing to make that tradeoff
Why do you consider the second one better than the first (a:4, b:3)?
struct foo f = {
.bar = 1,
.baz = 2,
};Also previously covered here on Hacker News: https://news.ycombinator.com/item?id=38938402
Is there a PDF copy of this somewhere?
It describes underscores in numbers, case ranges, named parameters, nested functions, and full function variables.
https://bitsavers.org/pdf/metaware/High_C_Language_Reference_Manual_1.2_Nov85.pdf
See Appendix A - 50 odd pages from the end of the file.[0] https://publications.csail.mit.edu/lcs/pubs/pdf/MIT-LCS-TR-2...
There's a link to the compiler manuals at https://winworldpc.com/product/metaware-high-c-cpp/33x
The C manual PDF file mentions a 2007 copyright.
You will also discover the rich history of systems programming languages, and how similar C and Go design's are in ignoring what was being done in other ecosystems, and past experiences.
Rather JIS X 0201 ( https://en.m.wikipedia.org.org/wiki/JIS_X_0201 ) was used, on which Shift-JIS is based.
Ada has:
- Labels in the form of: Call (Param_A => 1, Param_B => "Foo");
- Underscores can be used in numbers of any base: X : Integer := 1_000;
- Nested subprograms
- Range based tests
C rode the wave of UNIX's success, it is like someone getting into computers in 2024 and praising Javascript's greatness for Web development.
ESPOL in 1961 already had unsafe code blocks, almost a decade before C came to be.
I don't know enough about Japanese orthography or keming rules to be sure but it looks very much like they took a variable width font with both kanji and latin characters and then hard formatted it into fixed width cells?
Either way, it's nice that the code examples aren't in 8pt font like a lot of the books I have...
1. Standardized on two's complement.
2. Little endian.
3. We went from word based memory systems to line based ones.
4. RISC lost out to super scalar designs.ZETA-C, a C compiler specific to the Lisp Machine, was already fully fledged by 1987 or there about. Don't have notes on when ZETA-C came to be, but it was much earlier than that, e.g., some of the headers are dated 1984.
One cool thing about ZETA-C was you could embed Lisp code in between C code:
extern FILE *stdin, *stdout, *stderr;
#lisp
;; We don't want this file to "own" these
(zeta-c:zclib>initialize-file-pointer |stdin| 0)
(zeta-c:zclib>initialize-file-pointer |stdout| 1)
(zeta-c:zclib>initialize-file-pointer |stderr| 2)
#endlispSince it's apparently from Fujitsu, I could see it being the former, but if so, I'm impressed with the quality of the English in the printf statements and code comments from non-native English speakers.
IMHO what I would need with C is a powerful pre-processor like jinja2 and some symbol manipulation features too.
That is what happens to languages that leave their authors behind and embrace design by committee.
Standardizing libraries is one thing I really favor. But force fitting them into the language itself maybe like over-engineering.
Let me have an example. qsort() and bsearch() are powerful but the call/return overhead is really unbearable. A (much) more powerful pre-processor which is really part of the language (and maybe syntax-aware) could help in creating templates that would generate solid code for either function with the bare minimum overhead.
I am not saying like C++ templates, but like C++ templates.
I think these are good ideas.
- Underscores in numeric literals: I think it is a good idea and is also what I had wanted to do before, too. (It should be allowed in hexadecimal as well as decimal)
- Case ranges: GNU C has this feature, too.
- Named arguments: This is possible with GNU C, although it doesn't work without writing it to handle this (although you can use macros to allow it to work with existing functions). You can pass a structure, either directly to the function, or using a macro containing a ({ }) block which extracts the values from the structure and passes them to the function (the compiler will hopefully optimize out this block and just pass the values directly). You can then use the named initialization syntax (which also allows arguments without named), and GNU C also allows you to have duplicates in which case only one of them will work, which allows you to use macros to provide default values. (I have tested this and it works.)
- Nested functions: GNU C also has it, but does not have the "full function value" like this one does, and I think it might be helpful. Nonlocal exits can also be helpful. (I also think the GNU's nested functions could be improved, by allowing them to be declared as "static" and/or "register" in order to avoid the need of trampoline implementations, although "static" and "register" would both have their own additional restrictions; "static" can't access local variables and functions from the function it is contained in unless they are also declared as "static", and "register" means the address can't be taken (therefore allowing the compiler to pass the local variables as arguments to the nested function).)
- Generator functions: I like this too and I think that it is useful (I had wanted things like this before, too). It is also interesting how it can work well with the nested functions.
There are some other things that I also think should be added into a C compiler (in addition to existing GNU extensions), such as:
- Allowing structures to contain members declared as "static". This is a global value whose name is scoped to the strucure within the file being compiled (so, like anything else declared as static, the name is not exported), so any accesses will access the single shared value. Even in the case of e.g. (x->y) if y is a static member then x does not need to be dereferenced so it is OK if it is a null pointer.
- Scoped macros, which work after the preprocessor works. It may be scoped to a function, a {} block inside of a function, a file, a structure, etc. The macro is only expanded where that name is in scope, and not in contexts where a new name is expected (e.g. the name of a variable or argument being declared) (in this case the macro is no longer in scope).
- Allow defining aliases. The name being aliased can be any sequence of bytes (that is valid as a name on the target computer), even if it is not otherwise valid in C (e.g. due to being a reserved word). Any static declaration that does not declare the value may declare the alias.
- Compile-time execution (with explicit declaration).
- Custom output sections, which can be used or moved into standard sections in a portable way. These sections might not even be mapped, and may have assertions, alignment, overlapping, etc.
- Allow functions to be declared as "register". If a function is declared as "static register" (so that the name is not exported), then the compiler is allowed to change the calling convention to work better with the rest of the program.
// Declaration:
void plot(float xlo, float xhi, float ylo, float yhi, float xinc, float yinc);
struct plot_a { float xlo, xhi, ylo, yhi, xinc, yinc; };
static inline void plot_i(struct plot_a _a) {
// inline thunk to allow arguments to be passed in registers
plot(_a.xlo, _a.xhi, _a.ylo, _a.yho, _a.xinc, _a.yinc);
}
#define plot(...) (plot_i((struct plot_a){ __VA_ARGS__ }))
// Call:
plot(alo, ahi, blo*2.0, bhi*2.0, .yinc = y, .xinc = f(x+z)); plot((myfoo){x,y})
Macros will take the struct literals as multiple parameters: plot(
.arg0=(myfoo){x,
.arg1=y}
)
C macros are best left unused when possible. $ cpp -P
void plot(float xlo, float xhi, float ylo, float yhi, float xinc, float yinc);
struct plot_a { float xlo, xhi, ylo, yhi, xinc, yinc; };
static inline void plot_i(struct plot_a _a) {
// inline thunk to allow arguments to go in registers
plot(_a.xlo, _a.xhi, _a.ylo, _a.yho, _a.xinc, _a.yinc);
}
#define plot(...) (plot_i((struct plot_a){ __VA_ARGS__ }))
plot((myfoo){x,y})
plot(.yinc=(myfoo){x,y})
^D
[...]
(plot_i((struct plot_a){ (myfoo){x,y} }))
(plot_i((struct plot_a){ .yinc=(myfoo){x,y} }))
You could argue this is excessively clever, but when you need it, you really need it, so it could deserve known idiom status in the right situation.I've probably written millions of lines of C so far, and I don't think I have ever needed it.
There could be reasons to use one of those still (e.g. extensibility while keeping a compatible ABI, as in setsockopt, pthread_attr_*, or arguably posix_spawnattr_*). But sometimes you really do need a finite, well-known but just plain large number of parameters that mostly have reasonable defaults. Old-style 2D APIs to draw and/or stroke a shape (or even just a rectangle) are the classic example. Plotting libraries (in all languages) are also prone to this. It does seem like these situations are mostly endemic to specific application areas—graphics of all kinds first of all—but that doesn’t make them not exist.
If you don’t want to use function-like macros for anything ever even if this particular one works, that’s a valid position. But it does work, it does solve a real problem, and it is less awkward at the use site than the alternatives.
Blame the committee for failing to specify an obvious and widely-demanded feature like named parameters.
The only explanation is that the people in charge of the language don't write much code.
Have a link to any articles/sites/books which has a compendium of more tricks ?
Pascal lets you match a range of values with case low..high; wouldn't it be great if C had that feature? High C does, another feature standard C and C++ never adopted.
https://gcc.gnu.org/onlinedocs/gcc/Case-Ranges.html Nested functions
https://gcc.gnu.org/onlinedocs/gcc/Nested-Functions.html Generators
GCC doesn't do those --- looks like a fun feature though!My favourite, which was sadly removed was doing:
foo ? zork : bork = 123;
Oh well...The headlined article doesn't mention it, but High C/C++ had modules all of those years ago, too. Anybase literals, as well. Tom Pennello participated in the standardization efforts back then, too, but none of this stuff made it in.
[1] https://bitsavers.computerhistory.org/pdf/metaware/High_C_La...
(-:
*(foo ? &zork : &bork) = 123; > foo ? zork : bork = 123;
That's kind of horrifying.