The provenance memory model for C
gustedt.wordpress.com
gustedt.wordpress.com
https://clang.llvm.org/docs/TypeSanitizer.html
https://www.phoronix.com/news/LLVM-Merge-TySan-Type-Sanitize...
> the functions `recip` and `recip⁺` and not equivalent
Several paragraphs after this got swallowed by the code block.
Edit: Oh, I didn't realize the article is by the author of the book, Modern C. I've seen it recommended in many places.
> The C23 edition of Modern C is now available for free download from https://hal.inria.fr/hal-02383654
The C23 edition of Modern C - https://news.ycombinator.com/item?id=41850017
Like this comment:
> Wow, the use of attributes like [[__unsequenced__]], [[maybe_unused]] and [[noreturn]] throughout the book is really awful. It seems pretty pedantic of the author to litter all the code examples with something that is mostly optional.
Or this one:
> Personally this just makes C much more complicated for me, and I choose C when I want simplicity. If I want complicated, I would just pick C++ which I typically would never want.
Examples of what people consider "bloat" in newer C standards:
_BitInt(N), guard, defer, auto, constexpr, nullptr
_generic, typeof, restrict, syntax based tlsEdit: oh, you actually did quote me, too: https://news.ycombinator.com/item?id=41854897
In any case, thank you.
This seems to have been fixed now.
From the PVI section onward it seems to recover, but if the author sees this please fix and re-convert your post.
[Edited, nope, there are more errors further in the text, this needed proper proofreading before it was posted, I can somewhat struggle through because I already know this topic but if this was intended to introduce newcomers it's probably very confusing]
To convert it to C syntax, it's a function with roughly this signature:
void* with_addr(void* ptr, uintptr_t addr)
https://github.com/protocolbuffers/protobuf/blob/ae0129fcd01...
An identifier is an arbitrarily long sequence of digits, underscores, lowercase and uppercase Latin letters, and Unicode characters specified using \u and \U escape notation(since C99), of class XID_Continue(since C23). A valid identifier must begin with a non-digit character (Latin letter, underscore, or Unicode non-digit character(since C99)(until C23), or Unicode character of class XID_Start)(since C23)). Identifiers are case-sensitive (lowercase and uppercase letters are distinct). Every identifier must conform to Normalization Form C.(since C23)
In practice depends on the compiler.
Definitely a questionable choice to throw off readers with unicode weirdness in the very first code example.
And yeah, the chinese tone in practice does not align with the idea of "down a little up a lot" either. It depends on context...
I still don't want anything as unpredictable as Unicode in my code. How many different encodings will display as the same variable name and how is the compiler supposed to decide?
If you're thinking of comments and user facing strings, the OP already excluded those.
The answer to your question seems to (still) be "no".
https://godbolt.org/z/qKejzc1Kb
Whereas clang loudly complains,
But not for char-typed accesses. And even for larger types, I think you would have to worry about the combo of first memcpying from pointer-typed memory to integer-typed memory, then loading the integer. If you eliminate dead integer loads, then you would have to not eliminate the memcpy.
Or from a pragmatic viewpoint, perhaps if the optimiser eliminates a dead load, then don't mark the pointer as exposed? After all, the whole point is to keep track of whether a synthesised pointer might potentially refer to the exposed pointer's storage. There's zero danger of that happening if the integer load never actually occurs.
However, if all of the program paths from that point onward behave the same as if the pointer was marked as exposed, that would be fine. It’s only “wrong” to track the incorrect abstract machine state when that would lead to a different behaviour in the abstract machine.
In that sense I suppose it’s no different from things like removing a variable initialisation if the variable is never used. That also has a side effect in the abstract machine, but it can still be optimised out if that abstract machine side effect is not observable.
CaSe Sensitivity
Weird pointer syntax
Lack of a separate assignment token
Null terminated strings
Macros - the evil scourge of the universe
What does that mean?
True is true, and false is false, if you're wondering whether this Doodad is Wibbly, you should ask that question not rely on a convention that Wibbly Doodads are somehow "truthy" while the non-Wibbly ones are not.
> CaSe Sensitivity
Wait, what, you.. you want a case-insensitive language? Like SQL?
I love SQL, but please no more case-insensitive programming languages!
Or better yet, don't let 'social pressure' influence your choice of programming language ;)
If your workplace has a clear rule to not use memory-unsafe languages for production code that's a different matter of course. But nothing can stop you from writing C code as a hobby - C99 and later is a very enjoyable and fun language.
It’s hard. Programming is a social discipline, and the more people who work in a language, the more love it gets.
Of course it's still really nice to just have C itself being updated into something that's nicer to work with and easier to write safely, but Zig seems to be a decent other option.
Does Zig document the precise mechanics of noalias? Does it provide a mechanism for controllably exposing or not exposing provenance of a pointer? Does it specify the provenance ABA problem in atomics on compare-exchange somehow or is that undefined? Are there any plans to make allocation optimizations sound? (This is still a problem even in Rust land; you can write a program that is guaranteed to exhibit OOM according to the language spec, but LLVM outputs code that doesn't OOM.) Does it at least have a sanitizer like Miri to make sure UB (e.g. data races, type confusion, or aliasing problems) is absent?
If the answer to most of the above is "Zig doesn't care", why do people even consider it better than C?
In a sibling comment, I mentioned a proof of concept I did that if I had the time to complete/do correctly, it should give you near-rust-level checking on memory safety, plus automatically flags sites where you need to inspect the code. At the point where you are using MIRI, you're already bringing extra stuff into rust, so in practice zig + zig-clr could be the equivalent of the result of "what if you moved borrow checking from rustc into miri"
[0] type erasure, or using "known dangerous types, like c pointers, or non-slice multipointers".
As for null pointer problems, while they may result in CVEs, they’re a pretty minor security concern since they generally only result in denial of service.
Edit 2: Here's some data: In an analysis by Google, the "most frequently exploited" vulnerability types for zero-day exploitation were use-after-free, command injection, and XSS [3]. Since command injection and XSS are not memory-unsafety vulnerabilities, that implies that use-after-frees are significantly more frequently exploited than other types of memory unsafety.
Edit: Zig previously had a GeneralPurposeAllocator that prevented use-after-frees of heap allocations by never reusing addresses. But apparently, four months ago [1], GeneralPurposeAllocator was renamed to DebugAllocator and a comment was added saying that the safety features "require the allocator to be quite slow and wasteful". No explicit reasoning was given for this change, but it seems to me like a concession that applications need high performance generally shouldn't be using this type of allocator. In addition, it appears that use-after-free is not caught for stack allocations [2], or allocations from some other types of allocators.
Note that almost the entire purpose of Rust's borrow checker is to prevent use-after-free. And the rest of its purpose is to prevent other issues that Zig also doesn't protect against: tagged-union type confusion and data races.
[1] https://github.com/ziglang/zig/commit/cd99ab32294a3c22f09615...
[2] https://github.com/ziglang/zig/issues/3180.
[3] https://cloud.google.com/blog/topics/threat-intelligence/202...
Anyways, I am optimistic that UAF can be prevented by static analysis:
https://www.youtube.com/watch?v=ZY_Z-aGbYm8
Note since this sort of technique interfaces with the compiler, unless the dependency is in a .so file, it will detect UAF in dependencies too, whether or not the dependency chooses to run the static analysis as part of their software quality control.
On one side are the many C++ “static analyzers” like Coverity or clang-analyzer, which work with unannotated C++ code. On the other side is the “Safe C++” proposal (safecpp.org), which is supposed to achieve full safety, but at the cost of basically transplanting Rust’s type system into C++, requiring all functions to have lifetime annotations and disallow mutable aliasing, and replacing the entire standard library with a new one that follows those rules. Between those two extremes there have been tools like the C++ Core Guidelines Checker and Clang’s lifetimebound attribute, which require some level of annotations, and in turn provide some level of checking.
So far, none of these have been particularly successful in preventing memory safety vulnerabilities. Static analyzers are widely used in industry but only find a fraction of bugs. Safe C++ will probably be too unpopular to make it into the spec. The intermediate solutions have some fundamental issues (see [1], though it’s written by the author of Safe C++ and may be biased), and in practice haven’t really taken off.
But I admit that only the “static analyzer” side of the solution space has been extensively explored. The other projects are just experiments whose lack of adoption may be due to inertia as much as inherent lack of merit.
And Zig may be different… I’m not a Zig programmer, but I have the impression that compared to C++ it encourages fewer allocations and smaller codebases, both of which may make lifetime analysis more tractable. It’s also a much younger language whose audience is necessarily much more open to change.
So we’ll see. Good luck - I’d sure like to see more low-level languages offering memory safety.
Not just that, but the committee accepted a paper that basically says it's design is against C++'s design principles, so it's effectively dead forever.
Here's somebody who was in the room explaining how this was agreed as standing policy for the C++ programming language.
"It was literally the last paper. Seen at the last hour. Of a really long week. Most everyone was elsewhere in other working group meetings assuming no meaningful work was going to happen."
"I think this could be possible" isn't an enabling technology. If you write hard SF it's maybe useful to distinguish things which could happen from those which can't, but for practical purposes it only matters if you actually did it. Sean's proposed "Safe C++" did it, Zig, today, did not.
There are other obstacles - like adoption, as we saw for "Safe C++" - but they're predicated on having the technology at all, you cannot adopt technologies which don't exist, that's just make believe. Which I think is already the path WG21 has set out on.
Thanks! I think this could be implemented as a (3rd party?) compiler backend.
And yeah, if it gets done quickly enough (before 1.0?) it could get enough momentum that it gets accepted as "considered to be best practice".
Honestly, though, I think the big hurdle for C/C++ static analysis is that lots of dependencies get shipped around as .so's and once that happens it's sort of a black hole unless 1) the dependency's provider agrees to run the analysis or 2) you can easily shim to annotate what's going on in the library's headers. 2) is a pain in the ass, and begging for 1) can piss off the dependency's owner.
I will be very surprised if there's widespread adoption of Fil-C for many new projects though.
Literally no one, anywhere, wants to be forced to read source written in a language they can't read (or more specifically in this case: written in glyphs they can't even produce on their keyboard). That idea, for almost everyone, seems "horrific", yeah.
So a lingua franca is a firm requirement for modern software development outside of extremely specific environments (FSB malware authors probably don't care about anyone else reading their cyrillic variable names, etc...). Must it be ASCII-encoded English? No. But that's what the market has picked and most people seem happy enough with it.
This is blatantly false. I'd posit that a solid 90% of all source code written is done so by single, co-located teams (a substantial portion of which are teams of 1). That certainly fits the bill for most companies I've worked at.
However, everything else, from spreadsheet software to CAD tools to OS kernels to JavaScript frameworks is universal across cultures and languages. And for better or for worse (I'm not a native English speaker either), the world has gone with English for a lot of code commons.
And the thing with the examples in that post isn't about supporting language diversity, it's math symbols which are noone's native language. And you pretty much can't type them on any keyboard. Which really makes it a rather poor flex IMHO. Did the author reconfigure their keyboard layout for that specific math use case? It can't generically cover "all of math" either. Or did they copy&paste it around? That's just silly.
[…could some of the downvoters explain why they're downvoting?]
In heavily mathematical contexts, most of those assumptions get turned on their head. Anybody qualified to be modifying a model of electromagnetism is going to be intimately familiar with the language of the formulas: mu for permeability, epsilon for permittivity, etc. With that shared context,
1/(4*π*ε)*(q_electron * q_proton)/r^2 is going to be a lot easier to see, at a glance, as Coulombs law
compared to
1 / (4 * Math.Pi * permitivity_of_free_space)*(charge_electron * charge_proton)/distance_of_separation
Source code, like any other language built for humans, is meant to be read by humans. If those humans have a shared context, utilizing that shared context improves the quality and ease of that communication.
I guess maybe this is an argument for better UI/UX for symbolic input…
Because you made false assertions ("And you pretty much can't type them on any keyboard").
(Unless you're being pedantic because I wrote "keyboard" rather than "keyboard layout", or ignored the qualifying "pretty much". In either of those cases you're unwilling to communicate cooperatively and I can't help you.)
I don't need to do that because I actively use them myself and have a custom ~/.XCompose. Also, please try communicating less condescendingly.
There is no default compose sequence for ₚ that I can find, at least in my Debian installation.
So, again, please point me at the layout that can output these characters.
And even with that: if you don't think Compose sequences, possibly even custom, are covered by "pretty much impossible", I must seriously question your perception & bias of how common (or not) things are.
That's half tongue in cheek. I am fluent in three languages, but I program "in English" and I greatly appreciate that my colleagues who are fluent in languages other than the ones I'm fluent in (except English) also do. Basically English is the world's lingua franca today. Nonetheless if a company in France wants to use French for their symbol names, or a company in Mexico wants to use Spanish for their symbol names, or a company in China wants to use Chinese for their symbol names, who am I to stop them?! Surely it's not my place.
Source code is for humans, and thus should be written in whatever way makes it easiest to read, write, and understand for humans. If your language doesn't map onto ASCII, then Unicode support improves that goal. If your code is meant to directly implement some physics formula, then using the appropriate unicode characters might make it easier to read (and thus spot transcription errors, something I find far too often in physics simulations).
I'm a practitioner of neither though, so I can't condemn the practice wholeheartedly as an outsider, but it does make me groan.
This can be especially difficult if the author is trying to map 1:1 to a complex algorithm in a white paper that uses domain-standard mathematical notation.
The alternative is to break the "full formula" into simpler expression chunks, but then naming those partial expression results descriptively can be even more challenging.
Consider a simple programming example, in C blocks are delimited by `{}`, why not use `block_begin` and `block_end`? Because it's noisy, and it doesn't take much to internalize the meaning of braces.
It's probably also a great way to introduce almost undetectable security vulnerabilities by using Unicode characters that look similar to each other but in fact are different.
Yes, that would probably be one way to do it.
> Which would violate the whole "Code is meant to be easily read by humans" thing.
I'd think someone who's deliberately and sneakily introducing a security vulnerability would want it to be undetectable, rather than easily readable.
Such a silly issue too, you'd think we'd have come up with some automated wrangling for this, so that those experienced with a codebase can switch over and see super short versions of identifiers, while people new to it all will see the long stuff.
Maybe for code that was written in the early 90's, but the only 'tradition' that has survived is calling the vanilla loop variable 'i'.
My first thought before I saw this was “I wonder is this going to be an article from people who build things or something from “academics” that don’t.”
At least it was answered quickly.
> The functions recip and recip⁺ and not equivalent.
This is one of those examples of how optimizing code can improve legibility, robustness, or both.
The first implementation allows for side effects to change the outcome of the function. But the problem is that the code is not written expecting someone to modify the values in the middle of the loop. It's incorrect behavior, and you're paying a performance penalty for it to boot.
Functional Core code tends not to have this problem, in that we pass in a snapshot of data and it either gets an answer or an error.
I've seen too much code that checks 3 times if a user is either still logged in or has permission to do a task, and not one of them was set up to deal with one answer for the first call and a different one for any of the subsequent ones. They just go into undefined behavior.
https://www.godbolt.org/z/eEYf5c59f
Might be useful in some situations although I currently can't think of any :)
Doesn't feel particularly valuable to have that "help" from the compiler against "accidentally" taking the address of a variable… I mean, how do you even accidentally do that?
if ((Π⁻ < Π) && (Π < Π⁺)) {
if ((Π⁻ < Π) && (Π < Π⁺)) {I guess my question is: does this provenance model allow for recursive nesting of allocators with a separate notion of "storage" at each level?
Does it? It is quite simple for a struct A that has struct B as its first member to have radically different alignment:
struct B { char x; };
struct A { struct B b; long long y; };
Regarding your second point, if I understand the model correctly, there is only an ambiguity in pointer provenance if the adjacent objects are independent "storage instances", i.e. separately malloc'ed objects or separate variables on the stack — not between fields of the same struct.
This is great. I wonder what u/pizlonator thinks of it.
The problem is that the documented definitions of pointer provenance (which generally amount to "you must somehow have a data dependency from the original object definition (e.g., malloc)") aren't really upheld by the optimizer, and the effective definition of the optimizer is generally internally inconsistent because people don't think about side effects of pointer-to-integer conversion. The one-past-the-end pointer being equal (but of different provenance) to a different object is a particular vexatious case.
The definition given in TS6010 is generally the closest you'll get to a formal description of the behavior that optimizers are already generally following, except for cases that are clearly agreed to be bugs. The biggest problem is that it makes pointer-to-int an operation with side effects that need to be preserved, and compilers today generally fail to preserve those side effects (especially when pointer-to-int conversion happens more as an implicit operation).
The practical effect of provenance--that you can't magic a pointer to an object out of thin air--has always been true. This is largely trying to clarify what it means to actually magic a pointer out of thin air; it's not a perfect answer, but it's the best answer anyone's come up with to date.
Previously a lot of C code was non-portable because it relied on behaviour that wasn't defined as part of the standard. If you compiled it with the wrong compiler or the wrong flags you might get miscompilations.
The provenance memory model draws a line in the sand and says "all C code on this side of the line should behave in this well defined way". Any optimizations implemented by compiler authors which would miscompile code on that side of the line would need to be disabled.
Assuming the authors of the model have done a good job, the impact on compiler optimizations should be minimized whilst making as much existing C code fall on the "right" side of the line as possible.
For new C code it provides programmers a way to write useful code that is also portable, since we now have a line that we can all hopefully agree on.