Translating All C to Rust (TRACTOR)
darpa.mil
darpa.mil
"The purpose of this event is to provide information on the TRACTOR technical goals and challenges, address questions from potential proposers, and provide an opportunity for potential proposers to consider how their research may align with the TRACTOR program objectives."
Isn't it equivalent to statically determining the lifetimes of all allocations in the C program, including those that are implemented using custom allocators or which cross into proprietary libraries? There's been a lot of research into this sort of thing over the years without much success. C/C++ programs can do things like tie allocation lifetimes to what buttons a user clicks, without ref counting or other mechanisms to ensure safety. It's not a good idea, but, they can do it.
The other obvious problem with trying to write such a static analysis is that the programs you're analyzing are by definition buggy and the lifetimes might not make sense (if they did, they wouldn't have memory safety holes and wouldn't need to be replaced). The only research I've seen on this problem of statically detecting what lifetimes should be does assume the code being analyzed is actually correct to begin with. I guess you could try and aim for a program that detects where lifetimes can't be worked out and asks the developer for help though.
This isn't, however, DARPA's first foray into automatic program translation, or even automatic translation into Rust[2].
[1]: https://www.urbandictionary.com/define.php?term=DARPA%20hard
[2]: https://c2rust.com/
(But yes: DARPA's mandate is explicitly to discover and develop the next generation of emerging technologies for military use.)
If you count my number of attempts, sure.
If you count by impact, it's hard to come up with many things more impactful than the Internet...?
DARPA is a step closer to traditional research labs but there is obviously some overlap.
It's more like the NSF but focused on commercial grantees with project management thrown on top to orchestrate everything.
The really unique part is how much independence each program manager has and the term limits that prevent empire building.
(But even if DARPA was involved in Ada: I think it's clear, at this point, that Ada has been a resounding success in a small number of domains without successfully breaking into general-purpose adoption. I don't have a particular value judgment associated with that, but from a strategic perspective it makes a lot of sense for DARPA to focus program analysis research on popular general-purpose languages -- there's just more labor and talent available.)
I looked it up. DARPA was not involved.
You hate jumbo jets, high-speed trains, air traffic control, and satellites?
npm ERR! install Couldn't read dependencies
npm ERR! package.json ENOENT, open '/boeing/787-9/flaps-up.json'
npm ERR! package.json This is most likely not a problem with npm itself.
npm ERR! package.json npm can't find a package.json file in your current directory.once the maturity of the users advances to a sufficient point, then ada is the only solution.
"ada. used in creating reliable software since 1983"
when i first saw ada, i didn't understand the why. now i understand the why, but ada is effectively gone.
-- old fortran / C / Assembly programmer
Something unheard of, paying for software tools in 2024, who would imagine that.
gnat needs money. well deserved. but adoption needs a free, easy to install compiler.
5 years ago i had the pleasure of resurrecting a dead system. it was about 30k of ada, lets call it ada 87 (!). unknown compiler, 32 bit, 68K processor, 16 MB memory, unknown OS.
code was compiling in 2 days, running in 2 weeks. i needed to change from using 32 bit floats to 64 bit floats (seems positional data is a little more accurate in 2020). 1 declaration in 1 package spec and a recompile, and all my positions are good.
i love that language!
so, changed wordsize, processor, operating system... minimal source code impact.
How good the resulting performance would be like, that is another matter.
I'd say it's good timing.
But, this isn't just about rewriting code from one language to another. It's about reverse engineering complex information out of the code, which may not be immediately visible in it, and then finding a way to make it "safe" according to Rust's type system. Where's the training data for that? It'd be really hard even for skilled humans.
Personally I think the most pragmatic way to make C/C++ memory safe quicker is one of two approaches:
1. Incrementally. Make std::vector[] properly bounds checked (still not done even in chrome!), convert allocations to allocations that know their own size and do bounds checking e.g. https://issues.chromium.org/issues/40285824
2. Or, go the whole hog and use runtime techniques like garbage collection and runtime bounds checks.
A good example of approach (2) is Managed Sulong, which extends the JVM to execute LLVM bitcode directly whilst exposing to the C/C++/FORTRAN a virtualized Linux syscall interface. The whole piece of code can be sandboxed with permissions, and memory safety errors are caught at runtime. The compiler tries to optimize out as many bounds checks as possible. The interesting thing about this approach is it doesn't require big changes to the source code (as long as it's already been ported to Linux), which means the work of making something safe can be done by teams independent of the original authors. In practice "rewrite it in Rust" will usually mean a fork, which introduces lots of complicated technical, cultural and economic issues.
Managed Sulong is also a research project and has a bunch of problems to solve, for instance it needs to lose the JITC dependency and go fully AOT compiled (doable, there's no theoretical issue with it and much of the needed infra already exists). And performance/memory usage can always be improved of course, it regresses vs the original C. But those are "just" systems engineering problems, not rewrite-the-world and solve-static-analysis problems.
Disclosure: I do work part time at Oracle Labs which developed Managed Sulong, but I don't work on it.
Microsoft has this:
https://learn.microsoft.com/en-us/cpp/standard-library/safe-...
but it doesn't seem to actually make std::vector[] safe.
It's frustrating that low hanging fruit like this doesn't get harvested.
[1] "although there are precondition checks for some string operations, e.g. operator[], they will not always be run when using the char and wchar_t specializations (std::string and std::wstring)."
https://github.com/microsoft/STL/issues/586
> We talked about this at the weekly maintainer meeting and decided that we're not comfortable enough with the (lack of) design of this feature to begin documenting it for wide usage.
I tried a toy example with GCC [1], Clang [2], and MSVC [3], and none of them emit bounds checks with basic optimization flags.
[1] https://godbolt.org/z/W5e3n5oWM
That might not be too bad.
A combination of a formal system and an LLM might work here. Suppose we see a C function
void somefn(char* buf, int n);
If it's an array, how big is it? We don't have enough info to know that yet. But a reasonable guess, and one than an LLM might make, is that the length of buf is "n".
Following that assumption, it's reasonable to translate this to Rust as
fn somefn(buf: &[u8])
buf.len()
assert!(buf.len() == n);
somefn(buf, n);
fn somefn(buf: &[u8], int n)
The idea is to recognize common C idioms and do clean translations to Rust for them. This should handle a high percentage of cases.
But if you're going to do it that way, the right place to start is probably to a safer form of C++ not Rust. That way code can be ported file-at-a-time or even function-at-a-time, and so you'll have a chance to run the assertions in the context of the original code. Which of course may not have good test coverage, as C codebases often don't, so you'll have to be testing your assertions in production.
There's something to be said for that. You're going to need at least an internal representation that's a safe C/C++.
Most compilers do have flags to turn this on, which I use all the time.
The issue is the "performance trumps safety" culture that pushes back against using them.
You can attach about a hundred asterisks to that.
If anything, I think self the failure to hit L5 driving after billions of dollars and millions of man hours invested is probably reflective of how automatic C to Rust translation will go. We'll cruise 90% of the way, but the last 10% will prove insurmountable with current technology.
Think about the number of C programs in the wild that rely on compiler-specific or libc-specific or platform-specific behavior, or even undefined behavior plus the dumb luck of a certain brittle combination of {compiler version} ∩ {libc version} ∩ {linker version} ∩ {build flags} emitting workable machine code. There's a huge chunk of C software where there's not enough context within the source itself (or even source plus build scripts) to understand the behavior. It's not even clear that this is a solvable problem in the abstract.
None of that is to say that DARPA shouldn't fund this. Research isn't always about finding an industrial strength end product; the knowledge and expertise gained along the way is important too.
Even if it's "can handle well 90%" of the transition for any project, this is still interesting. Unlike cars on the road, most code transition project out there doesn't need to be 100% fine to provide some useful value.
Just getting something vaguely Rust shaped which can compile is the first step in overcoming the inertia to leave the program in its current language.
There is real utility in slowly fleshing out the number of transforms in a tool like c2rust that can recognise high-level constructs in C code and produce idiomatic safe equivalents in rust
It'll be great for "pure" functions. For the grimey parts of the world, funcs taking pointer args and returning pointers, for things that access and modify global data without locks, for threaded code with implicit (and undocumented) locking, the tool would add most value. If it can. Even only by saying "this code looks grimey. here's why. A bit of FFI will also be thrown in because it links against 100 libraries. I suggest changes along those lines ... use one of the 2000000 hint flags to pick-your-evil".
> You can attach about a hundred asterisks to that.
Not in San Francisco. There are about 300 Waymo cars safely driving in one of the most difficult urban environments around (think steep hills, fog, construction, crazy traffic, crazy drivers, crazier pedestrians). Five years ago this was "someday" science-fiction. Frankly I trust them much more then human drivers and envision a future utopia where human drivers are banned from urban centers.
To get back on topic, I don't think automatic programming language translation is nearly as hard, especially since we have a deterministic model of the machines it runs on. I can see a possible approach where AI systems take the assembler code of a C++ program, then translate that into Rust, or anything else. Can they get 100% accuracy and bit-for-bit compatibility on output? I would not bet against it.
Getting identical outputs on safe executions, catching any unsafe behavior (at translation-time or run-time), and producing efficient, maintainable code all at once is a million times harder.
In 10 (or 2 or 50 or X) years when Chevy, Ford, and others are rolling out cheap self driving this argument stops working. The important thing is that this argument stops working with no change in how hard C to Rust conversion is.
We really should be looking at the specifics of both problems. What makes computer language translation hard? Why is driving hard? One needs to be correct while inferring intent and possibly reformulating code to meet new restrictions. The other needs to be able to make snap judgments and in realtime avoid hitting things even if it just means stopping to prefer safety over motion. One problem can be solved piecewise without significant regard to time and the other solved in realtime as it happens without producing unsafe output.
These problems really aren't analogous.
I think you picked self driving cars just because it is a big and only partially solved problem. One could just as easily pick a big solved problem or a big unstarted problem and formulate equally bad arguments.
I am not saying this problem is easy, just that it seems solvable with sufficient effort.
I'd put money on the solutions to said problems looking largely the same though - big ass machine learning models.
My prediction is that a tool like copilot (but specialized to this domain) will do the bulk of source code conversions, with a really smart human coming behind to validate.
The director orders the use of the tool when the dev team got downsized (and the two most-seniors left for greener pastures just after that). Validation is in the "extensive" tests anyway, we have those, right, so the new intern shall have a look, make it all work (fudge the tests where possible and remove the persistently failing ones as they've probably been always broken). The salesman said it comes from the DOA or DOD or something. If the spooks can do it so can we.
Which are things that took 20 or 50 years longer than expected in some cases.
> I think you picked self driving cars just because it is a big and only partially solved problem. One could just as easily pick a big solved problem or a big unstarted problem and formulate equally bad arguments.
But C to Rust translation is a big and only partially solved problem.
This is an excellent example of government action I like to see, as it isn't about favoritism or the swamp dynamics. Just provide a target, a bounty and no or low barriers to entry.
This challenge does push Rust out in front of everybody. That's a mixed blessing. I hope this challenge gets modified to not specify the target language, but instead the requirement of memory and type safety. Rust is likely an intermediate stop on the way to something better, and it shouldn't matter if that language is called Rust 2.0 or something else.
and it's open source as well! http://rosecompiler.org/ROSE_HTML_Reference/index.html
I can't believe that works at all. I'll take a look for sure.
It would be most interesting if the approach was not to feed the program the original program but rather the manual for the program. That said it's rare that a manual captures all of the nuances of the program so a view into the source code is probably necessary, at least for getting the ground truth.
"AI more or less sort of summarizes the features of the program into some approximate kind of technical language, then the AI synthesizes something not too far from Rust code that hopefully covers aspirationally the same feature set".
Usually the biggest issue, is that most of the porting attempts don't start there, rather they go for the rewritte from scratch, and lets not pay the licenses for those cross-compilers.
Additional training data can be generated by running mrustc or by inlining unsafe code (from std/core/leaf crates) into safe code and running semantics-preserving mechanical refactorings on the code.
This can be closer to AlphaProof than ChatGPT
If you think it's hard wrestling with the borrow checker, just imagine how much harder it is to write automatic translation to borrow-checker-approved code that accounts for all the possible program space of C and all it's celebrated undefined behavior. A classic problem of writing compilers is that the space of valid programs is much larger than the space of programs which will compile.
A quick web search reveals some other efforts, such as c2rust [1]. I wonder how TRACTOR differs.
That’s not what they are aiming for. FTA: “The goal is to achieve the same quality and style that a skilled Rust developer would produce”
> just imagine how much harder it is to write automatic translation to borrow-checker-approved code that accounts for all the possible program space of C and all it's celebrated undefined behavior
Nitpick: undefined behavior gives the compiler leeway in deciding what a program does, so the more undefined behavior a C program invokes, the easier it is to translate its code to rust.
(Doing that translation in such a way that the behavior remains what gcc, clang or “most C compilers” do may be harder, but I’m not sure of that)
That's the kind of language lawyer approach that caused a rebellion in the last decade amongst C programmers against irresponsible compiler optimizations. "Who cares if your program actually works as intended? My optimization is legal according to the standard, it's your program that's written to exploit loopholes".
I don't see any evidence that that's the attitude being taken by TRACTOR — I sure hope it isn't. But hell, even if the result is unreliable in practice, I suppose that if somebody gets to claim "it works" then the incentives are aligned to produce garbage.
If your program invokes undefined behaviour, it's invalid and non-portable. Out of bounds array accesses are UB, yet a program containing them may just happen to work. It won't be portable even between different compiler versions.
The C standard is a 2 way contract: the programmer doesn't produce code that invokes undefined behaviour, and the compiler returns a standard conforming executable
I don't feel that the Rust language situation incentivizes such awful conflict, and it's one of many reasons I now try really hard to avoid C and use Rust instead.
But I don't think defining all behavior helps. The defined behavior could be /wrong/, and now you can't find it because the program using it is valid, so it can't be detected with UBSan.
Undefined behavior is usually a result of runtime situation, it is usually not obvious from just the code whether it could or could not happen, so the compiler cannot reject the program.
The 'UB-based' optimization is just assumption that the code is correct and therefore UB-situation could not happen in runtime.
> The standards added the forward progress guarantees to change an optimization problem from "solve the halting problem" to "there will be observable side effects in the forms of termination, I/O, volatile, and/or atomic synchronization, any other operation can be reordered". The former is generally impossible to solve, whereas the latter is eminently tractable.
But yeah, that's one of the more foot-gunny UB rules that Rust does not have. But it does mean it doesn't mark functions as `mustprogress` in LLVM IR which means it misses out on whatever optimizations that enables.
You significantly underestimate how much UB people write and overestimate the end-result if the current approach would not be taken.
I don’t see any way it can do otherwise. As a simple example, what would one translate this C statement to:
int i;
…
i = abs(i);
let i: i64;
…
i = abs(i);
unsafe {
let mut i: std::os::raw::c_int
= std::mem::MaybeUninit::uninit().assume_init();
// ...
i = libc::abs(i);
}
Given that generated code, I could then piecemeal migrate the unsafe bits to cleaner, idiomatic safe rust: possibly your code but more likely `i::wrapping_abs()` or similar.
What will TRACTOR choose? At least for this example, they don't have to choose inappropriate pruning of undefined behavior. They claim the following:
> The goal is to achieve the same quality and style that a skilled Rust developer would produce, thereby eliminating the entire class of memory safety security vulnerabilities present in C programs.
If they're going to uphold the same "quality", the translation you presented doesn't cut it. But you may be right and they will go down the path of claiming that a garbage translation is technically valid under undefined behavior and therefore”quality” — if so, I will shun them.
[1] https://play.rust-lang.org/?version=stable&mode=debug&editio...
Because of the leeway the C standard gives you, you can preserve the semantics of the C program by just calling abs, and I think that’s the best you can do.
What the compiler does may be different for different compilers, different compiler versions or different compilation flags, so if all you have is the C source code, there’s no way to preserve the semantics of the machine code that the C compiler generates.
You could special-case all of them, but even then, there is the problem that a C compiler, even in a single translation unit, can inline one call and then apply some transformations while compiling another call to a call to a library function, making the semantics of overflow in one location different from that in another.
If you want to replicate that, I’d say you aren’t writing a C to rust translator, but a (C + assembly) to rust translator.
Also, if you go this route, you’d have to do similar gnarly stuff for all arithmetic on integers where you cannot prove there will not be overflow. I would not call the resulting code idiomatic rust.
To uphold the spirit of Rust, a C program must go through a process where assumptions are laid bare and footguns are dismantled. Applying an automatic process which arbitrarily changes the behavior from the implementation-dependent compilation of a C program just gets you a messy slop of hidden bugs collected inside an opaque, "safe" garbage can.
You don't get to Rust's reliability by applying a translation which discards it!
> Also, if you go this route, you’d have to do similar gnarly stuff for all arithmetic on integers where you cannot prove there will not be overflow.
Damn straight. That's what C is! It was always this bad, as those of us who have struggled to control it can attest. Faithful translation to unsafe Rust just makes it obvious.
It may be better to just leave the assignment off the declaration. If the variable is read before it's initialized to something, we'll get a Rust compilation error, forcing programmer intervention. Detecting actual bugs that would result in memory errors and forcing them to be resolved is very much in the spirit of Rust. TRACTOR may aspire to gift C programs with memory safety for free, but it won't always be possible.
Of course if TRACTOR can determine through static analysis that the unitialized read can't cause problems, it might emit different code.
You assume that the compiler can determine what behavior is undefined. It can't. C compilers don't just look at some individual line of the program and say "oh, that's undefined, unleash the nasal demons". C compilers look at code, reason that if such-and-such variable has a certain value (say, a null or invalid pointer), then such-and-such operation is undefined (say, dereferencing that variable), and therefore on the next line that variable can be assumed not to have that bad value. Despite all the FUD, this is a very limited power. C compilers don't usually know the actual values in question, all they do is exclude some invalid ones.
Can't design a system that 100% crashes on invalid behavior if you've declared that behavior is valid, because then someone is relying on it.
And while DARPA doesn't directly care about research publications as an outcome, there's certainly a publishable research component to this, as well as a lot of lower papers-per-$ engineering and validation work. A lot of the contracts they hand out end up going to some kind of contractor prime (BBN, Raytheon, that kind of company) with one or more academic subs. The academic subs publish.
what you describe is exactly my experience as a darpa performer (on a program which dan is apparently now the pm for!)
You're just asking for people to bring out their pitchforks :P
> most interesting code is written in C++ anyway.
Really?! The Linux kernel is a _pretty enormous_ counterexample, as are many of the userland tools of most desktop Linux distros.
I am also a key developer of an entirely-written-in-C tool which I'd venture that [a large fraction of desktop Linux users in corporate environments use on a regular basis](https://gitlab.com/openconnect/openconnect).
Then you get into stuff that's not kernels and the user-spaces are again mostly all C++. The few exceptions that exist are coming out of the 90s UNIX culture, stuff like Apache or nginx. Beyond that it's all C++ or managed languages.
Whether or not the decision not to use C++ is good or bad, rational or irrational, what is the relevance?
The point is that the Linux kernel is technologically interesting and innovative, under very active development, and it's written in C.
> The few exceptions that exist are coming out of the 90s UNIX culture, stuff like Apache or nginx. Beyond that it's all C++ or managed languages.
I literally just told you about a software project to which I contribute, which thousands of people and organizations use, which is written in C.
Also, it wasn't written in the ’90s.
As a Rust advocate in this domain I have attempted to temper the expectations of those driving this proposal with due respect to the feasibility of automatic translation from C to Rust. The fundamental obstacle that I foresee remains that C source code contains less information than Rust source code. In order to translate C code to Rust code that missing information must be produced by someone or something. It is easy to prove that it is impossible to infallibly generate this missing information for the same reason that scaling an image to make it larger cannot infallibly produce bits of information that were not captured by the original image. Instead we must extrapolate (invent) the missing information from the existing source code. To extrapolate correctly we must exercise judgement and this is a fallible process especially when exercised in large quantities by unsupervised language models. I have proposed solutions that I believe would go some way towards addressing these problems but I will decline to go into detail.
Ultimately I will say that I believe that it is possible for this project to achieve a measure of success, although it must be undertaken with caution and with measured expectations. At the same time it should be emphasized it is also possible that no public result will come of this project and so I caution those here against reading too much into this at this time. In particular I would remind everyone that the government is not a singular entity and so I would not interpret this project as a blanket denouncement against C or vice versa as a blanket blessing of Rust. Each agency will set its own direction and timelines for the adoption of memory-safe technologies. For example NIST recommends Rust as well as Ada SPARK in addition to various hardened dialects of C/C++.
Thank you!
If you don't go for preserving the formal semantics of C code and instead only require the test-suite to still pass after translation that can provide a lot of wiggle room for the translation. This is how oxidation projects often work in practice. Fuzzers can also help with generating additional test data to get good branch coverage.
I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety. One gets the same safety as a gradual Rust rewrite, but the code base, knowledge base, and developers can be maintained.
There is no such mentality anywhere. There is a ton of software that's much better off left alone in a dynamic language, or a statically typed language with a garbage collector (like Golang). Good engineers understand the idea of using the right tool for the job.
The push is to start reducing those memory safety CVEs because they have been proven to be a real problem, many times over.
> mechanical translation is a poor means of doing so
Agreed. If we could automatically and reliably translate C/C++ to Rust it would have been done already.
> Spend the time planning better architecture and designing a better software system, and find a way to replace it piece by piece.
OK, I am just saying that somewhere along that process people might get a bout of confidence and tell themselves "oh, we're doing C much better now, we no longer write memory safety bugs, can't we stop here?" and they absolutely will. Cue another hilarious buffer overflow CVE 6 months later.
> I think a far better and more mature process is to update C to modern C and use a model checker such as CBMC to verify memory, resource, and integer math safety.
A huge investment. If you are going to do that then you might as well just move to Rust.
> One gets the same safety as a gradual Rust rewrite
Maybe, but that sounds fairly uncertain or far from a clear takeaway to me.
People say that, but the people who say this rarely have any practical experience using CBMC. It's very straight-forward to use. I could teach a developer to use it reliably, on practical software, in a month.
What I am saying is that if it were as simple as "just learn CBMC" then maybe Microsoft and Google would have not published their studies demonstrating that 60% - 75% of all CVEs are memory safety errors like buffer under-/over-flows.
My experience tells me that if a tech stack supports certain safety guarantees by default that this leads to measurable reduction of those safety problems when you switch to the stack. People love convenient defaults, that's a fact of life.
The apparently inconvenient truth is that most programmers are quite average and you can't rely on them going above and beyond to reduce memory safety errors.
So I don't buy the good old argument of "just hire better C programmers". We still have a ton of buffer overflow CVEs regardless.
And I never "pretended it's impossible to avoid safety issues in C". I'll appreciate if you don't clump me in some imaginary group of "Rust proponents".
What I'm saying is this: use the right tool for the job. The C devs have been given decades and yet memory safety CVEs are still prevalent.
What conclusion would you arrive at if you were in my place -- i.e. not coding C for a living for like 18 years now but still witnessing it periodically crapping the bed?
I'm curious of your take on this. Again, what other conclusion would you arrive at?
For example, let's look at the "hire better C programmers does not work" argument. Like every good propaganda it starts with a truism: In this case that even highly skilled C/C++ programmers will make mistakes that could lead to exploitable memory safety issues. The problem comes from exaggerating this to the idea that "all hope is lost and nothing can be done". In reality one can obviously do a lot of things to improve safety in C/C++. And even one short look at CVEs should make it clear that there is often huge room for improvements even with relatively simple measures. For example, a lot of memory safety bugs in C/C++ come from open-coded string or buffer manipulation. But it is not exactly rocket science to abstract this away behind a safer interface. But once this is understood, the obvious conclusion is that addressing some of these low-hanging fruits would be far more effective in improving safety than wasting a lot of time and effort in rewriting in Rust.
That's not "in reality", that's "in theory". Because in actual reality, people still write the good old buffer overflow bugs to this day.
I don't think anyone reasonable is disputing that we indeed can improve C/C++ programming. The argument of myself and many others like myself is: "a lot can be done but for one reason or another it is STILL NOT being done". Likely the classic cost cutting but there are likely other factors at play as well.
> But once this is understood, the obvious conclusion is that addressing some of these low-hanging fruits would be far more effective in improving safety than wasting a lot of time and effort in rewriting in Rust.
Explain why this has not been done yet. Explain why Microsoft, Google and various intelligence agencies attribute memory safety bugs to between 60% to 75% of all CVEs and demonstrable exploits that they are aware of.
Please do, I am listening. Why has almost nothing been done yet?
Secondly, "wasting a lot of time and effort in rewriting in Rust" is an empty claim. To demonstrate why, I ask you this: at which point the continued cost of investing in endlessly patching C/C++ and all its glorious foot-guns becomes bigger than the cost a rewrite?
Surely at one point just endlessly throwing money at something that gives you a 1% return of investment (in terms of getting more stable and less dangerously buggy) does indeed get more expensive than starting over?
I have no clear answer because it depends on the organization, the tenure of C/C++ and the devs in the org, and many others. It's strange that you pretend to have the answer.
That's because while the technology exists, it is not widely communicated. That's not a fault of C, and that's not something that any language can solve.
> Explain why this has not been done yet.
See above.
The technology to make C and C++ safer is not yet widely used. But, it exists and it is being used. I use it on every firmware and OS project that I currently work on. The code we produce is free of memory errors, integer errors, API misuse errors, resource management errors, cryptography errors, confused deputization errors, and a host of other errors that our specifications are designed to catch. That goes well beyond what Rust or any other language can provide on its own. But, to be fair, Rust developers can do this using similar tooling.
It's laudable that you wish to rid the world of memory errors. I want to normalize going three or four steps further. Rust by itself won't get us there.
If you say so. Rust clearly does, and before you go saying "but `unsafe` exists!" I'll have to remind you that (1) scarcely any Rust devs reaches for that and (2) it still keeps quite a lot of guarantees and only relaxes some. Some, not all. Not even most.
> It's laudable that you wish to rid the world of memory errors. I want to normalize going three or four steps further. Rust by itself won't get us there.
Well now we are on the same page. I never said "ONLY Rust will save us", I am saying that Rust clearly can get us further than we are right now. If there's something even more accessible, less verbose, and with not such a cobbled together Frankenstein async implementation like Rust, I'll start using it tomorrow.
Until it exists at the kernel layer, the firmware layer, the runtime library layer, and the application layer, these issues still exist. CVEs come out weekly for memory errors in Linux, in firmware, in operating system libraries, and in application libraries. We need to think beyond rewriting code in one language or platform, and instead think about technologies that we can apply to all languages and platforms, including C and Rust.
> I am saying that Rust clearly can get us further than we are right now.
As can bounded model checking, without having to teach developers a new language with new idioms.
> If there's something even more accessible, less verbose, and with not such a cobbled together Frankenstein async implementation...
Indeed there is. Reach for the bounded model checker that works with your existing language or platform. Pour over the manual, and look at existing practical examples.
If you like Rust, feel free to use it. But, if you prefer C/C++, Pascal, Ada, Python, C#, Java, or Modula2, that's fine. Either use an existing bounded model checker for that language or port CProver / GOTO to that platform. Rust developers ported CProver to Rust via Kani, because they also recognize that writing safer code can't be done by language alone.
I don't think it's necessary to push people to use different languages or platforms to write safer code. They just need to use or port existing tooling and learn safer coding practices. If I come at firmware developers or old school OS developers with "we need to use Rust", the conversation is immediately shut down and I'm considered a fool. If, instead, I show them tooling that allows them to maintain their existing code base and make it safer, I get much further.
CBMC translates C -- the same language -- to an SMT solver. A different target but the same language.
It is true that new idioms will often be discovered along the way of converting existing C to pass the bounded model checker in every branch condition and in every case. However, software that is already relatively safe will require very little modification. I've seen it go both ways. Simpler code bases can pass model checks relatively unscathed. More complex code bases require refactoring to pass model checking.
To my point, the code base can remain in C, and can be model checked gradually. It doesn't have to be ported to a different language or platform. But, it will require added assertions and some refactoring to make the execution of code more clear. It's still in C. The specifications are specified in C using regular assertions. The only thing that changes is that one will often use shadow methods -- still written in C but simpler than the functions they are shadowing -- in order to model check other functions.
Other bounded model checkers like JBMC, Kani, or PolySpace work in similar ways.
Better process and better tooling allows us to write better software. Bounded model checking is an incredibly useful bit of tooling that allows us, within context of the software, to demonstrate that certain conditions do not arise. This includes memory errors, resource errors, and other classes of errors. The limitation is the faithfulness of the translation to SMT and the complexity of the code being modeled. The former has gotten quite good with CBMC 6, and the latter can be managed through careful refactoring and shadow function substitution.
Is it magic? There is no such thing. But, it is a practical tool that is available for use today.
One need not wait until an entire web browser is verified using it. It can scale to this, but given the unreasonable size and scope of web browsers with respect to this challenge, which are basically operating systems and suites of software in one these days, that's like saying, "verify all software then blog about it."
Solaris has already been doing it since 2015, ARM more recently, we have Microsoft putting the big buckets into CHERI (including custom FPGA boards for testing), the new CoPilot+ PCs architecture with Pluton, and while AMD/Intel attempts weren't quite right like MPX, they will surely do something for x64 as well.
How, because other solutions are being explored? That's not due to a failure of one thing, but because both defense in depth and a desire to fix existing systems with no additional engineering are paths that security researchers and vendors explore. Not everyone will converge on a single solution, even when that solution is practical.
Just because something is not being used universally doesn't mean that it has failed. Moreso, it is not widely known about, and there persists rumors that it requires extraordinary effort, often reinforced by well meaning, but rather outdated advice.
Hardware security is complementary to software security. Mitigations at the hardware level, the hypervisor level, and the operation system level complement architectural, process, and tooling decisions made at the software level.
Defense in depth is a good thing. There can always be errors in one layer or another, regardless of software solution, operating system, hypervisor, or hardware. I can wax poetic about current CPU vulnerabilities that must be managed in firmware or operating systems.
Many of the issues caused by C, are solved by Modula-2, Object Pascal and Ada, we didn't need to wait for Rust. But those aren't the languages that come for free with UNIX.
Or even better, they would be solved by C itself, if WG 14 cared even a little about providing proper support for slices, proper arrays and proper string types, or even as library vocabulary types.
But what to expect, when even Dennis Ritchie wasn't able to get his approach to slices being worked on by WG 14.
So hardware memory tagging, and sandboxed enclaves it is.
I make extensive use of bounded model checking in my C development. I also use privilege separation, serialization between separate processes, process isolation, and sandboxing. That's not because bounded model checking has somehow failed, but because humans are fallible. I can formally verify the code I write, but unless I'm running bare metal firmware, I also have to deal with an operating system and libraries that aren't under my direct control. These also have vulnerabilities.
That's not a trivial thing. The average software stack running on a server -- regardless of whether it is written in C, Rust, Modula-2, Pascal, Ada, or constructively proven Lean extracted to C++ -- still goes through tens of millions of lines of system software that is definitely NOT safe. All of that code is out of a developer's control for now. Admins can continually apply patches, but until those projects employ similar technology, they are themselves a risk.
One day, hopefully, all software and firmware will go through bounded model checking as a matter of course. Until then, we work with what we can, and we fix what we can. We can also rely on hardware mitigations where applicable. That's not failure as you have claimed, but practical reality.
I would absolutely love it if you were the majority, alas you are not.
I emulate exhaustive pattern matching in my main language of choice because it does not have it (it's not Rust or OCaml or Haskell) but because I saw how beneficial and useful it is. And sadly, many of the other devs using that language don't do so, and I have made a good buck going after them and fixing their mistakes.
I don't doubt your abilities as a person. I doubt the abilities of the corpus of C/C++ devs at large.
I can say, without ego, that I'm a reasonably good software developer. But, it is the tooling and process that I use that allows me to build safer software and that makes me a reasonably good developer. The same is true of Rust developers.
I can teach these skills to other developers, and in fact, I have plans to do so.
I don't expect things to change overnight, any more than I expect things to be rewritten in Rust overnight. C++ has been around for nearly 40 years, and software is still written in C. But, we can do better, and we must do better.
We both want the same thing: safer software.
I, too, enjoy sci-fi.
> Just because something is not being used universally doesn't mean that it has failed.
You are only correct in the dictionary sense of these words. Fact is that a lot of the programmers are vain creatures prone to ego, and they make their chosen technical stack part of their core identity. This prevents them from being flexible, they get rigid as they age and they become part of the problems they so passionately wanted to fix when they were young.
None of that is made easier by the managerial class that absolutely loves and financially stimulates the programmers who don't want to rock the boat.
So I'd say if the said CBMC, and likely other tools in the same area, has more or less failed if it could not convince a critical mass of C/C++ devs to use it and finally start keeping up with Rust (and the other languages @pjmlp mentioned).
> Moreso, it is not widely known about, and there persists rumors that it requires extraordinary effort, often reinforced by well meaning, but rather outdated advice.
The victims of Heartbleed and many other CVEs don't care. The breaches happened anyway.
I am amazed at your desire to downplay the problem and keep claiming that eventually stuff will work out.
I disagree. And I'll repeat a very core part of my argument: C/C++ devs were handed a monopoly in their areas for decades and they still can't arrive at a set of common techniques that reduce or eliminate memory safety bugs.
I am not impressed. And I am not even a particularly good programmer. Just a diligent guy with average programming ability whose only unique trait is that he refuses to accept the status quo and always looks at how can stuff be improved. But this has taken me a long way.
I was characterizing these hardware changes as being fantasy, so I'm glad you agree.
> So I'd say if the said CBMC, and likely other tools in the same area, has more or less failed if it could not convince a critical mass of C/C++ devs to use it
So, in the same vein, Rust has failed because it has only been around for a similar amount of time and people still use C/C++?
> The victims of Heartbleed and many other CVEs don't care. The breaches happened anyway.
I fail to see how a CVE that occurred due to poor engineering practices has anything to do with the adoption of good engineering practices and tooling. Yes, Heartbleed is why we need this tooling.
You are simultaneously arguing that if we could just adopt Rust, our problems would be solved, but since another technology has not yet been adopted, it has failed. Rust isn't adopted due to programmer ego, but the use of tooling that does the same thing as Rust and more has not yet been adopted because it has failed. Do you not see the logical inconsistency in your position?
Yes, it kind of failed there indeed. And I even hinted at why: Rust is far from perfect and its async implementation is a cobbled together mess. Golang's model reads much better, though I hate their foot-guns quite a lot (like writing to a closed channel leads to a panic; who thought that was a good idea?).
> I fail to see how a CVE that occurred due to poor engineering practices has anything to do with the adoption of good engineering practices and tooling. Yes, Heartbleed is why we need this tooling.
You can't see it? But... the good practices do lead to less of these CVEs as you yourself seem to realize? I don't get this part of your comment.
> You are simultaneously arguing that if we could just adopt Rust, our problems would be solved, but since another technology has not yet been adopted, it has failed.
You have answered it yourself: a lot of people see manual wrangling of `void**` as a badge of honor and their ego takes over (and the fear of being displaced, of course). I claim that Rust is not being more widely adopted due to programmer ego and fear of being obsolete. The fear of the end of nice salaries because they belong to a diminishing cohort of old-school cowboys.
Who would not fear that? Who would want that to end?
> Do you not see the logical inconsistency in your position?
No, and I don't get your argument. The reasons for C/C++ devs not improving the memory safety of their code, and the reasons for them not adopting Rust are very different. Not only is the analogy bad, it is plain inapplicable.
---
But it also does not help that HN reacts like a virgin schoolgirl pinched on the arse when Rust is mentioned. I've coded it for a few years, I loved it, I hated the bad parts and called them out, but even to this day I very quickly and easily get branded as a Rust fanboy even if my comment history shows balanced criticisms towards it. People don't care. People are emotional and are quick to put you in a camp that's easy to hate.
That is the part that I truly hate. No objective debate.
Too expensive to move to Rust? GOOD! That's an amazing argument, we can talk that for weeks and get very interesting insights in both directions.
People unwilling to get re-trained? Also a good argument, with big potential for interesting insights!
But most of everything else is at the level of a heated table debate after the 11th beer. Pretty meh and very uninteresting. No idea why I keep engaging, I think I am just bitter that people who REALLY should know better are reacting on emotion and not on merit. But that's on me. We all have our intolerances to the reality we inhabit. This is one of mine.
There definitely is. Mainstream and official Rust community material is generally sane, but the meme did not come from nowhere. The rewrite-everything people are out there.
Meh, there are zealots in every community -- we're not even talking programming language communities only. Not even programming either. Everywhere.
No idea why people over-reacted so much to one particular 0.1% fanatics. It's a pretty normal state of affairs. Point me at your hobby group and even if it is only 20 people I can bet my balls at least 1 of them is a fanatic.
No chance. CBMC is amazing, but have you actually tried formally verifying a "real" program?
I agree replacing with a hand-architected Rust version is clearly the better solution but also more expensive. I think they're going for an RLBox style "improve security significantly with little-to-no effort" type product here. That doesn't mean you shouldn't do a full manual rewrite if you have the resources, but it's better than nothing if you haven't.
Yes. Every day. It's actually quite easy to do. Write shadow methods covering the resources and function contracts of called functions, then verify the function. Repeat all of the way up and down the stack. It adds about 30% overhead over just TDD development.
The biggest issue you will run into with bounded model checking is recursion and looping. In these cases, you want to refactor the code to make it easier to formally verify outside of the loop. Capture and assert on loop variants / invariants, and feed these forward in assertions on code.
There's no way I can capture all of this in an HN comment, but to get CBMC to work, you need to break down your code.
Think of shadow functions as the specifications that you are building. Unlike proof assistants or Frama-C, you write specifications in C itself, and they work similarly to code. Often, the same contracts you write in these specifications can be shared by both the shadow functions and the real functions they shadow.
I take a bottom-up approach to model checking. I'll start by model checking the lowest level code, then I'll shadow this code to model check code that depends on it. In this way, I can increase the level of abstraction for model checking, focusing just on the side effects and contracts of functions I shadow, and move up the stack toward more and more general code.
But there's a non-zero chance that someone could develop a skillset for iteratively cleaning up into something tolerable.
And then there are non-goal things that could grow out of the project, e.g. some form of linter feedback "can't translate into tolerable rust because of x, y and z". C people could look into that, and once the code is translatable into good rust, why translate.
If that was an outcome of the project, some people might find it easier to describe their solution in runnable C and let the "translator/linter" guide them to a non-broken approach.
I'd certainly consider all these positive outcomes quite unlikely, but isn't it pretty much the job description of DARPA to do the occasional dark horse bet?
If you give developers a shitty codebase then those developers will leave to work somewhere else.
After a few years of working on this codebase we had 88% turnover. 1 in 10 developers remembered the original project's design philosophy and intention.
It wasn't a good situation.
CVE database proves that those kind of errors keep coming up in 2024, regardless of those changes.
Not only do they keep coming up, the monetary cost of fixing those issues has raised up to a level that now even governments are looking into this.
There are more slaves in the world now than ever before in history, but global society has still made great progress on eliminating it in the last thousand years.
Not that it matters, but isn't that technically ANSI C(89)? If I remember correctly, the first ISO C standard is instead C90, which is basically identical to C89.
I could not find an open-source static analyzer (including -analyzer) that would actually pick up the flaw before someone tries to exploit it.
And that's a simple example.
We can't tame the dragon C is, empirically nobody can.
Practical formal methods using a tool like CBMC can make C safer. The existing code base can be made safer without porting it to a new language or using experimental mechanical translation. This isn't just something for C. Such tools exist for many languages now, including Rust, so that even Rust can be made safer.
WG14 can solve those problems, they decided it isn't their priority to fix C.
That's like saying, "Getting everyone to use Rust or TDD or X is like trying to boil the ocean."
It's impossible to solve all things for all people at once. But, that doesn't mean that we can't advocate for tooling that can be used today to build safer software. This goes beyond C, as such tools and techniques are being ported to many languages and platforms.
Rust is a solution that works for some people. Modern C with bounded model checking is another solution that works for some other people. I'm certainly not going to change the minds of folks who have decided to port a project to Rust and who are willing to spend the engineering budget for this. But, hopefully, I can convince someone to try bounded model checking instead of maintaining the status quo. Because, the status quo is where we are with projects like the Linux kernel. Linux may pay lip service to Rust folks and allow them to write some components in that language, but the majority of the kernel is still in C and is not being properly vetted for these vulnerabilities, as we can see with the stream of CVEs coming out weekly.
> WG14 can solve those problems, they decided it isn't their priority to fix C.
WG14 must maintain some semblance of backwards compatibility with previous versions of C. It's no good to make a feature that breaks older code. This happens from time to time -- old school K&R C won't work in a C18 or C23 compliant compiler -- but efforts are made to keep that legacy code compiling, for good or ill.
I prefer to work toward fixing what is. We are unlikely to see things like array slices in C, and even if such features were added, this does nothing to fix the billions of lines of legacy code out there.
1. C and C++ just aren't safe enough at large. Even with careful programming and good tooling, so many vulnerabilities are caused by their unsafe by default designs. Therefore, as much code as possible should be translated to or written in "safe" languages (especially ones that guarantee memory safety).
2. We are funding and calling for software to translate existing C code into Rust.
It's not a consensus to rewrite the world in Rust. It's a consensus to migrate to safe languages, which Rust is an example of, and a program that targets Rust in such migration.
So when those languages have 'unsafe' constructs what are the rules going to be around using those? Without a defining set of rules to use here you're just going to end up right back where you started.
> to migrate to safe languages, which Rust is an example of
Rust has a safe mode. It is _not_ a safe language. To do anything interesting you will require unsafe blocks. This will not get you very much.
Meanwhile you have tons of garbage collected languages that don't even let the programmer touch pointers. Why aren't those considered? The reason is performance. And because Rust programmers "care" so much about performance you're not ever going to solve the fundamental problem with that language.
Do you want performance or safety? You can't have both.
This is not true.
Burying unsafe blocks in unevaluated cargo modules does not make this true. You're just taking the original problem and sweeping it under the rug.
And unsafe isn't a problem. It's a point of potential danger to be heavily audited, tested, and understood. Having the entire language unsafe by default is an obviously worse situation. This is throwing the baby out with the bathwater, like rallying against seat belts because you can still die while wearing one. An improvement is still an improvement. I don't understand why people criticizing Rust tend so heavily to let perfect be the enemy of good.
if you've convinced yourself that you're special and all problems with c are solved by trying harder, clearly everyone else is just lazy. with that line of logic, there's nothing to fix with c. rust is not just redundant, but also aggravating, since its popularity causes the cognitive dissonance to start creeping in.
maybe i can make mistakes? should we improve tooling somewhat? no, it's the children who are wrong.
If you're even remotely familiar with professional C development then you should know this is unironically true. Tooling does exist to offer memory-safe features in C, they're just far more complicated than using a safe language from the offset. Nobody wants to use Valgrind when your linter can do the same job without leaving your editor.
Most of today's high-performance C code is compiled using the same IR that LLVM generates when compiling C. Unless you're a GCC pundit it doesn't make sense to reject the direction the industry is headed in.
> maybe i can make mistakes? should we improve tooling somewhat?
After a while, being allowed to make mistakes starts to pile up: https://www.zdnet.com/article/microsoft-70-percent-of-all-se...
1. There are plenty of interesting programs which don't require unsafe.
2. Even if your program does require unsafe, Rust still limits where the unsafety is. This lets you focus your scrutiny on the small section of the program which is critical for safety guarantees to hold. That is still a win.
this is just flagrantly false, have you no shame?
This is largely untrue. You can use proven abstractions over 99% of cases that would require unsafe.
That would at least get you to as much rust as possible, and then let engineers tackle rethinking just those concepts.
transform CtoASM, then ASMtoRust.
what you need to avoid is incompatibilites between different high level languages with a low level intermediary so you arent stuck attempting to convert high level hardware abstraction directly to another high level hardware abstraction.
The problem you run into is that the conversion is so pedantically correct that the resulting code is useless. The result retains all of the problems that the C code has, and is so far from idiomatic Rust that it's easier to toss the code and start from scratch. Progressive lifting on unsafe Rust to safe Rust is a very difficult order, and the tool I mentioned had a tool to do that... which is now abandoned and unmaintained.
At the end of the day, the chief issue with converting to safe Rust is not just that you have to copy semantics over, but you also have to recover a lot of high-level preconditions. Turning pointers into slices is perhaps the easiest task of the lot; given the very strict mutability rules in Rust, you also have to work out when and where to insert things like Cell or Rc or Mutex or what have you, as well as building out lifetime analysis. And chances are the original code doesn't get all these rules right, which is why there are bugs in the first place.
Solving that problem is the goal of this DARPA proposal, or perhaps more accurately, determining how feasible it is to solve that problem automatically. Personally, I think the better answer is to have a semi-automated approach, where users provide as input the final Rust struct layouts (and possibly parts of the API, to fix lifetime issues), and the tool automates the drudgery of getting the same logic ported to that mapping.
The hardest part may be Rust's affine type rules. Reference use in Rust is totally different than pointers in C/C++. Object parenting relationships are hard to express in Rust.
Assume the AI actually needs to figure out lifetimes and so on to be actually useful and make valid programs. Which would be impressive if it does.
You also don't have to one-shot translate everything. One of the valuable things about the Rust compiler is it gives lots of specific information that you can feed back into an LLM to iterate.
I've been working on similar problems for my startup (grit.io) and think C -> Rust is definitely tractable in the near term. Definitely not easy but certainly solvable.
ASTs also don’t have much info on threading (that’s more or less limited to “the program starts a thread with entry point foo at some time”, “Foo waits for another thread to finish”)
ASTs also generally don't actually have magical information in them. They won't solve the lifetime issues for you.
If we ignore multi-threaded programs is long term aliasing actually ubiquitous in C programs? For many programs, I would expect most of it to happen within the scope of a single function (and within it, across function calls, but there, borrowing will solve this, won’t it?)
If so I would trying to tackle that as one sub-problem (you have to start somewhere), and detecting how data gets shared between threads as another. For the latter, I expect that many programs will have some implicit ownership rule such as “thread T1 puts stuff in queue Q where thread T2 will pick it up” that can be translated as “putting it in queue transfers ownership”.
Detecting such rules may not be easy, but doesn’t look completely out of reach for me, either, and that would be good enough for a research project.
If you stick to extremely stringent coding practices and incorporate third party static verification tools that require riddling your code with proprietary situations, then sure, you can achieve comparable results with C/C++.
Or you can just use Rust.
The real question is what tradeoffs you are making and what you are gaining. Rust makes certain memory safety guarantees about the program at compile time, but at the same time it disallows perfectly safe constructions, which can exist in C++, as well.
The reason I personally chose Rust as my go to language for low level programming is that despite learning systems programming in college I pretty much never used it outside of school. Meaning I didn't have any of that knowledge that c and c++ programmers had built up over years of experience. So I decided that instead of having to deal with the unknown skill deficiencies in writing concurrent software and memory management I'd rather just have a compiler scream at me. I don't regret the decision.
Also, I remember writing an async TCP implementation in college with c++ using boost. Rust tooling is just so far ahead of that.
I see this differently: DARPA bets on different baskets in parallel. This is just one basket, if they are wrong it doesn't matter because there are other bets to reduce the general risk.
I don't see anyone defending JavaScript. In fact a whole lot of people are using typescript now because JavaScript is just so bad.
As for python, that's a good point. I guess it's just because it's easy to use and all the numerical stuff is done with c-bindings anyway?
But the C++ Situation is genuinely different. There's a reason governments are now calling upon developers to just let it die already[0]. That design flaw is so bad it's causing genuine harm.
[0] https://www.cisa.gov/news-events/news/urgent-need-memory-saf...
No, it's horrible, because now you have both python types and numpy types, which don't really interact well with one another. If you are using a language made for numerical analysis (e.g. Julia), a lot of headaches disappear instantly.
Python is 100% just a case of a language being used because it is being used. It has, by itself, few merits to many of the tasks it is actually being for.
>design flaw
It is a tradeoff though. Rust is paying that tradeoff by being very restrictive about certain patterns and being in general quite complex to learn.
Sorry, autocorrect, I typed this on my phone.
In C, having two separate data structures which carry an identical pointer and are writing to it is a common occurrence. This can not be trivially replicated in rust and will need some reasonably clever intervention.
What you won’t get is an output which is less buggy, or a process which automatically generates the program structure which yields Rust’s reliability.
Obviously some C code isn't just "not verifiable correct" but "actually wrong in a memory unsafe way". That code isn't going to be automatically translated without human intervention because, how could it be, there is no correct equivalent code. The tooling is going to have to have an escape hatch where it says "I don't know what this code is meant to do, and I know it isn't meant to do what it does do (violate promises to the compiler), help me human".
On a theoretical level it's not possible for that escape hatch to only be used when undefined behaviour does occur (rices theorem). On a practical level it's probably not even desirable to try because obtuse enough code shouldn't just be blindly translated.
So what I imagine the tooling ends up looking like is an interactive tool that does the vast majority of the work for you, but is guided by a human, and ultimately as a result of that human guidance doesn't end up with exactly equivalent code, just code that serves the same purpose.
1) Rust contains no memory bugs 2) C can be automatically translated to it
Then all memory bugs can be fixed automatically, which is almost certainly untrue. This task is very likely completely impossible in the general case.
So what would it take to actually do this right? The two big problems are 1) array sizes, and 2) non-affine pointer usage. Pointer arithmetic is also hard, but rare. Most pointer arithmetic can be expressed as slices.
Every array in C has a size. It's just that the compiler doesn't know what it is.
Where is this being discussed in detail?
In my understanding, this is a call for proposals to do the work, there is no detailed discussion yet. That will come when there's actual responses to this call.
(To give you a sense, it translates something like a + 1 to a.unwrapped_add(1i32), and my recollection is that for (int i = 0; i < 10; i++) gets helpfully turned into a while loop instead of a for loop).
In general, the various challenges that all need to be solved that aren't solved yet are:
a) when is integer overflow intentional in the original code so that you know when to use wrapping_op instead of regular Rust operators?
b) how to convert unions into Rust enums
c) when pointers are slices, and what corresponds to the length of the slice
d) convert pointers to references, and know when they're mutable or const references
e) work out lifetime annotations where necessary
f) know when to add interior mutability to structs
g) wrap things in Mutex/RwLock/etc. for multithreaded access
We're a very long way from having full-application conversion workable, and that might be sufficiently difficult that it's impossible.
Rust has to use reference counts in such situations. You have an Rc wrapped around structs, sometimes a RefCell, and .borrow() calls that panic when you have a conflict. C code translates badly into that kind of structure.
Static analysis might help find .borrow() and .borrow_mut() calls that will panic, or which won't panic. It's very similar to finding lock deadlocks of the type where one thread locks the same lock twice.
(If static analysis shows that no .borrow() or .borrow_mut() for an RwLock will panic, you don't really need the RwLock. That's worth pursuing as a way to allow Rust to have back references.)
In any case, it's the difficulty of that reconstruction step that leaves me thinking that automated conversion of whole-application to Rust is a near-impossibility. Conversion of an individual function that works on plain-old-data structures is probably doable, if somewhat challenging.
An off-the-cuff idea I just had is to implement a semi-automated transformation, where the user has to input what a final conversion of a struct type should look like (including all Cell/Rc/whatever wrappers as needed), and the tool can use that to work out the rest of the translation. There's probably a lot of ways that can go horribly wrong, but it seems more feasible than trying to figure out all of the wrappers need to be.
I and someone else ran this on a JPEG 2000 decoder that sometimes crashed with a bad memory reference. The Rust version crashed with the same bad memory reference. It's bug-compatible.
What comes out is totally unreadable and much bigger than the original C code. Manual "refactoring" of that output is hopeless.
It still leave you with Rust code that you can improve piecewise. The only question is if something like it is better than FFI calling the C code.
That would be useless when translating C to Rust. Yes, I would expect the tool to point out the flaws in the original memory handling and only translate the corrected code. This is far from easy, since some information (intent) is missing, but a good coder could do it on decent codebases. The question is, can an automated tool do it too? We'll see.
Of course it is. The README says it generates unsafe rust in the first paragraph, what did you expect?
I think it's a really fascinating experiment, and IMHO it's pretty remarkable what it can do. This is an incredibly difficult problem after all...
But also, it's definitely not a growable array. Box::new_uninit_slice makes the thing you've got here, a heap allocation of some specific size, which doesn't magically grow (or shrink) and isn't initialized yet.
Using "unsafe" doesn't automatically make Rust useless, of course, but the example on the c2rust website itself doesn't make any effort to verify its usage of unsafe (you can easily read memory out of bounds just by changing "n" to "n + 1" in the example loop). Sadly, that is a much, much harder problem to solve even for fairly basic C programs.
Not to mention that this quote is incredibly scary. This is someone we are trusting to make this decision?
"You can go to any of the LLM websites, start chatting with one of the AI chatbots, and all you need to say is 'here's some C code, please translate it to safe idiomatic Rust code,' cut, paste, and something comes out, and it's often very good, but not always," said Dan Wallach, DARPA program manager for TRACTOR, in a statement.
1. A code base written in C and a team of C engineers that have a good mental model of the code base to be able to maintain it.
2. An automatically translated Rust code base. Potentially (I'd say probably, but that is just my gut feeling) harder to read and understand than the original one.
3. Now you need a team of Rust engineers that have a good mental model of the code base that was generated.
If you already have that team of Rust engineers, I'd rather let them rewrite the code manually as they can improve it and have the correct mental model from the start.
One might argue that re-writing from scratch is the safer option; and a re-write is also an opportunity to do things differently (read: improve the architecture by using what one has learned), despite the much-feared "second system" syndrome.
But nothing wrong with spending some research dollars towards tooling for "assisted legacy rewrites". DARPA and her sister IARPA fund step innovation (high risk, high reward), and this is an area where good things can come potentially come from.
Async is still a half finished mess even for people that use it every day. And that is my main annoyance but there are many (trait specialization, orphan rule limits, HKT, etc.)
(I'm only being half-facetious, I fear I may never break in!)
https://www.youtube.com/watch?v=QIE5nV5fDwA
iirc, they were able to transpile 90%+ (without AI) and manually did the rest
That said, while I can acknowledge the benefits of memory safety, I would personally choose zig over rust.
All things considered I know you can do some safety check for C using the compiler, and that helps reduce the odds of memory issues.
Idk what rust mail libraries look like( are they even called that?) But I know C's standard library's have made learning stuff easier. Is their any way to know if your libraries in rust are using unsafe code? Will that just spit out compile time errors?
Considering we already know how to make C/C++ programs memory safe, it's bizarre that people would ditch all of their expertise, and the years and years of perfecting the operation of those programs, and throw all that out the window because they can't be bothered to use a particular set of functions [that enforce memory safety].
If you're going to go to all of the trouble to gain expertise in an entirely new tool, plus porting a legacy program to the new tool, I think you need a better rationale than "it does memory safety now". You should have more to show for your efforts than just that, and take advantage of the situation to add more value.
(It's also not clear that they aren't trying hard enough: Google, Apple, etc. have billions of dollars riding on the safety of their products, but still largely fail to produce memory-safe C and C++ codebases.)
The OpenSSL Foundation is not exempt from criticism despite inadequate funding. Heartbleed was discovered by security researches using fuzz testing, but proactive fuzz testing should have been a standard practice from the start.
[1] https://arstechnica.com/information-technology/2014/04/tech-...
The practice of using variable sized structures that began with type and size info and ended with a char[1] was commonplace.
https://hex-rays.com/blog/igors-tip-of-the-week-94-variable-...
Good True C Programmers had guard rails | canary bytes | etc. to detect and avoid actual buffer overflow (into unallocated memory) rather than technical buffer overflow (reading|writing past the end of a char|byte array).
I think that the legion of memory bugs which still occur in C/C++ programs are proof of one of two things:
1. We (the industry as a whole) do not actually know how to make these programs memory safe, or
2. Knowing how to make programs memory safe in C/C++ is not sufficient to prevent memory safety issues.
Either way, it seems clear that something needs to be done and that the status quo in C/C++ programming is not enough. I'm not saying Rust will be the right answer in the end (I do like it, but there's a ton of hype and hype makes me distrustful), but I can't fault people for wanting to try something new.
Thats mostly a joke. But AI-hardened-C seems like it could be much better than current-human-only-C.
Bash C/C++ all you want. I'm happy to keep using it to my advantage.
I'm not dissing C or C++ in any way. I've used it. But I recognize there are some major footguns that aren't easy to avoid, causing a much longer learning curve than necessary to get things built. Rust at least seems determined to address them, good or bad!
I use emacs(and vim), make and Boost's b2 build system for most of my programming. Although on Windows, Visual Studio is a joy to use. On Linux I use gdb. Works fine. I also use static analysers and valgrind. But I come from a tradition of Unix and living on the command line.
I've tried CLion, because I pay for IntelliJ IDEA for other programming (I also have to write Javascript, and Python) But while its nice, there is nothing there that I couldn't do without.
If you stick to C++ standard libraries, Boost, and turn on all warnings, and are reasonably competent, you won't encounter any bugs that are so serious that your program crashes inexplicably.
Also the lowest hanging fruit in C would be adding the ability to box and unbox fat pointers to objects.
Your priority should be to learn how to build better software and not force a new language onto people.
do you remember the age old saying about nature and fools?
id be excited if this gets solved. rust is a lot more comfy for higher level kernel stuff.
1. Create a tool that scores code depending on translation difficulty.
2. Automatically translate all code that can be directly translated.
3. Refactor the remaining C code into C or C++ code that is easier to translate.
4. Create a tool that suggests a translation but have a human review the changes.
5. Finally refactor everything else by hand.
You're welcome, DARPA.
https://support.apple.com/guide/security/memory-safe-iboot-i...
In Rust you presumably do all of that with raw pointers, which leaves you with a big unsafe mess to clean up over time, and I imagine a lot of the hard work of this project is trying to minimize that mess. But at least the mess that you have is recognizably Rust, and incremental cleanup is possible.
It is the systems-level code with UNIX heritage that is the main problem.
Thus the push for C and C++, alongside security certifications, where those languages feel like using Ada with a C like syntax.
https://www.perforce.com/blog/kw/NASA-rules-for-developing-s...
https://yurichev.com/mirrors/C/JPL_Coding_Standard_C.pdf
Nowadays we live in a world where developers refuse to pay for their tools like other professionals, but hey, Rust is free beer, not like the several millions per seat licenses used by Ada vendors, of whom there are still 7 vendors in business.
Why can't AI learn to do that? Its reward could be getting past the initialization and getting to the default state of the driver. It could be trained on hundreds of GPU drivers, not only for the minutiae of how to load values into the control registers, but the bigger picture of what it actually means.
Because it isn't magic.
>It could be trained on hundreds of GPU drivers
Do you know what an AI trained on hundreds of books looks like? Even with millions of books it can not write a coherent chapter, much less an entire book.
This is a genuinely terrible idea. It is exactly the thing AI is bad at, high degree of accuracy over long stretches of output.
So graphics drivers are magic? They weren't designed in a determinate way, with a known interface and specifications?
LLMs are bad at exactly those things which you need to make a GPU driver. Extremely high accuracy over long distances. AI totally falls apart when trying to write a novel, how could it write a GPU driver?
>They are excellent at stealing artists' content.
AI doesn't "steal" anything. It is matrix multiplication. AI companies are exercising fair use to create derived works using matrix multiplication. Not only is it obviously fair use, it is also what every other artist does.
>I don't think it will be long for them to steal content from authors.
Most authors who have digitally available works have almost certainly have had their works used as training data.
By your logic every artist who has ever viewed another artists image are thieves.
Also intellectual property isn't real. Copy right needs to be abolished. Most artists are terrible and should be replaced by AI.
So, they should solve total, automated testing first. Maybe in parallel. Then, use it for equivalence checks.
for Python - awesome
for golang - awesome
for JavaScript - awesome
for Zig - not awesome, AI doesn't get it, maybe training data set too small
for Rust - terrible - AI really doesn't get how it works, especially the hard bits
Your list reflects that.
What happens if an app you are developing gets translated to a language you do not know?
- Rust's compiler is 1.8 million lines of recursively compiled code, how can you or anyone know that what was written is actually trustworthy? Also memory safety is just a very small part of being able to actually trust code.
- C compiles down to straightforward assembly, almost like a direct translation, so you can at least verify that smaller programs that you write in C actually do compile down to assembly you expect, and compose those smaller programs into larger ones.
- C has valgrind and ASAN so it's at least possible to write safe code with code coding discipline, and plenty of software has been able to do this for decades.
- A lot of (almost all) higher level programming languages are written in C, which means that those languages just need to make sure they get the compiler and GC right, and then those languages can be used for general purpose, scripting, "low level" high level code like Go or OCaml, etc.
- There are many C compilers and only one Rust compiler, and it's unclear whether it'll really be feasible to have more than one Rust compiler due to the complexity of the language. So you're putting a lot of trust into a small group of people, and even if they're the most amazing, most ethical people, surely if a lot of critical infra is based on Rust they'll get targeted in some way.
- Something being open source doesn't mean it's been fully audited. We've seen all sorts of security vulnerabilities cause a world a hurt for a lot of people that came from all open source code, and often very small libraries that could actually be much easier to audit than lines with millions of lines of code.
- Similarly, Rust does not translate to straightforward assembly, and again would seem to be impossible to do given the complexity of the language.
- There was an interesting project I came across called CompCert, which aims to have a C compiler that's formally verified (in Coq) to translate into the assembly you expect. Something like a recursively compiled CompCert C -> OCaml -> Coq -> CompCert would be an interesting undertaking, which would make OCaml and Coq themselves built on formally verified code, but I'm not sure if that'll really work and I suspect it's too complicated.
- I think Rust might be able to solve some of these problems if they have a fully formally verified thing, and the formally verified thing is itself formally verified, and the compiler was verified by that thing, and then you know that you can trust the whole thing. Still, the level of complexity and the inability to at least manually audit the core of it makes me suspect it's too complicated and would still be based on trust of some sort.
- I still think that static analysis and building higher level languages on top of C is a better approach, and working on formal verification from there, because there are really small C compilers like tinycc that are ~50k LOCs, which can be hand verified. You can compile chibi-scheme with tinycc, for example, which is also about ~50k LOCs of C, and so you get a higher level language from about 100k LOCs (tcc and chibi), which is feasible for an ordinary but motivated dev to manually audit to know that it's producing sound assembly and not something wonky or sketchy. Ideally we should be building compilers and larger systems that are formally verified, but I think the core of whatever the formally verified system is has to be hand verifiable in some way in order to be trustworthy, so that you can by induction trust whatever gets built up from that, and I think that would need to require a straightforward translation into assembly, with ideally open source ISA and hardware, and a small enough codebase to be manually audited like the tinycc and chibi-scheme example I gave.
- Worst case everyone kind of shrugs it all off and just trusts all of these layers of complexity, which can be like C -> recursively compiled higher level lang -> coffeescript-like layer on top -> framework, which is apparently a thing now, and just hope that all of these layers of millions of lines of code of complexity don't explode in some weird way, intentionally or unintentionally.
- Best case of the worst case is that all of our appliances are now "smart" appliances, and then one day they just transform into robots that start chasing you around the house, all the while the Transformers cartoon theme is playing in the background while, which would match up nicely with the current trend of everything being both terrifying and hilarious in a really bizarre way.
PS: Java or even JavaScript are memory-safe :)
Definitly better than plain old C, but not what is being looked for here.
... and when done, please port SQLite too :)
Ada, is not popular enough, is my guess. To be fair, writing everything in Ada Spark would make code way more secure, simply because you'd need to write your pre-condition, invariants and post-conditions upfront, and prove they hold, but no one seems to want to think about lifetimes, let alone think about programming in more mathematical terms.
lol
The programmer gets to decide. DARPA does not expect the translator program to autonomously output a perfect Rust program. It just wants a "high degree of automation towards translating legacy C to Rust" (from the sam.gov link in the submission, emphasis mine).
Anything it wants... and that includes doing something entirely safe and reasonable.
If you write out of bounds, the compiler is allowed to shut the program down in a controlled manner. It's allowed to transparently resize the array for you. Etc.
Hence a rust translation can do these things.
If the compiler can, compile-time, detect that code is prone to memory corruption, it can warn the developer.
If it can't detect it at compile time, will and shall it add some sort of magic signal handler heuristic to determine whether a segfault occurred due to a runtime-provable specific instance of memory corruption and hence format your harddrive, while for runtime-indeterminable kinds it'd rather fry cpu core seven preemptively ? But that behaviour changes in the next version to blink sos on the network cable leds ?
I mean, it were cool if compilers used their "freedom" here to output nagging messages "the mem-safe UB brigade told you so, told you so, told you so ...". The fact they don't tells me, at least, that compiler developers follow Postel's law - be strict at what you emit but lenient at what you process. They're reasonable people. Not some sort of crusader out there to get you in the most excruciatingly painful ways. Undefined behaviour isn't unreasonable behaviour.
Consider the following program:
#include <stdio.h>
int main(void) {
int a[10];
a[20] = 100;
printf("%d\n", a[20]);
}
#![allow(unconditional_panic)]
fn main() {
let mut a: [i32; 10] = [0; 10];
a[20] = 100;
println!("{}", a[20]);
}
(In this particular case the error is simple enough that the compiler catches it and we have to tell it to go ahead anyway, but in more complicated cases it wont be. So please don't get hung up on this point)
I think Ada has a lot of technical merit but it's just not fashionable the way Rust is, for lots of uninteresting reasons.
In the early 1980s, microcomputer implementations such as UCSD Pascal were absolutely horrific in terms of performance plus missing the features you'd need to do actual systems programming work. In the middle of the decade you saw Turbo Pascal which could compile programs before you aged to death and also extended Pascal sufficiently to compete with C. But then you had C, and the three-letter agencies were still covering up everything they knew about buffer overflows.
It could have pushed for more Pascal adoption.
Then again, Borland also did their own mess, when decided enterprise should be their customer focus, not the small developer communities.
Would you mind explaining to a dev that doesn’t know much (anything) about Rust, how does this settle any debate?
It's a concatenation of several logical fallacies in a row; equivocation, straw manning, binary thinking about safety, several others. It's hard to pick the main one, but I'd go with the dominant problem being a serious case of binary thinking about what "safety" is. Of course, if the commentor is using anything other than Idris for all their programming, they're probably not actually acting on their own accusations.
I'm sure the Idris compiler has bugs somewhere too. If the OP actually programs, they are violating their rationale (I'm quite sure assembly or assembled binary aren't ok either).
From the Miri github: > Miri is an Undefined Behavior detection tool for Rust.
> The preferred approach is to use “safe” programming languages
“Safe”. Terms and conditions may apply.
In other words, from "safer languages are necessary" it does not follow that "safer languages are sufficient".
Yet, the linked repository shows a huge list of cases in which simple, documented use of Rust can cause Undefined Behavior (a.k.a. 'UB')
Pretty much every argument of Rust advocates against C/C++ boils down to either 'but memory safety' or 'but UB'.
Yet there are many convincing counter-arguments that boil down to 'but CompCert' or similar, and, as the linked repository shows, there might be at least some truth in there?
Also, many people cite things like Cargo as a reason to prefer Rust over C and C++, as well as other things. UB is a big part of it, of course, but it isn’t the only thing.
No, but there are a lot of people claiming that Rust cannot ever have any problems.
Just look at this thread. I merely linked to MIRI, and am currently at, like, -10 just for that.
Lots of people claiming that it just applies to 'unsafe Rust': is that true or not?
Regardless of anything else: can you, as a Rust community leader, please state clearly: is UB in generally safe Rust possible or not?
UB is not possible in safe Rust, by design. The root cause of UB is always in unsafe code. Miri is useless if your code is 100% safe Rust.
The only exception to this is bugs in the compiler, of which there are a few. They’ll be fixed.
You're available as an expert witness to that fact?
Because, eh, well, in at least one of the Rust-related situations that I'm involved in right now, someone might soon very well require the services of a person both as wise and reluctant-to-offer-any-kind-of-compromise as yourself...
In any case, I'm truly done here, in all senses of the word, but I still I wish you and your acolytes the absolute best.
Let me be even more explicit than steveklabnik here. If your code, including any libraries you link to, is 100% Rust and free of any unsafe blocks, then (barring compiler bugs) it is impossible to execute undefined behavior. If your code has an unsafe block, then it is possible execute undefined behavior. Note that it is possible for safe code to execute undefined behavior, IF there was an unsafe block that did an operation that requires the programmer to promise something was true that was not true.
For example, there is an unsafe method that will let you convert a pointer to a reference with an arbitrary lifetime. If you wrap that in a safe function, you can return a reference to an object whose lifetime has ended, and cause undefined behavior in attempting to use that lifetime--the attempt can even be outside the safe block. But were that unsafe block that upgraded the lifetime not present, then you couldn't cause the later undefined behavior to happen.
In short, an unsafe block is where the compiler can no longer guarantee that the conditions that prevent the ability to observe undefined behavior are present, and it is up to the programmer to ensure that these conditions are met, and even and especially ensure that they continue to be met after the unsafe block completes. I do worry that too many programmers are blasé about the last bit, and it sounds like your coworker may fall into that category. But Rust has always maintained this principle.
a) There is no such consensus. The actual consensus is that even if Rust solved all problems, it would not be financially feasible to rewrite pretty much any substantial project.
b) While Rust does solve many problems, it is nowhere close to solving all safety, otherwise there would be no `unsafe` keyword. Alas, fully proving safety in an impure, turing-complete language is mathematically impossible.
c) The only reason you would think that there's some sort of woke Rust lobby, is if you spend way too much time subjecting yourself to opinions of literal sixteen year olds on twitter.
No, that’s not the consensus. This is a strawman.
This is obvious example of strawman. Why are you doing this?
However, as evidenced by the arguments and voting in this thread, Rust proponents will take no criticism, whatsoever.
I linked to a GitHub repository that documents many, many instances in which generally safe Rust causes UB.
The same kind of UB that recently hit one of my coworkers, caused a 3-day outage and now (despite all my counseling to the contrary!) will burn them out permanently.
My only request: can you guys please back off just a little bit? Programming is already hard enough without the purity wars you're stoking all the time...
how often do we hear something like "C and C++ are horribly flawed and completely unsafe. it's basically a crime against humankind and gross negligence to use them"?
i get weary of that kind of thing too. i wouldn't approach it by reacting in the same way as the GP comment, but i get it. and it's not really that much of a strawman. it's more exasperation and sarcasm.
personally, i'm very interested in rust. but everytime someone at best "overhypes" it or at worse, outright dogs on other languages, it's a negative point toward dealing with the whole rust ecosystem.
I do, however, see people trot out the oft-repeated "rust evangelists want to rewrite everything in rust" or "rust people say programming C++ is a crime against humanity", but it seems to me that's the only place I see this argument. In other words, it's a simple strawman.
People can, in the most neutral way possible, point out facts about how safe or unsafe Rust is compared to C and C++. People will STILL complain about how the Rust zealots are bullying their language. This is how it plays out every time.
You can look at this thread. The “exasperation and sarcasm“ is stupid and one-sided. “But” they always say “that’s just a reaction to a previous debate”–because the Rust zealots are always in the rear-view mirror, never in front of them.
How about complaining about something in Rust… that is bad? Like how un-ergonomic Async is? Or how pointy and awkward the syntax can be? Instead they choose to fight the losing battle over how C and Rust are equally unsafe or how actually Rust’s safety doesn’t matter, depending on the phase of the moon. Then they whine about tone and zealotry when they realize arguing against Rust safety from the C and C++ side is a losing battle and they have run out of arguments.
This does not settle the debate because unsafe is rarely needed for a typical Rust program. In addition, the presence of an unsafe block also alerts the reader that the set of possible errors is greatly increased for that part of the code and more careful auditing is needed.
It’s a little like saying traffic lights are useless because emergency responders need to drive through them sometimes, so we should just leave intersections completely unsignaled and expect drivers to do better.
Rust is by default restrictive and requires you to explicitly make it unsafe, C/++ are by default unsafe and require you to explicitly make them restrictive.
What on Earth do you mean?
Getting people to adopt a new language can be a lot of work. I remember people claiming they missed headers files in Swift so they wanted to stick with Objective C.
That documented use of safe Rust can easily lead to UB, which this infernal 'internal compiler representation' demonstrates.
I'm not even sure what is even remotely confusing about that?
> ... detect unsafe code that fails ...
Show me the documented safe Rust code that causes UB without using any unsafe blocks outside of the standard library.
So, the reason I posted my original reply, is that at one of my $DAYJOBs, we recently had a 3-day outage on some service, related to Rust. Something like using AVX to read, like, up to 7 bytes too many from an array.
Nothing really major -- we have a 10-day backup window, and the damage was limited to 4 days, so we were able to identify and fix all identified cases. But the person-to-Git-blame for this issue happened to be one of my mentees, and... they were blown away by it.
As in: literally heartbroken. Unable to talk about it. "But the compiler said it was okay!", crying. One of my coworkers pointed at MIRI, which correctly warned about the issue-at-hand, at which point I recommended incorporating that tool into the build pipeline, as well as (the usual advice in cases such as this) improving unit tests and focusing on X-1 and X+1 cases that might be problematic.
To this day, I'm truly worried about my mentee. I'm just a C# wagie, and I fully accept that my code, my language, my compiler, and my runtime environment are all shit.
But, as evidenced by my experience and supported by the voting in this thread, it seems that Rust users seem to self-identify with the absolute infallibility of anything relate to the language, and react quite violently and self-destructively to any evidence to the contrary.
As a community leader, do you see any room for improvement there? And if not, what would it take to convince you?
Conversely, if you never encounter bugs in a component, it means it's not being improved fast enough.
This would require using unsafe code.
> As in: literally heartbroken. Unable to talk about it.
I would hope that this person improves as an engineer, because this isn't particularly professional behavior, from the way you describe it.
> "But the compiler said it was okay!"
Given that you'd have to use unsafe to do this, the compiler can't say it was okay. It sounds like this person may not fully understand Rust either.
> it seems that Rust users seem to self-identify with the absolute infallibility of anything relate to the language, and react quite violently and self-destructively to any evidence to the contrary.
I don't see how this generalizes. You had one (apparently junior, given "mentee"?) person make a mistake and respond poorly to feedback. You also barged into this thread and made incorrect statements about Rust, and were downvoted for it. That doesn't mean that Rust users think everything is perfect.
> As a community leader, do you see any room for improvement there?
I do think sometimes enthusiastic people who don't understand things misrepresent the thing they're enthusiastic about, but that's a human problem, not a Rust problem. I do not think there's a way to fix that, no.
I think the story is just somebody who didn't actually learn unsafe Rust properly (and I'm struggling to give it the benefit of the doubt, as it sounds quite exaggerated; I couldn't imagine a novice Rust dev literally crying because they thought unsafe blocks couldn't cause UB. If you were that emotionally attached to the language, I'd expect you to have learned what unsafe means).
Sure, not being able to do basic things like IO might be a bit of a limitation at first, but, that's all worth it, I guess?
Again: I'm pointing out to you that your absolutist stance on 'unsafe' and 'UB' is doing more harm than good.
You continue to choose to ignore this, which is your right. But as a "community leader" you could and should to better. As could I, I guess, by simply ignoring you, but, the mental health issues I see you cause in real-life make that sort-of hard...
The points are:
* Unsafe Rust is required to uphold specific guarantees to not cause undefined behavior. This can be tricky, but it's not impossible, it just involves a lot of care and some tooling like Miri for those specific situations. The situation is the same as pretty much the entirety of the C and C++ languages, plus Rust reference safety.
* Safe Rust is designed to not cause any UB on its own. It can only "bleed" UB from incorrect unsafe code. Without any incorrect unsafe code, this is easy to work with and involves much less work and care.
* Therefore, keeping your unsafe blocks small and in dedicated crates where they can be individually tested increases the quality and reliability of the codebase.
Surely you can see that it's an improvement over the previous status quo. I don't know what absolutist stance you're talking about. Most Rust fans I know, including myself, accept that Rust is an imperfect language, representing an improvement over C and C++. It's not just hypothetical either. Rust has brought demonstrated improvement in reliability for us, and for some of the biggest companies in the world who now lean on it to reduce their rate of defects.
* like un-unified representation of nullable reference types and structs under generics for example, or just the weight of features over the years, still makes most other alternatives look abysmal in comparison
What is shit about those things for C#? That’s the application programming language that seems to get the least flak out of all of them.
If I’m using an alpha or beta compiler, I might suspect a compiler bug from time to time… not really when I’m working in a decades-old, very established language.
So they fixed the biggest issues (at least it has value types), but it has nullable classes, collection types are mutable, integer overflow doesn't trap, it doesn't have nearly enough program verification features (aka dependent types), etc.
Worst of all it was written by enterprise programmers, who think programs get better designed when you put all their types four namespaces deep. I assume whoever named System.Collections.ArrayList keeps everything in their house in one of those filing cabinets with the tiny drawers.
Those are bugs. They are faults in the code. They need to be fixed. They are not UB-as-a-feature like in C/C++. “Well watch out for those traps every time you use this.”
This is like getting mad that a programming language boasts that it produces great binaries and yet the compiler has a test suite to catch bugs in the emitted assembly. That’s literally what you are doing.
Rust has UB-as-a-feature too. They could have eliminated UB from the language entirely, but they chose not to (for very valid reasons in my opinion).
UB is a set of contracts that you as the author agree to never violate. In return, you get faster code under the assumption that you never actually encounter a UB condition. If you violate those contracts in Rust and actually encounter UB, that's a a bug, that's a fault in the code. If you violate those contracts in C++, that's a bug, that's a fault in the code. This is the same in both languages.
It's true that Rust UB can only arise from unsafe blocks, but it is not limited to unsafe blocks. Rust UB has "spooky action at a distance" the same way C++ UB does. In other words, you can write UB free code in Rust, but if any third party code encounters UB (including the standard library), your safe code is now potentially infected by UB as well. This is also the same in both languages.
There are good reasons to favor Rust's flavor of UB over C++'s, but I keep seeing these same incorrect arguments getting repeated everywhere, which is frustrating.
Tell me what I wrote that was incorrect. I called them UB bugs in the standard library. If they were trivial bugs that caused some defined-behavior logic bug while used outside of the standard library then it wouldn’t rise to the level of being called an UB bug.
That's the part that's incorrect. That, plus the implication that UB is a bug in Rust, but not in C++. As I said, the existence of UB is a feature in both languages and actually encountering UB is a bug in both languages. You can play with the semantics of the word "feature" but I don't think it's possible to find a definition that captures C++ UB and excludes Rust UB without falling into a double standard. Unfortunately double standards on UB are pretty common in conversations about C++ and Rust.
Do you think UB-as-feature is something that someone would honestly describe C or C++ as? It’s a pretty demeaning way of framing things. Indeed it’s a tongue-in-cheek remark, a vhimsical exaggeration/description of the by-default UB of those languages which was added to the end of the completely factual description of the role that finding UB in the Safe Rust subset of the standard library of Rust serves.
Of course one cannot, from the Rust Side so to speak, use tongue in cheek, off-hand remarks in these discussions; one must painstakingly add footnotes and caveats, list and mention every trivial fact like “you can get UB in unsafe blocks”[1] or else you have a “double standard”.
[1] Obligatory footnote: even though all participants in the discussion clearly knows this already.
Yes. That's how I describe it. That's also how Ralf Jung (long time Rust contributor and one of the main people behind Miri) describes UB in both Rust and C++ (although he says C++ overdoes it) [1]
The thing I edited out of my comment was "motte and bailey fallacy" because after reflecting a bit I thought it was unfair. But now you're actually trying to retroactively reframe as a joke.
[1] https://blog.sigplan.org/2021/11/18/undefined-behavior-deser...
Okay. Then I was wrong about that.
> The thing I edited out of my comment was "motte and bailey fallacy" because after reflecting a bit I thought it was unfair. But now you're actually trying to retroactively reframe as a joke.
What a coincidence. I had written on a post-it note that you were going to pull out an Internet Fallacy. (I guess it’s more about rhetoric.)
I guess you’ve never seen someone explain after the fact that they were being tongue in cheek (it’s not a joke, it’s an exaggeration)? Because jokes, sarcastic remarks are always clearly labelled and unambiguous? Okay then. I guess it was a Motte and Bailey.
This is correct, and it's hard to teach, and I agree that a lot of folks get it wrong. (Here's my attempt: https://jacko.io/safety_and_soundness.html.) But I think this comment is understating how big of a difference this makes:
1. Rust has a large, powerful safe subset, which includes lots of real-world programs. Unsafe code is an advanced topic, and beginners don't need to learn about it to start getting their work done. Beginners can contribute to big projects without touching the unsafe parts (as you clarified, that means the module privacy boundaries that include unsafe code, not just the unsafe blocks), and reviewers don't need to be paranoid about every line.
2. A lot of real-world unsafe Rust is easy to audit, because you can grep for `unsafe` in a big codebase and zoom right to the parts you need to look at. Again, as you pointed out, those blocks might not be the whole story, and you do need to read what they're doing to see how much code they "infect". But an experienced Rust programmer can audit a well-written codebase in minutes. It's not always that smooth of course, but it's a totally different world that that's even possible.
The only thing that comes to mind that this could be referring to are the open bugs at https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais.... Are these what you're referring to?
> this infernal 'internal compiler representation'
What makes MIR "infernal"?
> I'm not even sure what is even remotely confusing about that?
You posted a link to a tool that executes pure rust libraries and evaluates memory accesses (both from safe and unsafe rust code) to assert whether they conform to the rust memory model. It sits in the same space as valgrind. You left it open to interpretation with really no other context. We can be excused for not knowing what you were trying to say. I personally still don't.
And miri is a blessing. There even is a known case where someone found a bug in C by translating it to rust and then running it through miri.