Integrating “safe” languages into OpenBSD?
marc.info
marc.info
I get the same feeling on Hacker News about Rust. There is a point where well-meaning evangelism does more harm than good. I sometimes get the feeling people spend more time writing defences of these languages than they spend time writing programs in them.
It sounds and seems like a language worthy of more investigation on my part, but I have to agree that the militant, presumptive attacks on any language besides their Chosen One has left a bad taste in my mouth.
I don't think anyone is arguing that we should throw everything out, however for all my greenfield stuff Rust has been a welcome breath of fresh air. Yeah, there are some people who are a little too over the top but a large part of the support behind Rust is from those of us who've used it in production and found it to be a fantastic language and ecosystem.
If you can write things from scratch in a clear room fashion, perhaps.
C++ is very good, when you need to integrate a lot of 'low-level' libraries (that are alredy in C or C++) together in a meaninful way, like for instance, browsers do.
Like the capacity to 'talk' with FFMPEG, levelDB, linux, etc.. in a complete native fashion basically.
I think in that regard Rust is sort of late to the party, and will find a wall hard to beat down, once the adoption rate get some maturity.
If you are already a experienced C++ dev, compared to modern C++, Rust has very little to offer. Of course is more modern, (maybe) more ergonomic?
But little things like package manager or explicit lifetime management by itself wont cut it.. at least not for people with large codebases already in C++.
When you are younger, programming languages look like that secret ingredient that will turn whatever you do into a magical tool.. But experience tells you that nothing beats hard work or that you should understand the strenght and weakness of each language, and know when and how to use it.
So for the problem domains where C++ are being used, it will get hard to beat, because a lot of great software are already there, and can be integrated in a native fashion, with whatever you do on top of it.
And by the way, modern C++ is already secure enough, making some selling points for Rust, a little bit of cosmetic right now.
If you throw a language like Swift into the equation.. particularly i think they(C++ and Swift) form a lovely and unbeatable couple.
The reason I ask is just about every item you laid out I've had the opposite experience.
Talking with C/C++ is trivial with bindgen and clang. Embedding in C++ is likewise straightforward. I mean Firefox is one of the larger C++ codebases out there. The package manager is great and makes dropping in things much less painful.
On the security front I don't think you can compare the two, I can easily think of things like iterator invalidation that C++ just doesn't handle well.
The last big project I did in Rust ran on 6 different targets(x68 win32/linux/osx, android armv7/x86, wasm) targeting 3 different rendering stacks(OpenGL, HTML Canvas, Android Views) integrated deeply with a Lua interpreter. Having done cross-platform stuff like this in the past it was a build/toolchain nightmare. With Rust everything was straightforward and pretty darn painless.
I follow Rust since Graydon Hoare's initial vision for it. Which i love it when it first came out. The garbage collected Rust had so much potential, in my point of view.
Then when i started to follow it, it was chaging into this current encarnation trying to chase C++, and market as a better C++.
I've coded in it for a period of 6 months, i like it, i understand its value, but somehow with the explicit lifetime thing, it started to become even more painful to code in it, even compared to C++, where you still have to create a header file and a impl file, without a proper module system.
The first Rust vision, now was re-created even more beautifully in Swift.
I get it what Rust is, i think it has its value and its place, im just saying that it will get a time where it will be very harsh to Rust get more adoption, because, even if you like it, you just cant get out of the great ecosystem all coded in C and C++. Interfacing with C for other languages is pretty easy, but you still have to wrap C++ interfaces, and use just some features, because you lost access to the whole thing (think LLVM for instance).
So C++ has this net effect, which it achieved by the virtue of all this amazing software ecosystem, where giving its talking its native language its much easear to interact with.
Had you said Poney I'd have agreed, but I don't see how Swift matches the first Rust vision (a safe concurrency model).
Rust toolchain still fails support on mixed mode debugging.
With C++, I can have a mixed .NET, JS and C++ solution and without problems debug and single step across all languages, regardless of targeting desktop, server or UWP.
Similar applies to Java or Android tooling.
Additionally if one is into native desktop coding, bindings to Gtk and Qt are still work in progress, and there isn't nothing that would match something like Blend.
Finally, the borrow checker still hinders some programming patterns that are quite common on GUI code, a few of them will be tackled with NLL.
Ah, and cargo does not support binary libraries although rustc can handle them, which is an issue for companies selling libraries.
I am aware all these issues will be eventually fixed, it is just as I see the situation currently.
I'll also counter that having worked in a fairly math heavy 3D graphics space I've never needed them, although our requirements may be different.
But there are a ton of domains where C++ doesn't have anything special to offer and you could just as reasonably write the code in rust.
There is not a single large-scale network-facing widely attacked piece of software I'm aware of in C++ that has not fallen to some memory safety problem. Memory safety issues frequently produce RCE.
Sorry. In hindsight, and out of the context like this, i think i picked up harsher words than i should. I dont meant to be disrespectful to all the great and valuable work that is going on the Rust community.
> There is not a single large-scale network-facing widely attacked piece of software I'm aware of in C++ that has not fallen to some memory safety problem. Memory safety issues frequently produce RCE.
Sure, Rust will be better at this. No doubt. But lets not forget that a lang like C++ needed to evolve while still working, so how much of those security issues are there given the use of ancient code practices of the past? software coded in the 90's even the 80's. I guess we at least should wait for when Rust has millions of tools, and billions of line of codes running in production, to see what sort of issues might plague Rust code the most. But Rust will need to get there first. So lets not forget that its a language created in the 70's and that is responsible for great tool, and is still being used to build big project, with millions of LOC and with many devs working in groups in some very sophisticated pieces of software, and this is no small feat.
What i meant to say, earlier, that maybe is not so clear, is the choice is not as unidimensional, by just cherry picking one point of view when you need to decide what kind of tech, fits the best for a given scenario. Im trying to point out that other languages like C++ or even C, still can be good bets, when you look into more vectors of influence, not only in the security aspect, or what language have more FP idioms in it. In the real life, is not as easy as some Rust evangelists imply it is.
C++ its not as good as Rust in the security aspect, theres no doubt about it, but the modern C++ is fine when you sum the vectors of what language will fit the best for a given scenario. I mean, you can be explicit about ownership, and make the api clear about ownership and lifetime. Of course Rust choose to be more pedantic about it. But as a C++ dev i have no issues with lifetime or owership management just by using smart pointers, std::move, etc.. (even in a multi-threaded environment)
But i dont think is as easy, as the Rust community is implying, that is to "just use Rust" for a project, where C++ may also be a good fit.
Even for new, clean room projects, i think C++ may be a better fit for certain scenarios, and im glad that theres also Rust as a choice. But i think sometimes this zealot type of evangelism sometimes more based in spreading fear, than by selling Rust for its own virtues, can hurt more than it looks like.
I guess by now, Rust doesnt even need to be compared to anything else, thanks to great hackers like you its already showing its own virtues. Im just trying to say that by now it probably reached a peak of users that can use the language in more low-level scenarios, and it probably should aim to get more of the Ruby, Python, PHP, NodeJS crowd, to fill its army.
Because from now on it would be harsher for Rust to get more adoption, without a big ecosystem backing it up (Like Swift has with Apple, Java with Android and C and C++ has with Unix)
But C++ was a mess from the very beginning, many of its problems are inherited from C, and the language itself was an ugly extension to C, not a carefully elaborated new language. So most new features were introduced to repair the breaches of such messy design.
Why to have nullptr dereferencing? Why to have an undefined behavior on option<> dereferencing? Why to have dangling refs? Why not to check borrow possibility statically? God, we don't event have a decent and safe variant (sum) type in c++. Adding all that (unnecessary) complexity c++ has (how many whatever-values and initializations does it have?). It is unsafe as hell, and lots of stuff could be made safer trivially in a new language (say, rust or F*).
>What makes it worse is that a null pointer in an option<> is handled as None instead of Some
What? How could NULL be treated as Some?
That's sorta funny that you use that example, since rust is a Mozilla project, and they've already moved some parts of Firefox over to rust, and plan to move more in the future. Clearly this is not an impediment; reasonable C/C++ interop was a must-have for rust's design.
> If you are already a experienced C++ dev, compared to modern C++, Rust has very little to offer.
That's a weird argument to make. Essentially you're arguing that you should never learn any new languages once you have proficiency in something similar? That would seem to be a bit short sighted.
> ... at least not for people with large codebases already in C++.
Sure. No one's saying "throw out all your C++ code and rewrite everything in rust" (and if anyone is, they can be safely ignored). But that doesn't preclude taking a look at some parts of your code that might benefit from being written in a safer language, or rewriting small tools in rust, or considering it for new projects.
> When you are younger, programming languages look like that secret ingredient that will turn whatever you do into a magical tool.. But experience tells you that nothing beats hard work or that you should understand the strenght and weakness of each language, and know when and how to use it.
No one's saying languages are magic, just that, by some metrics, some are objectively better. This might be an odd example, but I did Java for many years before switching to Scala a couple years ago. In the past few months I've had to go back to Java, and it's been frustrating that it's so easy to write certain classes of bugs that I just never see or write in Scala. Does that make Scala some magical tool that fixes all my problems and means I don't have to do any work? No, of course not. But it is objectively better than Java by some metrics (and sadly, worse by others).
> modern C++ is already secure enough
That's a pretty bold claim, and I'm certain it's not true. You might have a different idea of what "enough" is than I do, though. I'd believe that the "default" state-of-the-art C++ use is more secure than it was 10 years ago, but that doesn't mean that a language like rust can't offer superior safety guarantees.
And there's something to be said about encouraging safer practices through language features and convention. If you can make certain kinds of errors in C++ programs (I don't see pointer arithmetic going away any time soon, or people always using vectors and the like and never raw arrays), then you (or someone else) will, regardless if there are safer ways that help you avoid those kinds of errors. Given a language with a huge surface area like C++, and disagreements on what are the "safe" parts to use, it's inevitable.
> If you throw a language like Swift into the equation.. particularly i think they(C++ and Swift) form a lovely and unbeatable couple.
Despite Swift's open source status, I really don't see it making any meaningful foothold outside macOS/iOS.
I don't have a horse in this race. I abandoned C++ years ago, and I've only started learning rust a few weeks ago. I like it, but I have a lot to learn, and I've already found some rough spots that aren't so great. No tool is perfect.
*or fortran or pascal
At least for me, part of the attraction of Rust is the development tools around the language itself (e.g. cargo).
Seamless cross-compile. I've done a ton of cross-compile projects, cargo/rustup is the best by far.
One-line dependency import. Saves *hours* setting up build chains.
First-class win32 support. Hard to stress how rare and awesome this is coming from a gamedev background.
Integrated testing. So nice just to have.
Integrated benchmarking, ditto above.This 100 times over. It's a language that seems novel and strong enough to stand on its own merits - but there's a section of the community that makes me want to stay away.
I wish they'd spend less effort re-writing already working tools in C and more effort writing new, better tools. A drop in replacement for "ls" isn't that exciting for me.
Note, that the answer doesn't say that it can't benefit security, which would be a principal argument for OpenBSD. It brings side reasons as to why it's not easy to do (like integrated tools, developers' availability and so on).
Oh wait, that's right, none of them has even gotten around to writing a replacement for ls, grep, .....
And a good getting your feet wet project for someone else :)
Is this really a thing? Folks keep pointing out this bugbear of folks going out and asking projects to rewrite in Rust and ... there aren't that many examples of this.
If anything most of the more famous Rust projects (ripgrep, redox, etc) are clean-slate.
Personally, I don't find this a bad idea, as long as it is not the primary thrust. The reason it is needed is that you aren't going to convince everyone to change the default tools they've used for however many years. If you want to make things safe, you have to swap out implementations and keep same/similar interfaces.
That doesn't change the fact that Rust programs are, when memory safety is concerned, more secure than C++ programs. This is true both in theory and in practice.
Why would not knowing C++ bar people from anything? It's a wildly specific skill, diminishing in importance every day.
The game goes like this: start discussing something: say, including rust in OpenBSD.
- Assume it (here, using rust in oBSD) is a great idea.
- For each objection, explain why it is trivial. Your tools are your own reasoning and knowledge of programming, the ability to ignore, minimize or belittle any and all reasons why things are the way they are, assuming away any legacy issues, and remembering that all needs and use-cases in the world exactly match yours.
- When you get bored, just walk away. It isn't your problem, after all, and you were just trying to help. High-fiving your cohort optional.
It ends up simultaneously coming off as arrogant and clueless. Last time I had a small problem and the time to try something new, I started it in rust. After getting a series of jackass responses to a simple question[1], I wrote it in C, because I just don't care enough to put up with twits fluffing their own ego at the expense of newbies.
This problem is hardly unique to rust - unfortunately it happens everywhere nerds gather. But rust has a really bad case of it.
[1] Found the answer a few days later when I thought of a better search term.
[1] https://chat.mibbit.com/?server=irc.mozilla.org&channel=%23r...
Getting those responses where, exactly?
If you had mentioned literally any other language with a FLOSS community around it I wouldn't question that statement. However, Rust proponents so clearly claim to do the opposite-- you've even got Steve below explicitly stating that jackassery isn't allowed in Rust forums.
What's the data to the contrary?
I think I recall reading somewhere on Rust's website that telling project maintainers to switch to Rust was frowned upon, however I can't find it right now. Sometimes I feel like it should be written in a 20pt font on the home page. Or maybe written in blinking letters every time you run rustup.
I always feel it's very presumptuous and somewhat disrespectful to second guess the devs of a project, in particular if you're not a big contributor yourself. Pay your dues. These messages are effectively not a whole lot more useful or insightful than saying "Rust rulez, C droolz" or "have you considered indenting with spaces instead of tabs?".
At the same time, I don't see it actually happening as much as people say that it happens. Maybe I just don't see those things. If I did, I'd tell them to cut it out.
Language advocacy is far older than Rust. And many language advocates are far more obnoxious than what I read on HN with regard to Rust. I actually have the feeling that the C++ crowd is just a bit sad that the usual defense of "well, yeah, other languages do those things better, but the code they produce is far too slow for our problem, so they're useless!" doesn't cut it anymore.
Java is a great language in theory, the problem with Java is that people have evolved to use layers upon layers upon even more layers of abstractions, the tooling is horrenduos (try configuring maven vs npm) and especially all complex Java applications end up being as slow as molasses (e.g. SAP GUI, Lotus Notes, Eclipse, jDownloader, Vuze on the GUI application side).
This leads to people projecting their horrible experiences with the Java ecosystem and applications to the language itself, which is a pity.
vs Cargo (Rust) - http://doc.crates.io/build-script.html (simplicity)
vs elm-package (Elm) - https://github.com/elm-lang/elm-package (automatic semver compliance)
vs Stack (Haskell) - https://docs.haskellstack.org/en/stable/GUIDE/ with stackage-sets (packages tested to work well together) - https://www.stackage.org/
That's kind of a weird comparison since npm is a package manager and maven is a build tool, packager, dependency resolver, test runner, ...
A better (but still lacking) comparison would be "try configuring maven vs. npm+webpack", and then I'd say... oh god, I'll take maven in a heartbeat.
Also there are lots of similar architectures in the .NET world with WCF, Web Forms, EF, SharePoint, ...
Anyone that thinks JEE is bad, should spend some weeks trying to do the same with CORBA, DCOM or SOM, in either C or C++.
Same thing will happen to Go, JavaScript, Scala, or whatever language gets adopted into the enterprise cathedral.
Example "Lifecycles" [0]. The documentation says "Maven is based around the central concept of a build lifecycle. What this means is that the process for building and distributing a particular artifact (project) is clearly defined." The first sentence just says "this is important!". I have no idea what that second sentence is supposed to tell me. The documentation goes on without a clear definition what a lifecycle is. I figure it is a kind of command. Afterall, the basic lifecycles are "default" which is "make all", "clean" like "make clean", and "site" like "make docs". I'm not sure, there is no good definition of "lifecycle" in the documentation there. Hard to understand.
Well, there is a predefined set of lifecycles and each lifecycle has a predefined list of phases, where stuff can hook into. For example, with java the compile phase will turn my java files into class files. The test phase will turn class files into test reports. Recently, I wanted to merge multiple coverage reports [1]. Turns out, this is done in a compile phase, which is reasonable. However, the reports are generated in the test phase, which is after the compile phase. At this point, I'm lost. This corset of phases is restricting me.
[0] https://maven.apache.org/guides/introduction/introduction-to... [1] http://www.eclemma.org/jacoco/trunk/doc/merge-mojo.html
Gradle has all sorts of failings, and is overly complex in its own way, but at least it gets the meta-model right: provide fundamental tools for building a graph of tasks with dependencies between them, and ship a set of hopefully useful tasks built on top of that.
The reaction of some parts of the Java community to Hypertable choosing C++ instead of Java was nothing short of embarrassing for instance, but it's not just that, over the years I have seen too many examples to list :)
IME a lot of thin skinned people on the internet always view the former as the latter. It's equally as tiresome.
Pure fanboyism. Better than C yes, but not better and much slower and unsafer than ATS or Pony.
You nearly made me spray coffee all over my precious Microsoft Natural Ergonomic Keyboard. (Which - no joke - would have been the third I would have destroyed this way.)
Thank you for starting my day with a healthy dose of laughter! (I know that comic is trying to make a serious point, but the surreal example it chose is just so hilarious!)
Programmers love language wars.
Once your instincts adapt to something intimately (and programming languages require that); that's what feels right. You'll argue vociferously against a change and use what are objective reasons. (The Thunderbolt could dive faster, it was a brick.)
Of course, in the United States Army Air Force, you still had to switch.
The old hands usually remain convinced of the superiority of what they know, and irritated by Young Turks telling them something else, nicely or not. Because the Young Turks can't be right. But they are, often.
Programming Language Advocates love language wars
Most programmers just like to be aware of ALL the PROs and CONs of the languages and choose the right one for the task at hand.
But it will not stop _everyone_ from adopting better languages, and their efforts will eventually surpass the older, less secure systems.
When there's feature parity, each and every new exploit will be called out: "this wouldn't have happened in our system'. And that is a good thing. There _are_ better alternatives to C.
As for the point about how nobody is working on replacements, that's wrong. There's (partial) replacements for most coreutils written in rust, and a whole kernel has been under active development for years now.
The system being secure is a secondary benefit to it being comprehendible and coherent to an individual. It's not enough for the output of a magic box to be a better widget, even if the widget is better in every measurable way. The box itself must not be magic.
This isn't "right" or "wrong". It's simply a stance that values an individual's ability to understand their computing device from top to bottom.
Rust is an amazing and wonderful piece of technology, produced by a great investment of energy by brilliant minds. I'm incredibly grateful it exists and look forward to it's continued development and adoption.
But I'm not brilliant, and I don't have a lot of energy, and so I'm also grateful there exists an operating system I can understand, and other people who continue to work to make that operating system useful.
Much open source technology is created by people with a financial incentive for others to use it. That's fine. OpenBSD is written by people who want to use it. It's usefulness to others is coincidental. That's fine too.
I'm glad there is so much money funding the development of open and free computing innovations. The world is a much better place for it.
But there isn't a lot of money funding the rest of the unix philosophy besides the "open" part, like the parts about composability and anti-monolithic design. Because those things are not valuable in financial terms, and are potentially even destructive towards the purpose of capturing value at all.
I am grateful there is a small radical free operating system defending our freedom from monoculture.
And I'm similarly glad for their defacto antagonism towards proselytizing of all kinds, even if that proselytizing is done with good intention, and even if that proselytizing turns out to be right.
That's why you need to show them the code. When you show the code, it means you understand. Until you understand, you're not free.
At least, this is my interpretation. I'm not affiliated with the project in any way. Just a fan.
Writing correct, efficient C takes more years to master than people take to learn Rust that I've seen. Also, the C compiler and many other parts of OpenBSD are black boxes to their developers. Your worries apply equally to their situation unless you've read and understood all their dependencies. On top of it, the OpenBSD people are always rewriting stuff for claimed benefits in maintainability or security. It's just when we talk a safe, systems language that can be as simple as a Wirth language or complex as Rust they suddenly can't justify the effort of even piecemeal replacement.
Then, next week, they'll put piles of effort into a mitigation across their toolchain whose benefits are so probabilistic even they can't tell you what attacks will fail or succeed. It's worth it, though, to improve their security standpoint. Unlike the pain of recoding even one utility in something like Rust. That's where this email draws the line.
Of course, I encourage people to do exactly what he asks every time another BS argument is raised. He's worried about drawbacks of a non-C language? Make something like Cyclone or get a Wirth language better at selectively turning off safety compiling to C w/ great C FFI. He's worried about compile times? Fix the compiler. He says utilities aren't rewritten in the better language? Rewrite them showing its advantages esp against the bug reports in OpenBSD's tracker. Just keep pounding away at the problems until he runs out of excuses to import stuff or is extra clear they simply don't like language/method X for arbitrary reasons. Regardless, you get a pile of safe utilities/modules for OpenBSD to do useful things on top of fast, safe tooling. Win, win. :)
Alternatively, contribute those ports to OS's that want to bring in best-of-breed tooling for boosting safety and security. They're usually smaller, less mature, and need all the help they can get.
I wouldn't be too worried about Rust, it will have a niche at least as big as Ruby's is and IMO it will be way more entrenched since its target domain moves way slower and the barriers to entry are way higher than for scripting languages and web frameworks, IMO.
If it were that simple, the compiler wouldn't be slow in the first place. Rustc and ghc have both been too slow to be usable their entire lives. There doesn't seem to be any reason to believe they can be made fast enough to consider using.
This is a really interesting and compelling philosophy to me, but it’s the first time I’ve heard OpenBSD described this way! Why is this not mentioned on the project’s homepage?
2) Nothing wrong with kernel and base. It provides the kernel, clang, X.org, documentation and BSD-games, among the rest of Unix tools.
3 The drivers are in base and NOT in a module form, as the GNU/Linux crap with incompatible vendor releases and binary blobs tied to a version.
I acknowledge the strong pull to go and re-implement existing standards like POSIX in a new language like Rust.
But, having had to again deal with all the corner cases of signal handling and threads for a Linux application, I encourage any prospective Rust OS developers to pick a different path.
Please, please, let's move beyond POSIX. Give me a sane set of operating system API calls that work in a reasonable way together. And that can compose easily.
So that future developers don't have to deal with file read calls that may (or may not) be interrupted by a signal, and all that.
I do understand OBSD's objections to using a new implementation language for their core utilities. And I understand their philosophy for what they're doing and how they're doing it in the general case.
But the fact remains that POSIX, as it has evolved (or not, in some cases) makes writing correct programs harder than necessary. Especially modern multi-threaded programs.
I'd like to see something better and simpler. And maybe written in something other than C.
ChromeOS, the OS APIs are the web platform.
iOS, almost everything that matters is done via Cocoa
Android, Java reigns. There is very little UNIX exposed to userspace, less so since Google started clamping down NDK since version 7.
In general I think the "why don't you rewrite your project in $languageoftheday" remarks should be dismissed and ignored. Theo is entirely right when he says that if people think it's a good idea then they should spend less time bothering the devs about it and more time actually coding it.
You think Curl or Emacs should be written in Rust? Then do it. Don't bother the devs about it. If it's really better then people will make the switch.
If I decided to start practicing Formula One next week that won't automatically make me competition for Lewis Hamilton. Redox has some way to go before people can seriously consider using it over OpenBSD for real world applications.
I'm sure it will be good enough for that some day, but that day is not now.
>Such ecosystems come with incredible costs. For instance, rust cannot even compile itself on i386 at present time because it exhausts the address space.
Does he? He "stat[es as] fact" that
> There has been no attempt to move the smallest parts of the ecosystem, to provide replacements for base POSIX utilities.
which as xwvvvvwx notes is categorically wrong, then points out that rustc can't compile itself on i386, which is relevant… how?
Remember he is speaking as the leader of an operating system project. As in, a basic part of the project functioning normally is compiling the whole thing from scratch. If something needs cross-compilation to even get started it won't end up in OpenBSD base.
I seem to recall when it supported more architectures they made a public show about how they weren't going to cross compile even for targeting wimpy/sluggish machines, because recompiling the OS was a good stress test for the kernel itself.
Those lowest common denominator systems tend to find bugs not present elsewhere.
And stating that openbsd looks like performance art and security theater seems to indicate you haven't looked at what openbsd has done for security.
Yes, bugs are found at boundaries and interfaces, large or small. Something has to be different for the output to be different.
If someone writes a bug proof TLS implementation while at the bottom of the pool wearing SCUBA gear, it is still security theater.
Another point about the *BSDs, something doesn't have to be in base for you to use it. The base system is supposed to be small and not have a lot of dependencies. You are free to use things in ports and packages or compile them yourself. So this is not the same as never being able to use rust.
I'm actually gobsmacked this is the case. There used to be a saying, only half-joking, that a language that can't host/compile/bootstrap itself is nothing more than a toy. As others have more eloquently pointed out, it shouldn't have to be explained why people who write operating systems and compilers would consider that a no-go.
Note that Rust can (and does) target various 32-bit platforms (ARM and maybe RISC-V, not sure) for cross-compile. Self-hosting on a 32-bit platform is such a minor drawback these days. 64-bit ARM processors are becoming more common these days as well.
32bit is not even close to be being that old and lots of machines, applications or platforms require it and they need maintenance. Especially OpenBSD is a system that I as a Linux-fan recognize for supporting old hardware much better than Linux.
And it's not only 32bit x86, there is plenty of other architectures limited to 32bit, mostly from the ARM sector but I believe some older MIPS and other even more exotic arch's are supported.
If rust can't selfhost on x86 how can we expect it to selfhost on more exotic 32bit hardware that BSD supports?
So gobsmacked you apparently couldn't even begin to attempt answering the question but felt you just had to go on a rant as irrelevant as you believe it's righteous, uh?
> There used to be a saying, only half-joking, that a language that can't host/compile/bootstrap itself is nothing more than a toy. As others have more eloquently pointed out, it shouldn't have to be explained why people who write operating systems and compilers would consider that a no-go.
Rust has been self-hosted for almost as long as it's existed. The boostrapping OCaml compiler was left behind back in 2011.
Not on x86 which is what this whole conversation is talking about. So if OpenBSD used rust in base they would have to drop support for x86.
i386 is relevant because OpenBSD supports i386.
No, he is very explicitly saying that
> There has been no attempt […] provide replacements for base POSIX utilities.
Which once again is categorically false, a github repository purporting to do exactly that has been provided.
> i386 is relevant because OpenBSD supports i386.
i386 is supported, the issue is compiling the compiler on i386.
which is required for the system to be self hosting
seriously - what is being said is this:
" oh hey lets throw away the functional and perfectly good entire base set of utilities for this 1/2 complete project on github using a language that doesn't even natively build on all of our supported platforms and wouldn't even remove the need for a C compliler in base, and further complicate the base toolchain, not to mention breaking all kinds of other builds which use shell utilities expecting certain behavior, etc ,etc, etc, because somone thought it would be 'neat' to do this. And whyyyy aren't you taking me serously??? "
every few days (hours?) some noobish person desides to ask some fantasy question about whatever topic of interest they are noobing about on openbsd (and other OS) discussion lists, and then gets whiny when they are being called out for being 'green' about life itself. this is another of those cases, and I have no idea why it got crossposted here or upvoted.
You deliberately cut out the part which states he's talking about the ecosystem of OpenBSD, in an OpenBSD mailing list. That is an extremely disingenuous and uncharitable cherry-picked interpretation. He was categorically talking about efforts to port OpenBSD utilities to such a language and merge them into the project (i.e. the OpenBSD ecosystem). What you're suggesting he said is just plain FUD.
I can only imagine it works fine on dev machines with much faster quad+ cores and 64GB of RAM or whatever.
Just as an aside, it's done a lot to have the tablet be my primary "fiddle-at-home" machine: keeps me really conscious of resource limits, including ones I normally don't think of like screen size. (Most websites render terribly in landscape on a 10" tablet.)
The solution is to install 32-bit Linux on it. Then it won't suck.
Most applications are requesting hundreds of megabytes if not entire gigabytes, the system will swap to death after you open an app and a browser tab on facebook.
I remember a friend who bought a 2GB netbook, the thing froze to death whenever he opened just eclipse, he had to return it.
Performance matters. Even more than features.
Performance is like money: it's easy to squander and hard to acquire.
In this case, there were several decisions to not care about performance right now in order to emphasize correctness and shipping faster, while making sure there are no technical obstacles to making compilation faster in the future. The main time sink in Rust compilation is that all the abstractions in the Rust code get compiled into the initial bytecode representation passed to the LLVM side, and they are only reduced there, instead of cutting down on the hierarchies on the Rust side. This costs performance in several places -- the creation of all the bytecode, the copying of it, and then LLVM parsing it all in. The upside of doing it this way is that it makes the Rust-specific compiler much simpler and easier to implement, and that the optimizations that remove the towers of abstraction on the LLVM side are extremely well tested.
As Rust matures, optimizations that reduce the complexity of the created initial LLVM bytecode can be, and probably will be done on the Rust side.
Also, have you seen gameplay footage of in-development titles, particularly older ones before the days of Unity and UE4? Usually they're choppy as hell because the engine is under development at the same time as the game, and devs prioritize getting a golden but slow codepath working first so that the artists have something to go off of, and all the optimizations are shoved in to recover framerate in the months right before release.
To rephrase that a bit: OpenBSD is designed to be a system where any user on any platform can contribute to OpenBSD using the tools included in OpenBSD's default install. Deviations from that will almost certainly receive a cold reception at best.
The compilation phase can take hours in C++. Up to a day when compiling huge projects will all the optimization flags.
Live that for some time and it will quickly prove you that you were wrong. Compilation time matters.
Slow compilation: more than a minute (makes to start browsing HN, missing the end, thus losing even more time)
To have fast compilation even with big projects is hard. Go, C, and D are usually fast. Scala is usually slow.
I care about development builds primarily. The edit-compile-test loop must be really really fast. Optimization flags are irrelevant, because if performance matters you often must have them enabled for development as well.
But you do have a point. Things are so much better now than when I started programming 50 years ago. Machines and languages are so much better. Programming is a dream compared to back then.
(Appreciating as well that most incremental build gains come from avoiding unnecessary work, so they're as much the domain of the build system as they are of the compiler.)
Also, that optimizing compilation is important is no reason to not work on i386. This is a point Rust needs to fix. And not only i386 support, but also other architecture families, as host.
All the heavy API and computation libraries are wrappers are C binaries that are optimized to death.
Not many people are whining about our C compiler toolchains not fitting into our microcontrollers.
For example, the NetBSD project has a dreamcast port, but like most of their ports, it is crosscompiled. The last time I tried the port it would crash when put under high load for a while and would kernel panic when you tried to play audio. The netbsd dreamcast port is not functional in the sense that openbsd would like to enforce, something which is not relevant to netbsd and in no way denigrates them, but merely serves as an example.
Trite point? OpenBSD supports 1386[1] if they pull a rust compiler into base and start rewriting things in rust, then they can't support 1386. Dropping a supported platform is not a "trite point".
That is a choice they are entitled to make, the trade-off being it would appear to make most modern technologies a poor fit for adoption in OpenBSD - that's the price they have to pay. It is a problem of their own making.
...which is exactly the kind of a system one needs for production environments.
So you could have a scenario where the i386 binaries would have to be cross compiled from a 64-bit machine but the end result would work just fine on those machines.
I wasn't aware that OpenBSD's objectives included having each arch be able to build itself. Totally reasonable goal. It's harder to do "dreamcast port"s in those scenarios, though.
[0] https://www.amd.com/en-us/products/server/opteron
[1] https://www.thinkmate.com/systems/servers/rax/amd#browse
Consumer machines, AFAICT, are all amd64 (or x86_64, if you prefer that name). I understood the original post to mean i386 == x86, and I agree — where do you even find an x86 today (for sale, in a non-niche use case, i.e., "pretty easy")?
On a related note, we will eventually be running development tools on microcontrollers. Not that little 16bit parts will run the tools, but that 16bit parts are going away. In price sensitive areas this will not happen, but for things with a larger budget why not run the tools right on the target? If your controller is an RPi why not use it for development?
It includes 486, Pentium, etc.
> the 80386 instruction set, programming model, and binary encodings are still the common denominator for all 32-bit x86 processors, which is termed the i386-architecture, x86, or IA-32, depending on context.
That would be great! Get back to me when it exists, instead of trying to dissuade an already established project from one of it's primary goals and handwaving said goals as "outdated" and "unimportant". Heck, it's open source, so if someone feels like it, they can just fork and go hog wild!
If rustc cannot even build itself on i386, what kind of support can we expect for other platforms with an even smaller user base?
On a project such as OpenBSD they cannot suddenly drop platforms and only support amd64 as portability is one of their main "selling" points. Furthermore, that would also mean to lose the developers that are interested in these alternative platforms and probably chose OpenBSD because of the platform support. Furthermore, such developers are usually not only contributing to platform-specific parts, but also to system utilities and ports.
For the full list of platforms, see https://www.openbsd.org/plat.html
Can confirm, spent many happy hours hacking on an AlphaStation running OpenBSD!
They… don't need to? That rustc can't compile itself on i386 doesn't mean you can't ship a rustc for i386, it just means you have to cross-compile it.
Imagine as a developer who compiles base from source you had to find another system only to compile rustc and then transfer it to your machine. And you would not have to do this only once, but for every compiler bug fix coupled with the overall rapid evolution of Rust. I think many in the OpenBSD community would oppose such an approach, even without further considering other aspects such as security implications.
You always need an external system for bootstrapping, you're not assembling the base C compiler with which you're compiling everything else. At one point you need to obtain a compiler from somewhere else.
> In OpenBSD there is a strict requirement that base builds base.
i386 might not be as popular as it was on 'normal' OS, but I wouldn't be surprised if OpenBSD had a lot of people still using it.
I'm a big fan of what Rust promises, but the solution is not that OpenBSD changes its policies or that OpenBSD drops i386. Rust should become self hosting on i386.
These aren’t just philosophical comments, the implications for him and OBSD devs is to spend massive time developing these things. Dropping a supported platform and taking on a huge investment of effort needs serious justification.
I've seen i386 multiple times to also refer to any x86 Arch.
> All CPUs compatible with the Intel 80486 or better,[0]
$ uname -a
OpenBSD hostname 5.9 GENERIC.MP#6 i386Is cargo supported on i386 platforms? Also Rust complies itself, afaik there is no way to compile Rust/Cargo but to use previous version of it. If one of the past builds of Rust is backdoored, any version between then and now is backdoored, language is safe, environment... as safe as it was never compromised. OpenBSD compiles everywhere where C code works, Rust/Cargo works where it's supported and it will takes decades to catch-up on some architectures.
Rust absolutely works on 32-bit platforms, though we often use the i686 target rather than an i386 one. Platform support list is here: https://forge.rust-lang.org/platform-support.html
Theo is talking about building rustc, not compiling most Rust programs. That's the first distinction that it seems like many people are missing.
However, apparently the compilation process OOMs when building on a i386 box. I don't use those platforms, but I'd believe it. The Rust compiler is large. However, I thought (and looking at our CI, this seems to be true https://travis-ci.org/rust-lang/rust/jobs/311223817) we do compile with an i686-unknown-linux-gnu host (for this build), so I dunno. Maybe it was a fluke, maybe I'm misunderstanding, I'm not sure.
We often provide artifacts via cross-compiling, but this is unacceptable to OpenBSD. That's totally okay. They have good reasons for doing this.
The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc).
Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.
That bug really bites hard any code heavy on iterators (Rust often praised feature!). It has reliable reproduce test-case, but still it's already year old and was down-prioritized!
Hard to believe anybody uses Rust for real large project given so little attention to crucial details.
> I'm going to lower this from P-high to reflect reality. I'm still eager to investigate but haven't had time, and making it P-high is not helping =)
P-high means someone is actively assigned and working on it, so yeah in some sense this is a down-prioritization, but only from "Someone is working on this, so put your work somewhere else" to "this is open to work on"; the de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy.
So, "nobody wants to deal with" feels like a mischaracterization to me here.
> The de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy
And Niko put "medium" priority month ago :)
It does sound kind of bad for Rust, but the competition isn't doing much better :p (weak excuse, I know)
That's not to say either one is "worse" or "better", but comparing the two on an axis like platform support is like comparing tomahawk missiles and tall ships. Totally different requirements and use cases.
I still thought that we generally kept it down to around 2GB of space, but maybe that's wrong.
I may be remembering the details a bit wrong, but it was a good read.
https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...
https://en.wikipedia.org/wiki/Trusted_Computer_System_Evalua...
One compiler made to the highest standard in development assurance is CompCert.
It has specs of everything it does, proofs it does it in tool with minimalist checker, extracts to ML that can be compared against those specs in various ways (eg visually), can optionally be compiled with a mini-ML compiler or Scheme you homebrew yourself, and passed exhaustive testing by third party with only a few bugs in specs (not code). There's another using formal specs called KCC which could be ported to something bootstrappable like META II or Scheme.
The other requirement from TCSEC was that source be supplied to customer to build with their onsite, trusted tools. I looked into even having compilers done in Perl since it's already widely deployed. David A. Wheeler made brilliant suggestion of either bash or awk. I have put tools for those and more on rain1's bootstrapping page. rain1 or someone there called the concept "Ubiquitous Implementations." Note we've focused on technical content, not presentation, on that one being busy folks. Looks rough. :)
http://bootstrapping.miraheze.org/
You also need repo security to protect that source with it either cryptographically sealed and/or sent over secure transport. Link below on repo security from David A. Wheeler. Quite a few forms of transport security now.
https://www.dwheeler.com/essays/scm-security.html
After Thompson wrote on Karger's attack in 1980's, it took a life of its own among people that continue to mostly ignore the prior solutions. It's a problem absolutely solved to death starting with the person who discovered it in MULTICS Security Evaluation. Far as state-of-the-art, the current path of research is exploring how to integrate solutions for many languages, models, or levels of detail in one picture of a system with no abstraction gap attacks with proof of that for all inputs. That's a bit more complex but just an imperative language to assembly delivered and bootstrapped? Straight-forward but tedious, time-consuming work the first time it's done. :) Also, expensive if you buy CompCert which is only free for GPL stuff. Two of us are eyeballed CakeML's lowest-level languages as a cheat around that for verified bootstrapping.
EDIT: Btw, all that is technical discussion and argument. For fun, you might enjoy the story "Coding Machines" which is about only coding-related story I started reading and couldn't put down. Probably took an hour to read. It covers discovery of a Karger-Thompson-style attack along with how people might respond mentally and in terms of solutions. Some other stuff in that one.
Does he really not know this or is he ignoring them to make a point?
Only useful when you have mutable environment, most build spaces don't have it because it's insecure. So it's useless for big projects with external dependencies, you HAVE TO download them on each and every build.
>your build is broken": you only need to depend on external github references
Go projects use not only github repos, there is gopkg, gitlab and some others which I don't remember. All of them must be online and works fast, any lag will delay whole build system, which in many cases is pipe-lined. I can't imagine anyone interested in stability wants this.
The project in linked repository.
[1] - https://github.com/ericlagergren/go-coreutils/blob/master/xx...
[2] - https://www.openbsd.org/policy.html C-f GPL
There are already "better" and "safer" languages that could easily replace the entire unix userland. Heck, awk can do half of it, but all of that C code is already written...
And C needs C, or for more recent compilers, C++ needs C++. You just don't notice it because these two languages are already part of the base system.
So, bootstrapping matters to me, and the bootstrapping story for anything involving LLVM isn't very good (I'm actually having some problems with a language other than Rust in this regard).
Interpreters you can generally build fairly easily from source (Lua, Ruby, Python, Tcl, for example). But when it comes to compilers, bootstrapping is generally a much bigger obstacle. There are a few notable exceptions that only require a C compiler, no internet access during their build, and build in an acceptably short time:
* OCaml bootstraps from C via a bytecode interpreter, which is then used to build the native compiler. On my laptop (using four cores), I can actually do that in under a minute (two minutes if I want flambda).
* Nim compiles to C and hence builds in half a minute on the same machine from C sources.
* LuaJIT builds in a few seconds, assuming a dynamically typed language with a JIT compiler is sufficient for you.
most any scheme/lisp compiler as well
requiring internet access and binary bootstrap toolkits thing for the core piece of an ecosystem (e.g. the language compiler) is decidedly new-school, rooted in overly commercial projects, and for the worst, imho.
ps: lawn.
Every time you add a new language feature you increase them time to write your bootstrapping compiler. Garbage collection means you will spend a few more months building a garbage collector. (remember we are writing in machine language so most of the abstractions you are used to dealing with - even in assembly - are missing. Thus complexity is growing exponentially)
How is this a large advantage? When was the last time someone actually did this?
https://github.com/Wilfred/babyc http://www.sigbus.info/how-i-wrote-a-self-hosting-c-compiler... http://zserge.com/blog/cucu-part1.html https://c9x.me/qcc/ https://github.com/alexfru/SmallerC
I believe that Niklaus Wirth created his first self hosting implementation in a small subset of the language and expanded from there. Anyone thinking about doing this for a non-trivial language should consider following that example.
That's painful. It's one big point which Nim [1] does better. It compiles to C and bootstraps from C. That makes porting to other platforms much easier.
I don't understand why Haskell and Rust don't provide bootstrapping from C. Did they write the first compilers with Assembler?
> You just don't notice it because these two languages are already part of the base system.
C (not C++) has become the essential foundation on any platform. You will likely not be able to sell your embedded system unless it supports C.
We don't provide bootstrapping from C because no C-based toolchain has ever existed. However, it may in the future; see my link upthread.
The key word here is 'start'. I don't like to arbitrarily constrain tools, but I've always felt that cross-compilers were sort of a last resort, ie, you don't have a native compiler (you're writing the compiler or operating system), or are really pressed for time. Once you've got a native compiler, that should be it: self-hosted from there on.
I don't know about Haskell, but the first Rust compiler was written in OCaml. See for instance the comments at https://www.reddit.com/r/rust/comments/6nt2j1/is_there_any_e...
Interesting! So, it should basically be possible to bootstrap from source using OCaml and the old source from the Git repository.
It's interesting that OCaml was used to develop Rust since OCaml is also used for formal verification of SPARK (a dialect of Ada) which is another language for safety critical applications. OCaml seems to be a good choice to invent new big languages.
Unless you are talking about an architecture which doesn't have llvm backend, it takes the same effort as nim to port rust and haskell.
> Did they write the first compilers with Assembler?
Rust was written in Ocaml first, bootstrapping from C need decent amount of effort. I don't know the origins of haskell.
Nim doesn't depend on LLVM. It works also with GNU C.
There is also no effort to maintain the "old" version as a strict "bootstrap" since everyone just has binaries of the previous version lying around. :)
> The C code generator is only supported when GHC is built in unregisterised mode, a mode where GHC produces 'portable' C code as output to facilitate porting GHC itself to a new platform.
> Support for cross-compilation works reasonably well in 7.8.1. Previous versions had various issues which are usually work-aroundable.
(7.8.1 was released in 2014, GHC itself is 25 years old)
No it doesn't. You can use gccgo to bootstrap go.
Rust is a different kind.
Also, I've seen people doing generic low level network stacks (or at least articles about it) in Ada, seems lovely.
I fail to see what advantage rewriting existing and proven tools with a new language would bring. Shouldn't the main value new tools bring to be enable writing of new things?
Isn't focusing on existing utils more like a lack of imagination and OCD on optimizing a thing beyond any further value?
It ends up being a bit chicken and egg, right? Don't put a language into base because base doesn't use it. Packages in base can't use it because the compiler's not in base. So effectively someone needs to recreate enough core tools in base in the new language that it's worthwhile pulling in the language tools themselves. And then one must justify any compiler performance and platform limitations before the PR is approved, too, potentially solving those upstream in the language community.
It's a major undertaking, and people keep asking why someone else doesn't do the work. That's why Theo wants people to show progress on code before asking for the tools to be put into base.
This is not the first rewrite, and won't be the last one. Earlier environments were very memory-constrained, which led to optimizing for memory usage; a rewrite with less memory constraints can focus on speed (as mentioned in the GNU coding standards: "For example, Unix utilities were generally optimized to minimize memory use; if you go for speed instead, your program will be very different. [...]").
The current challenge is the "end of Moore's law" leading to an increasing use of multiple cores, instead of faster cores. Developers will have to focus on parallelism instead of raw speed, and new languages can help.
> Shouldn't the main value new tools bring to be enable writing of new things?
Or writing old things in a new way.
> Isn't focusing on existing utils more like a lack of imagination and OCD on optimizing a thing beyond any further value?
These tools are the base over which your system is built (their GNU version isn't named "coreutils" for nothing). Focusing more effort on them makes sense.
Moore's law is indeed nearing its end, but this also ends the multiple core trend.
If you're betting on multiple CPU cores taking off more than they did already, don't.
OpenSSL is a classic counter-example. Interestingly, there are two different approaches actually happening to fix OpenSSL:
- Taking the existing C code base, throwing a lot away and cleaning up the rest, e.g. LibreSSL (developed by the OpenBSD people)
- Taking the TLS spec and rewriting the library from scratch in a safe languange, e.g. ocaml-tls.
I think code safety is not an issue here because the Posix tools (and Unix/Linux as a whole) have already proven their extreme reliability.
The only real advantage to rewrite the Posix tools is motivation. A rewrite in a modern language like Rust will likely keep maintainance longer alive than old C code that no one likes to maintain.
It's the same reason why a new display manager (Wayland) is wanted instead of X11. There is basically no technical reason to replace the X server which works very well to this day. However, there are not much people anymore who like to maintain the old X code, or to support new graphics cards in X.
There is, the architecture is not made with the modern-day world in mind: an X11 application can read all keystrokes, mouse event, and do screen grabs of other windows. An X11 application can emulate your screen locker to grab your credentials. X11 does not support different scaling on different monitors that are connected to your computer.
I like that because I use key desktop macros heavily.
> and do screen grabs of other windows.
I like that since it enables me to make screen shots of any application.
> X11 does not support different scaling on different monitors that are connected to your computer.
I use two monitors with different resolutions, one horizontal, and one in portrait mode. They work perfectly, also with virtual screens. What you possibly mean is that X doesn't support that by default. Nevertheless, it's possible if the graphics vendor provides appropriate drivers.
Apparently Redox developers disagree :) https://github.com/redox-os/coreutils
These are even based on BSD coreutils.
rustutils? coreutils-rust?
name collision like this is how we have millions of people running around thinking that vim is vi while they bash emacs for having bloat as compared to the 'minimalism' of 'vi'..
but writing new tools in safer languages makes more sense to me.
yes, you would need the toolchain in base. not easy, but should be possible. i only know go. perhaps it's still too much of a moving target for openbsd.
you could write programs in new safe language and put them in openbsd ports. problem is that it then isn't a "part of openbsd". do openbsd developers really want to write their next daemon in c? for how long will they stick to c?
Uutils and Redox are setting out to provide POSIX compatible coreutils, and Redox builds from scratch in less than 30 minutes.
perhaps, in one sense - or, alternatively is busy not spending all his time chasing the latest 5000 fads and 100000 not-yet-implemented projects which may or may not ever end up being completed...
I WANT my computer and files protected by software written in a safe way using safe languages. I don’t care about the compile time of grep, I stopped caring about that kind of dick measurement competitions a long time ago.
Give me a safe, open operating system and I’m willing to trade Posix compatibility and compile speed.
That being said, I wonder in what safe language the OS you used to write this is written.
People love to beat up on C, because it does not have feature X or feature Y, and because there is plenty of bad C software to point at. However, it isn't the language that leads to bad software. Any entrenched popular language has plenty of bad software examples, and plenty of security vulnerabilities. The problem isn't the language, but rather, the developer. A good developer who understands and appreciates the problem can write good software in machine code, C, C++, Rust, Java, Haskell, C#, Go, D, or any other language or platform. A good developer who understands secure programming processes can write secure software in any of these languages, given enough time.
What these higher-level languages offer is better abstraction, which can make time-to-market faster. However, in a deeply entrenched system that is already in market, maintaining the current languages and platforms is typically the better play. This is especially true when the source code is as meticulously maintained as it is with OpenBSD. Someone who understands the basics of C can easily read the OpenBSD kernel or userland, and understand everything about how a given program works. Some knowledge of BSD Make, Bourne shell, Korn shell, and Perl may be required to understand the init scripts and build process, but in all, it is a system that is easy to understand, easy to maintain, and with an excellent development process in place to deliver two builds a year. OpenBSD isn't broke, and a new language isn't going to fix it.
That being said, there are tools that the OpenBSD team could use that neither impact the build time nor require adopting fad language of the week. There are excellent model checkers out there for C, and quite a few proof assistants are gaining the ability to check proofs on C code directly, using Separation Logic and Hoare triplets. These tools can complement an entrenched C system by providing an external mechanism of verification that allows designers to formally verify that implementation meets specification. While formally verifying all of OpenBSD would be a tall order, taking hundreds of man years, formally verifying critical pieces of the kernel and creating simplified contract boundaries for the remaining code would be an excellent complement to the code review process that the OpenBSD team already does. Better still, it is something that could pay off without reinventing OpenBSD to match the current fad, as such verification can be run independently of the standard build process. If I were to make a suggestion to the OpenBSD team, it would be to explore such tools. But, to de Raadt's point, such a suggestion would only be valid if someone did the heavy lifting to put such a tool in place, and if it were of utility to the team. They won't adopt something for the sake of that thing, but they are keen for anything that can practically improve the security of their system.
Another you might find interesting in alternate history where C programmers adopt better tech is Cyclone that tried to stick to C as much as possible. Rust's safety scheme took a lot of inspiration from Cyclone.
https://en.wikipedia.org/wiki/Cyclone_(programming_language)
http://trevorjim.com/unfrozen-cyclone/
" There are excellent model checkers out there for C, and quite a few proof assistants are gaining the ability to check proofs on C code directly"
I agree. Further, just using Design-by-Contract with property-based and fuzz testing on those contracts would improve things by itself. There was also an academic who applied Frama-C to one of their smallest, battle-tested components in I think a string library. Although no coding flaw found, the exercise did show the documentation was incorrect. Gotta wonder what benefit might happen for less obvious stuff than string operations.
Of course, if OpenBSD eliminates support for a bunch of the platforms it currently supports, such alternative languages become viable. However, the OpenBSD team purposefully maintains these platforms because they expose errors that would otherwise be missed by a scaled-down release. Software errors missed in x86_64 or x86 may be picked up in MIPS or SPARC because of endianness.
An alternative that I think is more tenable is a language that compiles to C. Such a language can piggy-back off of C's incredible market penetration while adding safety features. Barring some extensions to Cyclone, most of Cyclone's features, for instance, could be implemented as a source-source compiler that targets ANSI C99 or ANSI C11.
https://lobste.rs/s/4cf21p/re_integrating_safe_languages_int...
Ill be replying to followups on that later tonight or in the morning. I totally agree it should be default. I'll go further to say it should have C datatypes/sizes, calling convention, FFI with little to no annotations required, and compile to C. Each check should be removable per midule with a compiler option, pragma per module, or equivalent keyword/operator marked unsafe . It also needs inline ASM. For its own advantages, add macros, safe linking, and a REPL with incremental compilation.
That by itself would be better and faster to develop than with C with seemless integration with legacy codebases. Side benefit like in my Brute-Force Assurance method would be it benefiting from all tooling C ecosystem has to offer esp static analysis and certifying compiler.
What you think on that?
OpenBSD does not have the Bourne shell. One gets the C shell, the Korn shell, and the Korn shell in POSIX mode: no Bourne shell; nor Bourne Again shell.
It is interesting to note that in FreeBSD Perl was removed from the base operating system about a decade and a half ago. So there's apparently an argument to be had that knowledge of Perl need not be required. (-:
I've yet to see anyone with serious industry experience come out and say sane things about replacing C. It's just not going to happen. Not in my lifetime, unless someone decides to write a new operating system, and Linux subsequently loses footing.
Wrong way: write an e-mail
Right way: Here is my feature-per-feature, bug-per-bug rewrite of grep in Rust
This is where Theo appears to me to be quite different to eg Linus. He plays the man, not the ball, with personal attacks.
I find this response quite level headed and I like that some of his responses make it to the front page that provide insights into why such language changes are more difficult than they seem.
[1] https://www.openbsd.org/lyrics.html#51b
As the notes indicate, this saying goes back much further than this release song.
Whether the OP has or has not written Posix utilities in rust is of no relevance to the actual argument whatsoever, it is just a way to have a dig at the OP.
>You have a big nose, it makes you ugly to me.
And that is a statement of fact.