Safe and Secure Drivers in High-Level Languages [video]
media.ccc.de
media.ccc.de
While it's great that you get improved safety (and often nicer and easier to reason about code) by using something other than C, you can still have memory safety issues from cavalier or incorrect usage of unsafe APIs, since they undermine the guarantees the language provides with regards to correctness.
Also, unrelated: does anyone actually have the slides (you know, the presentation file with text in it, rather than the mp4 that CCC is offering me) for this presentation? It's really annoying to scrub through a video to find stuff on a slow internet connection :(
On the last Linux Kernel Summit, according to Google, 68% of Linux kernel security exploits are caused by C memory corruption issues due to lack of bounds checking.
[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus
Try to be that clever guy putting such gates into place without having the team be on the same wave length.
Github is a bubble, there are tons of software projects out there, using a myriad of build infrastructures, or even just doing plain old IDE builds (yes I know, but it is as it is).
In no point there was a mention of switching languages.
"Why do that, if there are languages that do it by design?"
But I see it wasn't your comment, my bad :)
While the OP may demonstrate that other languages aren't always that bad in practice, I think the consensus is that Rust and C/C++ are the appropriate languages when maximum efficiency and minimum overhead are desired.
While Rust is a good option, the (safe subset of the) language has an intrinsic shortcoming that doesn't seem to be generally acknowledged. The forthcoming C++ "lifetime checker" (static analyzer) has the same issue [1]. Essentially, if you have a list (or any dynamic container) of references to a given set of existing objects, you cannot (temporarily) insert a reference to a (newly created) object that may not outlive the container itself.
In my view, this is a problem. The workarounds are intrusive, inconvenient and/or involve significant overhead. Which makes it tempting to just resort to unsafe Rust in those cases. (And of course this specific example is representative of a whole class of situations with a similar dilemma.) The safe C++ subset doesn't suffer the same issue. (Because in some sense, C++ is still a more "powerful" language.)
[1] https://github.com/duneroadrunner/misc/blob/master/201/8/Jul...
Discussion about the C version on GitHub in 2017: https://news.ycombinator.com/item?id=16014307
You can't really get a faster speed than Rust here; the only minor thing that could be improved is the worst-case latency by isolating the core that the driver is running on (isolcpus kernel option) to prevent other random interrupts or scheduling weirdness. But that optimization is the same for all languages and should get rid of the (tiny) long tail.
http://www.nextcomputers.org/NeXTfiles/Software/OPENSTEP/Dev...
Objective-C was a great match for driver development, devices tended to have a very natural OO flavour to them and naturally sorted into classes.
Putting things in user-space seemed a natural extension, that sadly didn't happen at the time even though Mach had the hooks for it (we never got user-level pagers either, which would have rocked together with garbage collected languages). There certainly didn't seem to be a good reason why I had to reboot the machine and wait for fsck when there was a minor bug in the little driver I was writing to talk to an EISA printer controller that had nothing to do with the basic functioning of the system...
(Why would a printer controller be on an EISA controller, you ask? It directly drove a Canon Laser Copier, so, yes!)
Oh, and not surprised by the abysmal Swift performance. Alas, Apple's marketing of Swift as a "high-performance" language has been very successful despite all the crushing evidence to the contrary.
How do you feel about the current state of driver development on macOS, with Objective-C basically being replaced with Embedded C++ with partial reflection?
What I heard (quite some time ago) is that this move is now seen as a mistake.
I made a post a few years ago discussing the issue:
Android Things userspace drivers are done in Java, and since Treble, it is also possible to write Android drivers in Java.
MicroEJ and Windows IoT Core also allow for such capabilities.
Loved the talk.
Is it purely a matter of what is well-branded?
If anything, I can take solace in the fact that it is very much alive despite the exaggerated rumors of its death. I'd encourage everyone to try it out and steal all the ideas.
Yet, I'm genuinely curious, why isn't GCC up to snuff?
* Edit: Which of course limits general adoption.
Interesting. What is the reason for higher performance of user space C driver (and the other user space drivers for that matter) when compared to the kernel C driver? Will this hold for all driver types or is this a rather uncommon property of this particular kind of driver?
As our devices have gotten faster, the user-space/kernel boundary is becoming more and more of an issue. I was shocked when my supposedly super-fast MacBook Pro SSD (2+GB/s) was only giving me around 250MB/s.
Turned out mkfile(8), which I was using without thinking much about it, is only using 512 byte buffers...
https://blog.metaobject.com/2017/02/mkfile8-is-severely-sysc...
- Microkernel OSes - one driver failing is okay, because it runs in mostly in user-space. The big gotcha in microkernel design is transactions that touch multiple components. Some sort of across-driver transaction API is needed (start, commit, rollback) in order to undo changes across several userspace subsystems.
- A standard language (like the talk suggests) and shipped as portable bytecode to run on a VM or compile to native, so that drivers are portable and runnable without knowing the architecture.
- Devices themselves containing OS-signed drivers rather than each OS having a kitchen-sink installation of all drivers. Each bus would have an interrogation call to fetch the driver.
(The latter is a great thing to have built, of course, just thinking aloud about how this might replace existing drivers)
Now I have a talk to point out to those that always trump that referencing counting GC are so much better than tracing ones.
If you make use of the refcounting library types in Rust, from the std::rc and std::sync crates, the performace impact would be quite similar.
(Besides, I think std::rc has better performance than the refcounts found in Swift and C++, because it's used in cases that don't need atomic update, and yes this is statically checked too.)
Also known as the mythical "only other programmers commit errors, I am always correct".
If you could somehow write perfect code in a timely manner, you'd have no need for Rust. You'd likely also be a unicorn.
The fully-automated checking is precisely what allows one to write unmanaged code without tearing one's hair out.
Reference counting can be better in terms of ease of implementation, cross-language interoperability, and by reclaiming memory immediately when the last reference to it disappears.
Hence why it is usually one of the earliest chapters in any CS book about GC algorithms.
Reclaiming memory immediatly only works in simple data structures. Naive reference counting impletations have similar stop-the-world effects when releasing relatively big data structures, which can even lead to stack overflows, if the destructor calls happen to be nested.
In case of Objective-C and Swift, Objective-C tracing GC project failed due to the underlying C semantics, thus they took the 2nd best approach of enforcing Cocoa retain/release patterns via the compiler, which only applies to a small set of Objective-C data types.
Swift naturally had to support the same memory management model, as means to keep compatibility with the Objective-C runtime.
The fact is that Java isn't the only game in town, and all GC enabled system programming languages do offer multiple ways to manage memory.
Value types, traced GC memory references, global memory allocation, stack values, RAII, untraced memory references in unsafe code blocks.
Not every OOP language is Java, nor every GC is Java's GC.
Additionally, not every Java GC is like OpenJDK GC's, there are plenty to chose from, including soft real time ones for embedded deployments.
As for Android, it is a fork still catching up with what Java toolchains like PTC/Aonix are capable of, all because Google decided it could do better while screwing Sun in the process.
Since "GC-enabled system programming languages" is an oxymoron, a claim about what such languages may or may not include is just not very useful. But it's definitely the case that properly combining, e.g. "traced GC memory references" and RAII including deterministic deallocation for resources is still a matter of ongoing research, e.g. https://arxiv.org/abs/1803.02796 That may or may not pan out in the future, as may other things such as pluggable, lightweight GC for a subset of memory objects, etc., but let's stop putting lipstick on the pig that is obligate tracing GC.
- Mesa/Cedar at Xerox PARC
- Algol 68 at UK Navy computing center
- Modula-2+ at Olivetti DEC
- Modula-3 at Olivetti DEC/Compaq/HP and University of Washington
- Oberon, Oberon-2, Active Oberon, Oberon-07 at ETHZ
- Oberon-07 at Astrobe
- Component Pascal at Oberon microsystems AG
- Sing#, Dafny and System C# (M#) at Microsoft Research
- Java when running AOT compiled on bare metal embedded systems like PTC Perc and Aicas Jamaica
- D by Digital Mars
- Go at Google (Fuchsia) and MIT (Biscuit)
Lets stop pretending reference counting is the best of all GC algorithms, in spite the fact that is quite basic and does not scale in modern multi-core NUMA architectures.
The comment about "does not scale in multi-core NUMA" only applies if you have objects that are shared between threads because otherwise there's no atomics going on. For example, Rust has a generic ref-count mechanism that automatically uses atomic operations for ref-counts when an object might be shared between threads but otherwise does simple arithmetic. Non-atomic refcounts are most likely also going to be faster than any other global GC algorithm. Other languages require explicit differences but are still able to offer the same thing.
The fact of the matter is that the majority of objects do not require expensive GC of any kind & can live on the stack or have explicit ownership guarantees. Choosing defaults where everything might be shared is not a good default for systems languages as it pessimizes memory usage & CPU performance to a drastic degree.
That being said, GC does have its place in all manner of applications and has other advantages like making developers more productive which isn't a bad thing but these are domain-specific decisions. There are plenty of techniques - reference counting, memory pools, static memory allocation, various GC algorithms, etc, etc. Each has tradeoffs & every single GC system I've encountered means variable latency/stop-the-world and greedy memory usage (optimized for the 1 application). That's valid in some domains but certainly isn't desirable. If there were an awesome GC system like you claim that could perform that well it would have been deployed already to inemurable applications like all Java vendors, Javascript VMs, C#, etc, etc. It's an extremely complex problem.
Most of your links are niche commercial systems or even pure academic research systems. They're not proof of anything other than GC being possible to implement for various languages/machines which isn't a claim that's been disputed at all.
> Go at Google (Fuchsia)
AFAIK Fuchsia does not use Go for any systems-level portions. Those are written in C/C++/Rust last time I checked (with Rust being the official default going forward). Do you have any links to the contrary?
That sounds unduly broad. You really think the Linux kernel would be better off using a GC?