Arm releases experimental CHERI-enabled Morello board | Hacker News Reader

Arm releases experimental CHERI-enabled Morello board

lightbluetouchpaper.org

168 pointszxombie82 comments

> The CHERI memory-protection features allow historically memory-unsafe programming languages such as C and C++ to be adapted to provide strong, compatible, and efficient protection against many currently widely exploited vulnerabilities.

https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

My reading of that linked article on CHERI is two things. First that software needs to adopt the instructions in order to use it. It then raises a question of what is the benefit. What is the experience compared to today, and it should be that embedded software can gain some of the features that are generally reserved for user space. That’s virtual memory protection.

The experience then I would guess is that software will crash rather than, for example, read bad data from the wrong address space. A feature user space apps get from virtual memory (if it’s outside their processes memory space that is).

Did I get this right? Also, it should help Rust just as much, especially in unsafe code regions.

Software doesn't need to "adopt the instructions", it just needs to be recompiled in the same way as you compile it for a new architecture (CHERI is effectively like the 32-to-64-bit transition in that sense). Yes, having capabilities allows you to bring memory protection to the MMU-less embedded space (see for example the now somewhat old paper https://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/201810...).

Yes, if you attempt to access outside the bounds of a capability you will deterministically crash. This is true even if you do have virtual memory and there is memory there.

Yes, the use of CHERI to protect unsafe code in memory-safe languages like Rust is of interest to us. There is also the possibility of being able to remove some of the compiler-generated bounds checks by using the capability bounds instead, though some care is needed to preserve the precise semantics (but some may also be happy to slightly change the semantics if it means they can all be removed and potentially improve performance).

Thank you for clarifying that.

Are there estimates on the performance improvement people can expect with the bounds checks elided and the capability bounds used instead?

Not really, because it gets traded off with the increased memory pressure due to the larger pointer size, and it'd likely be workload dependent. It's not something we've explored to date beyond hypothesising that it could be a good thing.

> Yes, the use of CHERI to protect unsafe code in memory-safe languages like Rust is of interest to us.

Have you found that the relative difficulty of bootstrapping Rust on a new architecture, as mentioned in [1], has hindered your team's ability to research this? Or is it not as bad with CHERI on ARM, because aarch64 is already a Rust tier 1 platform?

[1]: https://drewdevault.com/2022/01/15/2022-01-15-The-RISC-V-exp...

I've bootstrapped Rust for RISC-V on FreeBSD, it's not that bad, the issues I faced were solely porting issues, not bootstrapping issues. I've certainly not had cross-compiling issues like Drew. The awful part about porting Rust is that you need Rust bindings for every type and function in your system's libc, which is fine if your OS+libc combination is already supported (though even then 32-bit and 64-bit need some separate implementation bits), but a real pain if your OS isn't yet supported. Rust on CHERI has the additional complexity of a poor design decision in the language that defines usize as a pointer-sized integer, conflating integers and pointers, rather than providing both size_t and uintptr_t equivalents like C has; see https://internals.rust-lang.org/t/pre-rfc-usize-is-not-size-... for a discussion of this and ways to resolve it without breaking existing software on non-CHERI which, as you might expect, is to introduce a new uptr type for the rare cases when a usize holds a pointer not an offset/length/machine word-sized integer, allowing a usize to be used in its place on non-CHERI architectures (at least for existing editions) but not for CHERI architectures. Someone needs to do that engineering though and it's not a priority for us.

It's important to note that this is "just" a recompile, but it's the kind of recompile that you need to port 32-bit software to 64-bit. Pointers on CHERI are 128+1 bits, and software that treats long longs as a good place to put pointers are common. There's a lot of code that is going to need some sort of rewrite to be able to take advantage of CHERI capabilities. (But the problem is not insurmountable: WebKit already runs to some capacity, so it's reasonable to take even very low-level bit twiddling code and make it work on this platform.)

Yes it's non-zero, though https://www.capabilitieslimited.co.uk/pdfs/20210917-capltd-c... is a recent exploration of what it takes to port X11 and KDE to CHERI. Of the around 6 million lines of C and C++ code involved, only about 0.026% needed to be touched, or just under 1.6k. That number will of course vary significantly between the type of code; boring applications code generally doesn't need changes (e.g. htop and sudo built and ran out of the box for me recently, as examples), but language runtimes will need significant changes. Pages 21, 22, 26 and 27 of that report have the per-component breakdown of that number.

With a program split into multiple small compartments one does not need to crash the whole application on out-of-bound access. It will be enough to signal the parent compartment that one of its children performed an illegal operation.

On most current archs:

> Any piece of code running in a process can construct an integer value and, if this integer corresponds to a valid location in the process’ address space, then it can access memory at that location.

What this adds:

> CHERI changes this. Every load or store instruction and every instruction fetch must be authorized by an architectural capability.

So it should be possibly to call into any function (e.g. from an untrusted blob, and given the capabilities are set up) and on return have the guarantee that none of the callers memory has been touched and all the side effects are contained in the return value, and maybe selected whitelisted addresses?

I remember the mill architecture[1] also claims to have that capability, I think they called these calls "Portals". Btw the talks by Ivan Godard are a must watch if you have any interest in hardware architecture.

But how can existing code be just a recompile away from benefiting from these features, don't the capabilities have to be set up somehow (unless it is purely functional language)?

1: https://millcomputing.com/docs/

The C startup code (for statically-linked binaries) and run-time linker (for dynamically-linked binaries) carve up initial capabilities provided by the kernel into capabilities that cover the various global variables and function pointers needed by the program and libraries, similar to how pointers are initialised for position-independent code (more complex, but same principle, just scan through all the relocations and apply them). When you mmap(2) memory from the OS, you get back a capability with bounds covering that memory. When you malloc(3) memory from your libc, it finds space in an existing mapping, takes that capability and restricts its bounds to the allocation size. When you take a pointer to a stack-allocated variable, the compiler inserts an instruction to set the bounds of that capability to just the memory it allocated for that variable. Every pointer, whether "language-level" (what is exposed in the language) or "sub-language-level" (the pointers in the implementation, like return addresses on the stack or the stack pointer itself), is a capability, and all you need to do is insert a bounds-setting instruction at the point of allocation to restrict its bounds. So your libc's malloc needs modifying, as does your kernel, but your C program that calls them just needs to be recompiled for the pure-capability ABI.

Edit: To answer the first question, yes, that is the primitive which enables CHERI to be used for in-address-space compartmentalisation rather than relying on an MMU for process-based separation and all the overheads that come from context switching address spaces.

Thanks for the explanation!

So these bounds are set by the software (and are guarded against manipulation). Then each read or write to memory is checked against these bounds by the "fine grained MMU" hardware.

Yes, though "software" is rather broad; where exactly the bounds setting happens is important as if you get it wrong it allows malicious software to not set bounds and be able to access memory outside of its allocations. Pushing it to the same place the actual allocation happens or, in the case of referencing global variables, the same place the loading and relocating happens, ensures that the only thing malicious software can do by not setting bounds is make itself insecure.

> Every load or store instruction and every instruction fetch must be authorized by an architectural capability.

This sounds great. But on the other hand ... Yikes! What if this tech falls into the hands of a big corporation and some manager needs a raise?

This isn’t a digital signature or anything like that. Instead, it’s increasing the size of pointers to include bounds and permission rights (R/W/X/etc), plus an extra tag stored somewhere else to prevent forgeries or mitigate corruption. So the only thing big corp gets out of this is software more resilient to memory corruption :-)

Then great, they push for the adoption of the technology and the world's computers become more secure. Arm's a big corporation and they're obviously pretty involved, and both Microsoft and Google are invested in the project.

> I remember the mill architecture[1] also claims to have that capability, I think they called these calls "Portals".

The Mill's "Portals" are more intended for inter-process calls than for compartmentalisation within a process. The Mill can have fine-grained protection of memory temporarily given to a callee because protection ranges are separate from paging. I believe there have been several research OS:es that did pass full pages back and forth over IPC, but at the cost of setting these up and restricting them to that purpose.

Because Portals are so cheap they could probably be useful for compartmentalising larger complex applications such as a modern-day web browser into smaller chunks than today.

BTW, Goddard has told that the Mill team once considered a capability-oriented architecture but they chose not to because the model they had in mind broke C's pointer semantics in some way. They have chosen to prioritise compliance with the C spec, so as to be able to market the architecture.

Hopefully ARM's MTE (memory tagging extension) will appear in Apple's 2022 SoCs (M2, A16), https://security.googleblog.com/2019/08/adopting-arm-memory-...

(2020) CheriBSD port to Morello, https://www.youtube.com/watch?v=7aVygpgkm1

(2021) GCC support for Morello, https://gcc.gnu.org/pipermail/gcc/2021-July/236868.html

(2021) OSS desktop software stack, https://www.capabilitieslimited.co.uk/pdfs/20210917-capltd-c...

> We measure a 0.026% Lines-of-Code (LoC) change rate in approximately 6 million lines of C and C++ code to introduce CHERI memory safety. In our review of past vulnerabilities, we see likely mitigation rates of 91% for X11, 82% for Qt, 43% for KDE, and 100% for other supporting libraries (typically image processing).

(2022) Microsoft Research, https://msrc-blog.microsoft.com/2022/01/20/an_armful_of_cher...

> We can implement this model on a variety of mechanisms, such as MMU-based isolation or software fault isolation, but expect that CHERI will provide better performance and scalability than anything on current commodity hardware ... If the Morello program can demonstrate that CHERI meets the performance goals for real-world use then it is a game changer for security, deterministically preventing spatial safety vulnerabilities and (with software support) heap temporal safety bugs, dramatically reducing the set of bugs that become exploitable as for anything other than denial of service.

They already have PAC, although not exactly the same.

Right. PAC is largely used for CFI, CHERI/MTE are more general memory corruption mitigations.

Previous related submission, with a nice context comment:

https://news.ycombinator.com/item?id=29951145

It's really great to see this level of hardware innovation and investment into security! Although it's a bit of a shame that we're down this path because of the inertia of unsafe programming languages and systems that put performance first. In some sense we're starting to pay back a big debt.

I try to choose my words carefully (these kinds of conversations can get pretty toxic pretty quick). But honestly I think in the long arc of history, C will be regarded like asbestos: very obviously dangerous in retrospect. I respect the designers of C immensely and the amount of work many thousands of people the world over have put into that entire toolchain and ecosystem, but we can't blamelessly have that conversation yet, so I guess I'll stop here.

The year is 1961 and C is still about 10 years away to become a reality, https://en.wikipedia.org/wiki/Burroughs_large_systems_descri...

It would be nice to see a side-by-side Linux port to log the number of issues that the Morello board caught for a system that runs some software that is actually in production.

Or test software that has known vulnerabilities and see if it actually prevents them.

The "Linux enablement" section of https://www.morello-project.org/ says that is "under development", with an initial prototype musl based system available now and a fuller-fat Debian scheduled some time this year.

Our work is based on FreeBSD as its tight integration makes it much easier to manage forking in a research setting, compared with the umpteen different repositories you need to fork and keep in sync to build a Linux distribution. Arm have a minimal Android stack and are working on a Linux distribution (but their current Linux kernel implementation does not enforce capability protection, it's done by a userspace wrapper, and only a select number of binaries in the Android image are pure-capability, many are still plain AArch64), initially based on musl, but it's still a long way behind where we are on FreeBSD where we have (almost) all of the userspace and kernel ported as pure-capability code (the "almost" is because we have not yet invested the engineering effort in porting DTrace and ZFS, but both are on our roadmap as they're important for real use).

Interesting. But: given the amount of server side software that is running on Linux I think a FreeBSD port, while useful is not going to see the kind of adoption that a Linux port would, it will serve as a useful POC but ultimately the challenge will be to get this into the mass market and there it will not really help.

Similarly if you really want mass market adoption then you need a Windows port, otherwise most consumer PCs will remain without it, and for mobile adoption you want an iOS port (though Android does at least contribute a sizeable chunk). Porting FreeBSD does, however, not just serve as a PoC but also let you port all the standard third-party software that runs on all Unix-like OSes (most ports need few if any changes, but with tens of thousands of software packages out there it does add up if you want a full set of packages available), as well as being a reference implementation for other CHERI OSes to use when being ported since we'll likely have already encountered most of the friction points they do. Plus FreeBSD has its Linuxulator which provides a binary compatibility layer for Linux binaries, so you could even develop parts of a CHERI GNU/Linux userspace on top of that without a real CHERI Linux kernel implementation (we have a proof of concept port of the Linuxulator, but it's not currently fully fleshed out, in part because there wasn't even a proper CHERI Linux ABI defined by Arm at the time).

I think this is an extremely important and timely development, I sincerely hope that you succeed and will be following the project closely from now on.

One more question: does the CHERI add on introduce new 'uLimit' like limitations, for instance the number of allocatable blocks or will it scale seamlessly with whatever the various memory management functions and system calls provide?

The only thing that gets weird is the primary thread's stack limit, since we need to construct that capability up-front with the right bounds. This means we have a somewhat arbitrary upper limit (which you could make tunable, though I don't think we do, just a #define'd constant) that should be big enough for any reasonable process and reserve all that virtual address space regardless of what the stack limit is (but it's still unmapped and not backed by anything, just a big region of "don't use this as another capability already overlaps with it"). You can then use rlimit(2) to configure your stack limit and the virtual memory subsystem will do the same as it does on non-CHERI, just you won't be able to exceed that threshold, and the capability's bounds for the stack pointer will always cover the maximum possible mapping, not the mapping you're actually using.

Everything else is the same, just "as much as your system can fit (and system policies will let you)". Which will be slightly less, because pointers are bigger, but we don't have any additional tables that impose arbitrary restrictions on what you can do.

Ok. Super cool that you have thought so far ahead in designing this. Looking forward to being able to use it in production!

How does it compare to previous proposed hw assisted ways to bolt memory safety onto C? Like Hardbound, In-Fat, MPX for example.

I'll leave it to others to go into technical details, but the most obvious answer is that this effort has major industry players behind it, meaning it might actually make it into production.

As mentioned in another reply, there are already a couple of tagging approaches in production.

Unfortunely x86/x64 don't have any, and MPX was broken from the get go.

This isn't a memory tagging system at all and has capabilities far beyond that (pun intended), so I don't know why whatever other approaches like MTE are out in the wild are relevant.

They're relevant because they're technologies relating to memory safety and provide some level of additional protection. However, they rely on secrets and are in general only probabilistic, so they don't deterministically mitigate all memory safety issues (you can deterministically mitigate some with clever allocations of memory "colours", but not all). CHERI and MTE-like schemes also both rely on the use of tagged memory, but in rather different ways.

They are 100% relevant in what concerns taming the native code produced by C derived languages.

The simple answer is that it actually works for real-world software, is microarchitecturally feasible and flexible, and architecturally enforces non-forgeability (which is crucial allowing in-address-space compartmentalisation of distrusting software). Most schemes that take the metadata-table-on-the-side approach fall down on those last two points. MPX is particularly notorious for tanking performance, having race conditions (because loading the bounds is not atomic with loading the address) and having an extremely limited number of bounds registers (I think 4? which is even worse than the highly constrained register set of 32-bit x86) so you're constantly spilling/reloading bounds data from memory. I don't think any of them have been shown to work across the entire software stack from the kernel to core userspace runtime parts to graphical desktops like KDE.

This is pretty interesting - even more so if we might see it in mainstream desktop and mobile ARM chips from the likes of Apple et al..

This is extremely exciting to me. While at my company we use Rust almost exclusively there's still lots of peripheral software we rely on in C and C++. Getting spatial memory safety nearly "for free" in those projects will make me feel way, way better.

I wonder how hard it will be to retrofit CHERI support into Windows, macOS, and Chromium, so we can have a new defense against browser sandbox escapes, making remote browser isolation products irrelevant.

Windows is likely a big task for the same reasons as SMAP (https://github.com/microsoft/MSRC-Security-Research/blob/mas...). XNU should be comparable to FreeBSD, which CheriBSD is a fork of, as both use Mach's VM for memory management and have a bunch of shared code in various places, but userspace is more of an unknown quite how much effort it'd be (you'll need to port Objective-C and, now, Swift, for example). For Chromium we have ported WebKit, so I'd imagine Blink isn't too dissimilar. V8 is likely interesting, though we have a version of WebKit's JSC JIT for Morello, which gives confidence in V8 being doable.

That is already slowly happening on the versions that have access to ARM hardware memory tagging and pointher authentication, specially on iOS and Android.

Solaris on SPARC has about one decade of experience via Application Data Integrity.

And Unisys ClearPath MCP memory tagging architecture goes back to its Burroughs B5500 roots.

Also in case you missed it, Microsoft is one of the CHERI sponsors.

Great to see further progress on CHERI!

See the following for theoretical foundations for the work:

Linux 60th Anniversary Keynote

https://t.co/IRe3vpMlWn

Your research on actor-based programming models has nothing to do with C/C++ spatial and temporal memory safety.

Foundation is about rigorously specifying the general Laws of Locality for Actor systems that CHERI implements.

CHERI is not an actor system. It is a capability system aimed at memory protection. It can be used, like any other architecture, as a basis upon which to build an actor-based framework/system, but it is no more of an actor system than, say, x86. The concepts are deeply rooted in the capability system literature.

ActorsTheory serves as the rigorous mathematical foundations of capability systems.

If I understand this correctly, we now get 128 bit pointers. The lower 64 bits are the address and the upper 64 bits are permissions.

Did I miss anything?

129 bits, there's a "valid" bit at the top. For most software this is invisible but some code will need to care (most notably, anything that copies pointers will need to preserve that bit.)

How does this compare to testing with address sanitizers?

Simple, testing won't prevent you from all the bugs bun only the ones you run over when testing (either fuzzing or unit testing). While enforcing some safety semantics like they do here through capabilities, does. This means, at least for using CHERI you have to go through a custom compilation stack, the CHERI team has already been working on this tooling for long.

Apart from some more interesting scenarios enabled by CHERI: you probably don’t want to run all your production software with address sanitizers, because it would be unacceptably slow. Here the performance overhead is negligible.

But maybe we can run sanitizers during testing and catch most of the issues CHERI will find without building it into hardware. OTOH that doesn't do anything to protect against malicious code, but that should be properly sandboxed anyway.

You should indeed run sanitisers during testing and catch most of the issues; we encourage this! What CHERI provides is twofold:

1. Memory safety issues not found in testing do not lurk as exploitable vulnerabilities; testing is never perfect, often far from it when it comes to edge/unexpected cases where vulnerabilities lurk (though fuzzing can help somewhat)

2. Sandboxing still needs some kind of isolation primitive, which CHERI can provide in place of the heavyweight MMU-based techniques that exist today

Plus let's not kid ourselves that all software is being tested with sanitisers. The vast majority of software running on your system probably is not.

Testing for security bugs means mostly fuzzing these days. AddressSanitizer is added instrumentation, flagging illegal program states even when the test case wouldn't trigger a crash normally.

Fuzzing helps but it's a probabilistic method with fallible search mechanisms, there's going to be cases left that an intelligent adversary can find by reasoning, a different/better fuzzer, better instrumentation, or alt techniques like symbolic execution etc.

I wonder if this is kind of return to segmented architecture? It so it seems flat address spaces backed by virtual memory where not so good idea to begin with.

CHERI is orthogonal to virtual memory, and the two complement each other. You still want virtual memory so you can do the usual paging tricks, copy-on-write, sharing of read-only pages, and so on. Plus the fact that there is a single page table entry for an address that affects all accesses is crucial for our experimental temporal memory safety implementation (see https://msrc-blog.microsoft.com/2022/01/20/an_armful_of_cher...). There's nothing stopping you from using segments instead of flat address spaces with page tables, but it's not really related to CHERI, you still have the same trade-offs as you do on conventional architectures.

What about RISC-V?

We also have a CHERI-RISC-V specification (https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-951.pdf), with support in CHERI LLVM, CHERI QEMU and CheriBSD, plus three open-source FPGA implementations (https://github.com/CTSRD-CHERI/Piccolo, https://github.com/CTSRD-CHERI/Flute, https://github.com/CTSRD-CHERI/Toooba) that span various parts of the microarchitecture design space, and it is the platform we use for our own research on architecture and microarchitecture. But for various reasons (e.g. proximity to the university, existence of competitive microarchitectures several years ago, ISA and ecosystem maturity, enthusiasm and interest on their part) Arm was the right partner for this program.

Then the HN title is inaccurate.

Are you saying Arm haven't released a CHERI enabled Morello board?

It's technically true, but it makes it look like CHERI is only for ARM.

I don't see why it implies that. "Arm releases experimental DDR5-enabled $NAME board" wouldn't make it sound like DDR5 is only for Arm, so why would "Arm releases experimental CHERI-enabled Morello board"?

The HN title is an accurate reflection of the article and the event. When the RISC-V group has a similar event, there will be a similar article. Nothing deceptive or incorrect here.

The HN title is literally the title of the post (if you had taken the 2s to actually open the link), since Arm was the one to release the board, and just because RISC-V specifications, models, implementations, etc., exist, doesn't mean that Arm wasn't involved...

I wager it will be hacked in under a year.

Nobody's claiming it's "hack-proof", that would be foolish, just that it removes certain classes of vulnerabilities that are the majority of CVEs for code written in memory-unsafe languages, thereby reducing the attack surface. Independent analysis by both Microsoft and Google has shown that's around 70% of vulnerabilities, which still leaves around 30%, but is a big step forward.

Not OP, but that's not how I interpreted their comment. I interepreted it as the 70% they hope to have fixed will end up having edge cases not yet considered, and the protections will end up weaker than desired. No-one designed a processor to be susceptable to spectre and meltdown, once something moves into production there is significantly more incentive to investigate and find these flaws.

There are no CVEs related to Solaris SPARC Application Data Integrity, or Unisys ClearPath MCP.

Either they aren't interesting for hackers, or they actually did a good job with hardware memory tagging.

I might imagine the former over latter

Might be, then again most of their customers are government agencies, not really something that hackers care about.

We do have formal proofs of various security properties at the architectural level that consider the entire architecture with all its complexities and warts. Speculative execution is of course a concern (and is an active area of research for us), though our belief is that the bounds information now present at the hardware level allows it to be tamed. Another concern is the interaction between undefined behaviour and CHERI; the former needs to be sufficiently constrained in order to not inadvertently turn code that would be memory safe with a naive CHERI compiler into code that is not. We also know there are still memory safety-like issues we can't protect against; we can stop pointer injection, but we can't stop tricking programs into copying the "wrong" pointer somewhere, or type confusion bugs that result in using pointers in an unintended way. Many of those exploit chains today rely on exploiting some other memory safety vulnerability we do protect against, but we cannot predict if people will come up with alternative approaches that avoid those in a world with CHERI.

Thank you. Your response's here and throughout this thread have been quite enlightening.

Exactly - these formal methods have implicit assumptions in them. Within their axioms/priors they are proven correct. But the devil is in those assumptions. And that is disregarding the rather foggy notion of 'proof'. Is this proof written in a formal proof system that automates the proof? Because if it isn't you have another source of error. Even if it is in a formal proof system, who is to say that there is no error in the formal proof system? I admit that each layer adds more assurance - but appealing to 'formal proof' is a bit like appealing to god, it's an appeal to an unassailable authority. When in fact, once you know the details these things are never so clear cut.

What exactly do you mean by hacked? This isn't a product with some singular security aim, it's an evaluation platform that's come from an ongoing large research project. Previous work (using FPGA and software simulation) will have found and fixed many architectural issues. No doubt more will be found here.

In a sense the entire aim of this is for it to be 'hacked' to further improve the security architecture.

Do you have criticisms of the feature or is this facile cynicism?

With funding of DARPA how sure can we be that it will not have a backdoor? Is there a way to proof that there is no backdoor to the security features?

The MIPS and RISC-V CHERI implementations are open, you can audit the BlueSpec. The entire software stack is open too.

Note however that CHERI is not just an implementation; it’s a design, an idea. You can’t backdoor that.

Anyway, it's harder.

Reply on news.ycombinator.com