GrapheneOS says Pixel 11 has MTE support after all
grapheneos.social
grapheneos.social
> Memory safety bugs, which are errors in handling memory in native programming languages, are common code issues. They lead to security vulnerabilities as well as stability problems.
>Armv9 introduced the Arm Memory Tagging Extension (MTE), a hardware extension that allows you to catch use-after-free and buffer-overflow bugs in your native code.
Of course, this doesn't come for free. Four bits per 16 bytes means a 3% increase in memory needed for tagged memory, hardware overhead for checking tags on memory accesses, and software overhead of setting/changing/clearing tags as necessary. This overhead is low (Apple shipped this in flagship hardware a year ago and nobody's complaining about performance there) but not zero, and an implementation with poor performance could be a real problem.
I know that it's a really improbable scenario and the OS would also just refuse you allocations at some point, but what would the malloc implementation and the MTE do in such a case ? Fail the allocation ? trap when reading the pointer since it would point to the "wrong place" ?
> Because EMTE tag checking imposes a performance cost, we designed Memory Integrity Enforcement to take advantage of our secure allocators first and use EMTE to protect only smaller individual allocations within a type bucket, which software allocators can’t defend on their own.
Which seems to imply that outside of Apple’s targeted use in the kernel, the developer use of MTE on Apple platforms requires that you migrate your code to use their typed allocators…which I don’t know how many people are going to do that (and it is clear their answer is going to be to point people to Swift since it makes these types explicit for the compiler and even then, I am sure there are bugs in Swift in certain cases). Memory tagging is neat tech, but I don’t know if it’s going to gain all that much traction. It requires a burden that I suspect most app developers are not willing to take on and given that the user interaction is a crashy app. This is a path to nowhere except those of us who understand the trade-off.
Theres a couple of apps that use C++ libraries for which I disable MTE.
Most android apps using Java or Kotalin which hugely helps with avoiding buggy apps crashing with MTE enabled.
Android can and does regularly force developers to make changes to their apps to improve privacy and security on the platform. Like theyve done with other changes they could gradually force developers to fix bugs in their apps. Maybe offer a MTE opt out for apps that use memory unsafe languages.
Think its likely Apple and Google will increasingly push forward the use of MTE in Android and iOS
MTE tracks provenance of pointers which means it catches bugs where a valid pointer derived from one allocation is used to access another allocation. Provenance is indicated by a fixed number of tags available. So there’s a 7% chance of not detecting an occurrence of a memory bug.
ASan is implemented differently: it adds red zones next to allocations. In theory it could have a false negative if a pointer jumps over the poisoned region. But it works well for stack memory in addition to heap memory. MTE doesn’t protect your stack allocated objects.
Kostya/et al who pushed for and designed the extension, was trying to accelerate address sanitizer so it could be on all the time. Among other things.
In fact, most presentations presented it literally as a way to do hardware accelerated ASAN (again, among other things), so the post you responded to is correct in that sense.
(I was there at the time, helping him figure out how to push for it)
It also explains that the fact ASan works well with, e.g., std::vector in C++ doesn't come for free and if you want ASan to detect bugs when your custom collections or allocators are used you have to use special ASan API to mark [in]accessible memory.
The thread makes it sound like it could have been disabled due to errata or performance issues. Basically, it looks like the software Google is shipping intentionally doesn't use MTE on the Pixel 11 hardware. That raises the question of...what does Google know is wrong with MTE on the Pixel 11?
So, I don't think it's accurate to say they don't use it yet. They were using it, and then they stopped. I think that's one of the reasons the linked GOS thread says they're concerned that "MTE may actually be broken due to CPU errata".
The thermometer was nice too if you have kids.
MTE ships two modes. synchronous mode and asynchronous mode. SYNC is slower but gives you far better traces and throws an SEGV_MTESERR as soon as violations happen. ASYNC however is async so there's a bit of a delay between a violation and the "catch" that throws SEGV_MTEAERR.
Strictly speaking async is worse for security because there's a brief window of time where the process "gets away with it" but the main differentiator is that because things don't stop the moment the violation occurs, SEGV_MTEAERR traces tend to be some degree of "out of date" vs SEGV_MTESERR which capture the exact state of the world the moment the error occurs.
The main tradeoff here is that async MTE has basically negligible cost. Something like 1-5% in practice but in microbenchmarks you can see up to 50%. Vs synchronous MTE where the penalty is on average closer to like 10-15% but in microbenchmarks it can be like 5-6x slower.
So yeah the perf cost is there but it's really not a major issue.
------------
The main issue is that apps and services crash when an MTE segfault occurs. So this means that to the uninformed end user apps appear spuriously unstable with no meaningful context. And it's not just apps. On graphene I see MTE segfaults semi regularly from various system daemons (mainly related to GPS/nav) and occasionally in Google Play Services itself.
From time to time I get them in Youtube and I constantly get them in the Twitch app. It's very annoying and the apps just crash when you happen to do some particular action leaving you walking on your toes to avoid accidentally tapping whatever magic pattern happens to invoke a MTE SIGSEGV until the next update. Doubly so since most apps provide no meaningful interface for uploading log traces to report these issues.
-------------
If Google wanted to roll this out without the spurious crashes they'd need to deploy it and eat the minor perf hit but they'd need to register a global signal handler to capture these MTE SIGSEGVs and report them back to the services in question without crashing the app.
And most app devs don't care so they'd be taking a minor (or major) perf hit in exchange for logging errors that developers always ignore. Doubly so in the modern day of "just have AI fix/do XYZ and who cares about the consequences as long as it runs".
I'm not surprised they dropped it but I do honestly wish they'd forced the issue and just deployed it and forced apps to fix their shit.
Likewise if it's because a language is being lazy with how it handles memory (see: wireguard-go SEGV_MTESERR because golang's string implementation was being overly permissive with pointers).
And of course this only applies to SEGV_MTESERR. SEGV_MTESERR has a much higher runtime overhead that Google and co would not tolerate (but Graphene does) so they'd likely only be willing to use SEGV_MTEAERR which provides traces that point at a completely different place than where the fault occurred (but still provide a little bit of useful info).
Apple replaced MTE with an upgraded version that can run in synchronous mode all the time without the performance hit.
> Consider that MTE can be configured to report memory corruption either synchronously or asynchronously. In the latter mode, memory corruption doesn’t immediately raise an exception, leaving a race window open for attackers... We believe memory safety protections need to be strictly synchronous, on by default, and working continuously.
https://security.apple.com/blog/memory-integrity-enforcement...
Google may just be getting ready to follow suit.
That may well be, but it's a regression for the time being, since Advanced Protection mode presumably no longer uses MTE (or a replacement for MTE), whereas it did on the Pixel 8, 9, and 10.
It seems like they should've continued to offer MTE until the replacement was ready.
Vulnerabilities in the world’s most popular (by volume) mobile OS could provide a plausibly deniable global espionage backdoor.
A lot of heavy lifting is involved at the lower levels of security and Graphene doesn’t have to deal with this work, generally. I guess my surprise comes at the “fit and finish” stage where Google regularly seems to be lacking.
I personally don't bother with Google dependencies at all. If an app requires them to work, I will do without it.
the fact that someone can say that post-Dual_EC_DRBG/project-rubicon/eternalblue/eternalromance exposure boggles the mind.
That's not something that's really acceptable on a $500+ phone... so if you're Google, you're not going to turn that on by default and you're not really going to be interested in keeping it as a feature that users can turn on.
Graphene has a different focus, so they can push the tradeoff towards enforcement and crashes rather than non enforcement and fewer crashes.
Google could perhaps do more with their crash logs and pushing app developers to fix MTE crashes, but they don't often do a lot of that; and they get called out for being developer unfriendly when they do enforcement.
They should really do some sort of sampling thing to generate crash dumps and find big offenders and increase the coverage over time.
For Google employee devices, 0.1% of background execution starts while charging in an idle period (overnight bedtime charging) will use MTE. When any specific device hits an MTE crash, back-off sampling for 1 week on that device. Modulate the sampling rate so crashes are manageable.
When Google employee devices are not crashing overnight at a high rate, then start sampling background execution during the day for employees and overnight background execution on general user devices. Finally, sample on foreground execution, again for Google employee devices first.
If there's significant variation in crashes by app, you probably need to setup a way to set sampling rates to zero or very low for application versions that have been identified as a known problem and don't need additional traces.
2 apps have C++ libraries and MTE makes these crash due to memeory bugs, so I disable MTE for these.
So what are you on about man - it's easy to take someone else's work and put tweaks on it that wouldn't work at scale.
I don't interact with Graphene or any of their folks at all; I'm just a distant observer like most folks here.
Still, I can't help but think that Graphene seems to want to complain about everything and anything that doesn't fit their niche use case. (As much as it seems beloved here, people that flash custom Android OSs are the very definition of niche users.)
My personal attitude to Graphene seems to get a bit more negative with each one of these "rants" and I doubt I'd go far out of my way to help them, even if I had exposure to them.
Not saying whether it's a good thing or a bad thing, but just the nature of reality.
Security should be the number one priority, frankly. Graphene has shown that this is possible, and they have tried multiple times to get Google to integrate their work.
What would you want them to complain about instead? Of course they'll complain about that, just like Googlers will complain about things affecting their stock price, no one is surprised that people care about stuff they're personally involved in, it makes a lot of sense.
Now if these complaints weren't accurate, then I'd walk with you and feel a bit more negative with each piece. But the ones I've looked into, have been spot on, so who cares if it's for their specific niche? I expect them to care about their niche, that's why those people all work together in that organization in the first place.
Let's be real, AOSP doesn't exist any more. Google have closed down nearly everything. All the development happens in private, you've stopped addressing bugs raised by the public, the source of patches are only infrequently released, device trees are gone.
Wouldn't you complain?
To emphasize this point a bit more: only "QPR0" (major release) and QPR3 are released as part of AOSP. QPR1 and QPR3 are not released at all anymore, but contain fixes for vulnerabilities that are not marked high/critical (so don't end up in ASB). It is not clear to me whether OEMs get access to QPR1 and QPR3, but Google are not only witholding features, but also a set of security fixes.
Besides that, they are torpedoing other systems through Play Integrity.
IMO it would be best if AOSP was spun off from Google into its own org that actually cares about developing an open source system for others (both open source systems like GrapheneOS/Lineage and commercial vendors like Samsung) and that would have an attestation system that is open to vendors that have good device security.
Stock Pixel is an awful experience. So many useless notifications, popups, ads, privacy not by default.
Company: "We care about your privacy" meanwhile 1400 corporations they share data with
GrapheneOS: "There's zero telemetry in GrapheneOS"
The more you read the more you realize they are nearly always correct.
Huh? What pixel are you on? You only get notifications from stuff you install after the initial setup is done. And even then you can outright mute applications, completely.
> there's 50 apps you don't need Where are you getting your pixels? Mine came with all the essentials, I don't remember disabling anything in particular. No game demos, no preloaded meta cancer, just the normal google suite.
> The OS feels super bloated compared to GrapheneOS.
When was that ever not the case? When was a stock OS lighter than a custom flavour on any phone? You can't ship Android in the state that custom OS-es do, you won't pass the certification to get Google's goodies like OTA service and preinstalled play store. Don't quite get the Gemini argument either. You're buying a Gemini phone that touts AI everywhere on its promotional page. What do you expect?
Hmm... Let's try reframing this: "as much as it seems beloved here, people that install their own operating systems on PCs are the very definition of niche users"
I'm absolutely certain that's how IBM felt before the clones. But the ability to install what they wanted on a defacto standard platform is what launched the computing revolution. I think we'd still be living in a sterile monopolistic environment with $10k compilers otherwise.
Folks installing their own ROMs on phones are only niche because they've been pushed out at every opportunity using locked bootloaders, embedded security processors, factory installed secret keys, etc.
Despite all that, there's still thriving communities developing and using custom ROMs on their phones. That demonstrates more than niche demand.
I mean yeah, 99% of people never installed an OS and never will, what's your point here?
That the 1% who do build visicalc, Linux, the internet, Google, and every application and innovation that happens outside the corporate wall. The entire ecosystem everyone else ends up using.
And that calling that niche is ridiculous, shortsighted, and shooting oneself as a platform owner in the foot.
It is a niche, it always was. There was never a mass community of users, the most installs you used to see on any given AOSP based project is maybe 20-30K for the most popular devices like Redmis, Pocos or Google Nexus phones.
It's still a niche, and things that users do and used to do to their phones are fundamentally incompatible with Android's security model. Root access, magisk, xposed, overall zero or near zero security validation from the security perspective on all ROMs but Graphene (maybe some others, I haven't followed the sphere for a while).
Again, only because the platforms are locked down and there is no single standardized target for universally bootable ROMs in the way a Linux ISO can boot on any PC.
Kinda sounds like you might be younger and didn't live through the 8bit micro -> PC transition. Standardization of the platform led to a cambrian explosion.
https://en.wikipedia.org/wiki/Influence_of_the_IBM_PC_on_the...
Couple of things here. Android was never that open to begin with. There were always (and still going) issues with the kernel sources being withheld indefinitely, AOSP side changes are almost never released at all, iirc only Google released anything. Bootloaders were always an issue, not all manufacturers had unlockable (or easily unlockable) bootloaders.
Ironically, Google were always the spearhead of the standartization in the Android world. Project Treble and stuff resulted in the fact that almost any Android device released these days can run a generic Android image, it's not gonna be as functional as the original Android, but it's something.
Yes, I am on the younger side, that's why I know how it was in the Android world. Nobody is interested in reinstalling OS-es on their phones because the amount of effort to create an Android ISO that's as usable as what's preinstalled is truly gigantic. Android flavours are heavily customized for the needs of the OEM, and for the hardware that they're gonna run on.
It's a niche, always was and always will be. A niche powered by a few large players like LOS or GOS, that actually get paid somehow or heavily work for the idea, and an army of teenage unpaid maintainers, who quit the maintainership as soon as they get a properly paying job that takes most of their time (me).
You've restated my thesis.
> Yes, I am on the younger side
I could tell.
Your custom rom experience sits wholly within a larger category of computer systems experience about which I've been speaking.
The sort of fragmentation you describe is a symptom, not a foregone conclusion.
It's easy when it comes preinstalled.
> And it's fundamentally incompatible with the concept of having a truly secure env on your device (same as in the PC space, ironically enough).
Where on earth do you get this misconception? Custom Android != has root. If you don't think Graphene is reasonably secure, explain how?
> By combining GrapheneOS’s pioneering engineering with Motorola’s decades of security expertise, real‑world user insights, and Lenovo’s ThinkShield solutions, the collaboration will advance a new generation of privacy and security technologies. In the coming months, Motorola and the GrapheneOS Foundation will continue to collaborate on joint research, software enhancements, and new security capabilities, with more details and solutions to roll out as the partnership evolves.
https://motorolanews.com/motorola-three-new-b2b-solutions-at...
To be frank, though, I don't see any of this line of reasoning as relevant; it's just appeals to greater authorities on either end. If AOSP is only for manufacturers there's really no reason for it to be open source in the first place. And then folks who care about actually improving security end-to-end outside of whatever's convenient to implement by those beholden to the quarterly profit metrics are up a creek.
Personally, if this whole GrapheneOS/Motorola thing doesn't improve the state of the ecosystem I'm going back to Apple or whatever other manufacturer makes it clear they take security seriously.
Also security and using non-Google OS are not niche usecases. I'm not sure how you are working on Android, the most popular OS while claiming security is a niche usecase. In fact, I have less confidence in security of your work.
Well we know Google isn't enabling MTE, while LLM-enabled exploits are multiplying rapidly. Maybe you need to get back to work?
This is a mischaracterization. The niche isn't Android hobbyists, it's people with what should be a basic expectation for privacy. I wouldn't flash GOS or any other OS if I could safely avoid it.
Too busy crippling sideloading I guess
Google Pixel has the marketshare it deserves.
Yet no GrapheneOS.
What's especially stuck in my mind lately is how insecure basically all desktop OS' feel. In at the point of buying a second and third GPU for my desktop to run my email and browser in dedicated VMs because everything feels as watertight as a sieve. Qubes seems more and more appealing in a world where every open source software supply chain is under seige, corporate software underprioritizes security, and most sites will stop at almost nothing to surveil you.
I truly lament this new reality where MY computers I PURCHASED feel to use like I'm reaching blind into a paper bag filled with razor blades.
I made the same observations and conclusions as you, but a few years ago. I have been daily driving Qubes OS exclusively since on my Thinkpads. If you haven't started to yet, I implore you to consider the following when buying a system for Qubes: mid-line CPU with plenty of PCs (turn off the ECs), As much RAM as you can afford (seriously, budgetmaxxx on RAM), dedicated secondary GPU with the minimum specs you need is a must for inference (may have to widen the budget a little bit after budgetmaxxing RAM).
Expect to not game on the platform. I have not tried this but if you can't cut the digital crack-cocaine habit then it may be possible to set up a gaming qube which takes the dGPU passed through: get a monitor + second USB keyboard & mouse, plug the monitor into the dGPU's HDMI ports, and pass through the secondary peripherals. Digital crack-cocai... err games... don't like having their frames pushed over X11 forwarding even if the dGPU is passed through, so you will need the monitor; the secondary input devices are so the mouse and keyboard inputs don't leave the gaming qube.
I ended up switching away from using my Mac to using an iPad M5 + Magic Keyboard as my sole computing device for the security benefits, and it’s been working pretty well and in retrospect I don’t miss anything from macOS aside from a chromium-based browser. However I’ve said it before that switching back to using a Mac after using iPadOS for so long feels like a serious regression, for all it’s faults the iPad + iPadOS feels like a futuristic technology preview of macOS.
For development work I use GitHub Codespaces (and self-host Coder in the future hopefully). I was moving towards cloud/container-based development already and it works nicely. Plus NPM vulnerabilities are no longer a serious issue. Again the only thing I miss is Chromium, but I’ve altered front-end development planning to target Safari/Firefox since websites shouldn’t be implementing non-standard Chrome-exclusive features anyway imo (I’m saying this as someone who highly prefers chromium lol).
GrapheneOS: "Multiple Google engineers we've contacted have said they aren't able to give us any information about this so we're left doing reverse engineering and relying on leaks. The leaks do not seem reliable and do not match what we see. Our concern is that MTE may actually be broken due to CPU errata."
What I'd like to know is, is it not sufficient to just check the feature bits in ID_AA64PFR1_EL1? (Isn't this the first thing you'd try when trying to determine if the hardware supports some feature?)
If they were making the determination solely based on the fact that `arm64.nomte` is being passed to the kernel (or based on what features are exposed by kernel interfaces), it may have been better to say "MTE is seemingly disabled in the current Android release" rather than claiming that it's simply not present in hardware. If you're asking the Linux kernel about hardware features, maybe it takes cmdline arguments into account when presenting that info to userspace.
The TRM[^1] mentions that some of the feature bits depend on BROADCASTMTE (presumably some CPU input pin), but maybe that signal isn't constant and is allowed to change based on what happens in firmware/the bootloader?
Also, why the claim about the lack of hardware acceleration for MTE in the caches, is there evidence for that, or is this also a misunderstanding?
I think it's reasonable to assume that the perf impact of MTE is non-negligible (on cores in older Pixel devices[^2], MTE apparently suffers from the fact that checked stores are serializing!), but it's entirely possible that this does not follow from some physical design concession when implementing the SoC. The characterization of all this as some kind of cost-cutting measure is not necessarily accurate.
[^1]: https://support.arm.com/documentation/108014/0101/?lang=en
https://discuss.grapheneos.org/d/41564-pixel-11-doesnt-meet-...
edit: Oh, I guess Linux does emulate the feature registers, doh. Since that user is booting with arm64.nomte, the kernel is changing those feature bits to zero! This probably explains the confusion here about hardware support: in Linux userspace, you still trap for system register reads, and this is just abstracted away from you...
https://github.com/torvalds/linux/blob/841e384b841a3d89c50b4...
Android 16 QPR2 Beta 4 was released around 2 days after we made our initial thread and added back partial firmware support for MTE. It's now possible to enable it again via a fastboot command, but not via the OS using Android Advanced Protection Mode or developer options. It's still disabled from the OS perspective but it can be forcibly used after enabling it in the firmware using ADB.
The way it shipped was that it was entirely unavailable due to the firmware not having support for setting it up for usage. It's not present in the OS in the way it is for the Pixel 8 through Pixel 10a either.
It's now usable with the Android 16 QPR2 Beta 4 firmware but it appears there's something wrong with it, otherwise it wouldn't be disabled tghis way.
I will make a note not to buy this model.
Is this the worst quality control lapse since the Huawei Nexus core failures?
Everything else is in the realm of bean counters, I would think.
https://axis-intelligence.com/google-pixel-statistics/
They’re also strongly tied in with Google Fi which is almost certainly very profitable.
> The security implication here is that the telephonic operating system has access to the same system resources as the application operating system at the same time, right?
> The baseband is isolated on all of the officially supported devices. Memory access is partitioned by the IOMMU and limited to internal memory and memory shared by the driver implementations. The baseband on the officially supported devices with a Qualcomm SoC implements Wi-Fi and Bluetooth as internal sandboxed processes rather than having a separate baseband for those like earlier devices.The way it should have been all along.
A separate OS runs the security chip (like Titan). Another could be running in EL3 (Trusted Execution Environment), yet many could be running in Realms (mutually untrusted VMs). The later stage bootloaders and some firmware (ex: GPUs) themselves could be considered OSes in their own right.
People bring up the baseband (ie. cell modem) having its own OS because someone gave a talk on it at defcon or something, but it's just one of many.
As currently written, one would take away an entirely different message from the headline.
iPhone 17 has it now. maybe Google's work here is done
https://www.reddit.com/r/apple/comments/1ndcrug/iphone_17_in...
It's actually recommended to get least of your apps as possible via the play store and use 3rd party stores if you are going for maximum privacy or de-googling.
I can load apps from the web (disabled by default iirc), the local file system, a sandboxed instance of the Play store, Obtainium, f-droid, wherever.
Took some getting used to, but it’s nice feeling like I actually control and can choose how my phone works.
90% of apps work, no finagling required
Next 6% might need some fancy setups, like setting up storage scopes or other privacy settings so they can function as expected. I won't pretend they are always intuitive, but if you really can't figure it out, someone on the internet has already had the issue and posted their solution online
Another 3% are your banking and similar apps. Some work, but some are needy, and I have these in another profile which has the sandboxed Google play mirror and other Google services that GrapheneOS mocks
Then is your last 1% that are so stubborn that not only require Google play, but they have the audacity to require you be logged in to Google play or they won't start. Personally, I just refuse to use these apps when I get one, as they are random dumb ones, like some incredibly privacy invading videogame app. However I'm sure there will be some app you find that is like this. I guess you can log in, or heck make a third profile and hopefully don't need that app often. I guarantee there is a friendlier alternative that is worth looking into if you do feel you really need that service.
I don't think there is enough evidence yet about either but I can attest that GPS feels improved over my 7. It's only been 1 week though.
The non-Pro 11 has a much larger photo sensor and has more base storage. Everyone has different priorities. I'm not saying which is better but trying to give a more complete picture.
Generally, do not to invest in features that the upstream vendor isn't fully committed to. It might work today, but tomorrow or next year, etc., it might not. They might even deploy other things that are incompatible or do other things that assume the feature is disabled. In fact, plans to do those things might be the reason they disabled MTE.
Also, never support products or projects that won't work well. When someone's Pixel 11 is slow, they will think and post that 'GrapheneOS is slow'. Blaming Google will not be very credible and will look like you're just trying to duck responsibility. Your brand's name goes whatever you release; people will associate their experience with GrapheneOS (also because GOS is the new, unconventional thing; Google is the established norm); if their experience is bad, they will associate it with GOS.
Their real focus is on their Motorola partnership, anyway.
Good security engineering isn't sufficient: GOS could accomplish the security engineering by making an OS for themselves on a private git server and never distributing it.
But if we want to help the public with security and privacy, then the outcome depends on GOS's brand image - perception drives public adoption, who will never read long posts on arcane engineering matters. It depends on GOS's relationship with Google and its individual engineers, a relationship in which Google and the engineers hold all the cards - they could choose to cut off GOS completely tomorrow at no cost to themselves.
It's an engineer's fantasy that you can ignore those things and just do engineering; life just isn't like that. Doing that would imperil or greatly limit GOS, which hurts everyone.
Why disabled? Well, its performance is apparently quite poor. But why?
...Apparently to save money.
Is the thread wrong or did I misread it?
So the TLDR is that the main issue is that it causes a whole bunch of stuff to segfault when bad memory accesses occur but where they wouldn't segfault without MTE. It makes the phone feel unstable to average users since stuff just crashes with no real end-user-facing explanations.