What Are Capabilities?
habitatchronicles.com
habitatchronicles.com
In Capsicum, capabilities are tracked on a per file descriptor basis. A typical program opens any input paths, restricts capabilities on open fds (including stdio), and then enters sandbox mode.
Once the capability mode sandbox is entered, no additional capabilities can be granted, and filesystem access via absolute paths is revoked. (Sandbox mode programs are expected to access files they already have open, or to use relative accesses to walk file trees.)
Erroneous system calls generate ECAPMODE or ENOTCAPABLE errors if they attempt to perform actions they do not have permission to do.
So for example, if you have an open dirfd with the LOOKUP capability when you enter the sandbox, you can access the following relative paths:
".", "./foo/bar", "./foo/.."
But you cannot access these relative paths:
"..", "./foo/../..", etc.
Talking to one of the "O-Cap Gang", Norm Hardy (http://www.cap-lore.com/CapTheory/) a few weeks ago, he said, to the point of creating secure distributed computing architectures: "Object capabilities are not easy, but they're the only way I see forward."
I'm biased to agree.
Capabilities are not the way developers are used to thinking. It may be that if people got used to it, it would make sense -- or it may be that it's simply too complicated and most developers will never wrap their heads around it. I honestly don't really know which it is. In this respect, it feels a lot like functional programming (FP and capabilities are very different concepts, but are similar in that it takes a while to fully understand how to use them effectively).
We do see capability systems appearing by accident all over the place. Unix file descriptors, for example, behave very much like capabilities. Once open() finishes successfully, all access checks are done -- they aren't repeated for every read() or write(). You now have a capability, and you can even send that capability to other processes (even running under different UIDs that wouldn't have permission to open the file directly) via Unix domain sockets.
People who really get how to exchange file descriptors build some amazing systems with them (without ever realizing they're building capability systems). But, for some reason, most systems do not really try to take advantage of FD passing, and instead opt to do things like listen on a globally-accessible port and perform an authentication handshake on incoming connections. I guess this kind of design just seems more obvious to people?
But often, as systems get more complex and involve more parties, they end up forced to use capability-ish designs, because ACL-based designs get intractable. OAuth's use of bearer tokens, for example, is very capability-ish, albeit not acknowledged as such, and a fairly primitive form of capabilities. This sort of design turns out to be necessary in large-scale systems. If people would learn capability theory before designing such systems, they'd probably be more secure and powerful.
1) Odd that something like this hadn't been written for Unix years before as a way to avoid "set uid" programs and
2) passing FDs in Unix is a pain.
3) only Linux supports passing a PID along with the FD request (and Linux may be the only one to pass UID/GID as well, which may explain why 1 wasn't done).
I don't really use that program as it's utility is limited. Shame, as I think it's a good idea but way too late to be of any use these days.
[1] https://github.com/spc476/ipacld
[2] You can see a set of "rules" in this file: https://github.com/spc476/ipacld/blob/master/ipacl.lua
None of which I really have excess time to noodle about, but is now unavoidable. Thank you, sir.
Too many people think OOP means deep class heirarchies, implementation inheritance, AbstractSingletonProxyFactoryBean, and other things you might see in Java. I think this turns a lot of people off.
Good OOP is none of that. It's simply associating data with code that operates on it, in order to get two good things: abstraction and encapsulation. Abstraction is by defining virtual tables, encapsulation is by hiding implementation details as private members.
Object-capability discipline leverages encapsulation as a security primitive (you can only perform the operations allowed by the public interface), and leverages abstraction to implement attenuation via wrappers (e.g. a wrapper that blocks certain methods, or a wrapper that can be revoked later). It has zero use for class hierarchies with implementation inheritance.
So yeah, I think studying ocaps does help one understand how to do OOP right.
And yeah, I think learning FP can indeed help you gain clarity over a different aspect of programming -- something basic about organizing computation and data structures, which complements OOP and ocaps. (I seem to be one of few people that don't see any inherent conflict between OOP and FP... they are orthogonal and can be used together.)
It really depends on which aspects you see as essential to the paradigms. If you define functional programming as programming that makes use of HOFs, or that encourages referential transparency, then you can combine those learnings with OOP nicely.
It's also possible to characterise them as in opposition. For example, you say
> Good OOP is none of that. It's simply associating data with code that operates on it
If I wanted to characterise FP in opposition to that, I would say that Good FP is keeping the state (i.e. the data) dissociated from the code in order to gain the benefits of referential transparency, composability, reusability and abstraction (attaching code to state massively reduces your ability to abstract).
FP tends to encourage the reuse of generic datastructures and vocabularies much more, while OOP likes to encourage you to create your own datatypes with their own vocabularies.
Personally these days I like coding in an OO style with lots of learnings taken from FP, but the worldview that sees these as different poles on a continuum is not difficult for me to appreciate.
https://books.google.com.vn/books?id=_bmyEnUnfTsC&lpg=PA56&p...
However it is used in an insufficiently fine grained way. It does highlight some ways that it may not work if used for desktop, programs would just ask for large-grained capabilities (can I access all your pictures/files etc) and you get back to ambient style authority.
I think expecting the user to be an intelligent judge of what capabilities a program needs, is not realistic. I think something like agoric auction of capabilities might force programs to provide useful features to get access to data, might work. I'm exploring it anyway.
The ability to substitute is critical to a good capability system, because it allows finer-grained control over what the app can or can't do. E.g. you could, say, implement a custom microphone that reads from the real microphone when you're out in public, but produces only silence when you're in your bedroom. Without the ability to substitute, we must rely on the OS designers to provide all possible options for us, and of course they don't care to implement many options.
(OTOH, if you were talking about Intents, they are pretty capability-ish, though not always used well. Or if you were talking about some of the system internals like Binder, yeah, there's a bunch of capability-ish stuff in there.)
(He made that suggestion back before Android etc. existed.)
People are pretty OK at judging and tracking what to entrust other people with. This suggests the problem shouldn't be impossible, but that our interfaces don't map well to those familiar abilities.
Isn't that similar to getting people used to installing anything they find on the web on Windows and just pressing accept for everything?
I'd say a viable system needs to explicitly address special authority and make users are of it as an important thing, but still within the normal course of UX, to keep the distinction meaningful.
The current state of Android is in the other direction: it explicitly addresses authority for many accesses, but they're not fine grain enough; so apps ask for a lot, getting people used to accept early without having a clear idea of what the app will do in the future, making it cheap for the programmer to ask for many accesses.
The authority is not fine-grained enough because grants of authority are not dynamic (meaning they happen up front when the app is installed, not while the user is using the app). If the grants were dynamic, the context is clear to the user, who can therefore 1. understand what the grant is for 2. make an informed decision 3. use UI patterns like "powerbox" in which there is no "security UI", just UI that would be needed anyway. When that is true, finer-grained can be easier for the user, not harder.
Of course there is more to it, e.g. kentonv's important point re substitutability. Some otherwise capable programmers don't seem to understand that point even in the domain of programming (as opposed to UI).
One possible answer: maybe the right system would use "ambient authority" (ACLs, etc) for the common case, but scale up to true capabilities for interesting corner cases.
Creating capability UIs that compete, in the common case, with the normal ambient-authority design, is ferociously hard. This seems to be where most capability systems falter. The hardest UI problem in the leading modern capability OS, Sandstorm, is certainly the capability "powerbox."
Think about all the places you can type a filename or a URL. If the designator and the authorization have to travel together, and the pair is an unforgeable capability, then user-entered designators (like paths or URLs) are not practical. Designators can't travel through unsecured paths, through the human mind, etc. This is a painful limitation.
That said, the OP is the best explanation of capabilities I've ever seen. Everyone really should read it.
Heh, I'm not sure I'd describe Sandstorm as "leading" anything right now, but thanks. :)
I think Sandstorm made the problem more difficult for itself by not just trying to solve this UI problem, but trying to do it in a way that can be integrated into existing codebases that were never designed with capabilities in mind. This is the really difficult part: dealing with legacy code. The powerbox itself is otherwise "just a picker", a fairly straightforward concept.
For example, the standard file-open dialog box in everyone's browser, when invoked from an upload button on a web site, is actually a powerbox: the web site itself is given no access to any files except the one that the user chooses, but the file-open dialog is able to see everything.
But imagine trying to graft this approach onto an existing native app, that is used to being able to read the hard drive directly and already has its own custom file picker UI built in. You have to convince the developers of that app to remove their nice custom UI and instead invoke the system's UI. Moreover, when they invoke the file picker, they now don't get back a file name, but rather an already-open file descriptor. So their code has to be adjusted to not expect a name. This could be hard, depending on how the code is organized: maybe it uses a bunch of libraries that expect to open a file by name, etc.
Sandstorm is trying to graft this model onto apps that are used to making HTTP API requests to each other directly.
We had designs figured out to make this work, and even make most of the process look like an OAuth request, so that existing OAuth-based apps might not need to change very much... some of this is implemented, but mostly we just didn't get there before running out of money. (I am still working on it in my spare time, though!)
This is a great answer. I think the impedance-mismatch question is a big part of the problem. Personally I prefer to boil the ocean rather than drowning in the impedance-mismatch tarpit. Both of these strategies are, of course, profoundly insane.
It's true that a file-open dialog box easily becomes a powerbox. But of course, you are using the ambient authority of the user's login session to ascribe authority to the path that gets typed. So in a way it's the exception that proves the rule. And does a URL bar easily become a powerbox? What about a command-line shell?
Perhaps we're both just agreeing that there is always trivial ambient authority, and the right way to scale up to nontrivial cases is a capability model.
I just worry that when too many people discover the beautiful screwdriver that is capabilities, they decide that everything should be a screw. But the world has nails. It may even be true that the world should have nails. And then you find yourself pounding in nails with the butt of your screwdriver.
Have you considered an ICO? Asking for a friend :-)
Whether or not the URL bar of a browser is a kind of powerbox depends on how extreme you want to get. In a pure capability model, yes, it would have to be. And hypertext would no longer be plain data -- it would be data with embedded capabilities (for each link), and would have to be transmitted via a protocol that can track capabilities. Obviously that kind of stuff, even though it would be amazingly powerful in a lot of ways, is not practical. :/
> Have you considered an ICO? Asking for a friend :-)
People keep asking me that, often citing Urbit as an example to follow. :) At the moment I'm having too much fun at my day job building Cloudflare Workers (an unusually practical project by my standards!), and want to focus my outside-of-work time on coding Sandstorm rather than fundraising for it. In the future, who knows.
The weak capability position is that it should be possible to construct an opaque combination of designator and authority, ie, a capability. The strong capability position is that designator and authority must always follow the same path.
So in the "pure capability model," you are actually subtracting a feature from the system. Now, of course, it's often the case that subtracting a feature makes a system better. But one needs to be really clear about confusing this with the positive virtues of just having capabilities.
Also, while I agree that capabilities are really great (Arvo has a somewhat capability-like internal communication model), it's important to point out that (a) capability vs. identity is a duality; and (b) it's cheating to compare a good capability system to a bad identity system.
For (a), note that identity systems can implement capability features like delegation and revocation; they just have to do it actively rather than passively. If I am willing to act as your deputy, I can delegate my power to act to you. Or some restricted subset thereof. Tell me what to do and I'll do it.
For (b), if you put a decent capability system next to the Internet, a network whose identity model is a burning-tire dumpster fire. If your identity system actually works and makes sense, competing is harder for the capability design.
I do think Urbit should have some kind of capability model. However, this has the feel of a 2.0 or even 3.0 feature. I hate to solve a problem until it is actually in my way. I feel we have come nowhere near the limits of the common case. A simple map from identity to privilege is certainly that.
And while capability UIs may be doable, identity UIs are trivial. This would not be the first case of humans being funky, hence screwing up our elegant architectural logic.
If you have a system software project and it can advance without being lashed to a company, definitely do it that way! It's always a poor fit -- they have to succeed on different timelines.
One thing I know: once we all have personal servers, today's cloud will look pretty lame in retrospect...
Can you explain that more? Do you mean that both caps and ACL systems are ways to organize an access-control matrix, either by rows or by columns? Because I think that idea over-abstracts the reality.
Of course, like every abstraction, this abstraction breaks down as you get into the details...
The most important things left out of that view I think are the way an agent expresses its intent (vs. ambient authority) and the way the whole system changes over time. I started writing some more, but it's late and http://srl.cs.jhu.edu/pubs/SRL2003-02.pdf goes into this at length, so I'll just leave that here.
Your example of the file open dialog box is identical to what Apple did on MacOS when they implemented sandboxing. Rather than using regular paths in NSString objects, they return you an NSURL with a "security scope" - a capability. These can be saved and reused across process terminations.
The docs provide useful information how they are used: https://developer.apple.com/documentation/foundation/nsurl
Unfortunately, the concept of "taming", where ambient-authority systems are used to implement functionality in capability-safe systems, is often a sad and frustrating path due to massive impedance mismatch. Like you say, paths are not caps, which means that capability-safe languages on UNIX-like systems are constantly dancing to tame the OS.
As usual, it would be great if you made your VM cap-aware, so that we may eventually get over this hump.
The traditional mitigation is to use hardware tagging and permit the tag bits to be set only via a privileged instruction, e.g. Multics. The x86 segmented memory model also allowed this, and I used this to build some experimental SUID memory-mapped the 1990s (you could call into them via special gates, but not read the instruction memory, for example). Although you those memory models are still supported on x86 for back compatibility I don’t believe they are particularly fast any more, unfortunately.
Edit: The lowrisc (RISC v) folks are implementing tagged memory also btw
Modern capability systems don't rely on hardware support and performance issues aren't usually cited as a problem with capabilities. In fact, capabilities can often perform better than ACLs since ACLs tend to require performing an expensive access check on every operation.
IMHO the 432 had other structural problems and it wasn’t the tagging that killed it but it had enough problems that this is definitely not worth disagreeing about — perhaps that was indeed the straw that broke that camel’s back.
Operating systems have tweaked the performance of user->kernel transitions to the point that anything placed on that path represents massive degradation in performance.
Until we start having to pay performance for security, no one is going to bother. Meltdown and Spectre are really the first security attacks that will require significant amounts of performance to mitigate.
Once we get enough of those kinds of attacks, people will start being willing to trade performance for security.
If your friend is really good at fooing widgets, you don't want to do it yourself, and you don't want to continually bring all of your heavy widgets to them, then you can make a copy of your key that can only access the part of your locker holding your widgets and give that to them so they can come by and foo your widgets for you.
With this system, you don't have to trust your friend to not mess with the other contents of your locker. You don't have to trust them to not take the money or personal documents you have in the other part of your locker. Maybe you believe you fully trust this friend, but maybe you don't fully trust their key-handling. If you gave them a restricted key, and you think one of your personal documents is missing from your locker one day, then you can skip worrying about whether it was your friend that's responsible. If your friend loses their key one day (they misplaced it, or someone stole it from them), then you don't need to freak out and worry about your personal documents; it's only your widgets that are in danger. If someone knows that your friend is given a lot of unrestricted keys to people's lockers, they might be incentivized to try to steal/blackmail/threaten the friend for their keys so they can get to people's important documents; if it's known that the friend always requests restricted keys from people to foo their widgets, then an attacker who doesn't care about widgets isn't incentivized to try to attack the friend.
This is precisely where "smart contract" comes from. I should probably write a blog post about this.
In particular, many IPC mechanisms that involve more than a few parties interacting -- especially when some or all parties are intended to be sandboxed -- become capability systems.
Examples of real-world capability systems include:
- Android's Binder
- Chrome's Mojo
- FreeBSD's Capsicum
- Linux's Bus1 (not yet merged into mainline)
- Classical Unix domain sockets (in that you can pass a file descriptor over a socket).
- Cap'n Proto RPC, especially as used in Sandstorm.io (disclosure: these are my creations)
- The Google Docs "anyone with the link can access" mode and e-mail invitations to non-Google accounts (this is a relatively weak and unintentional capability system).
- OAuth 2's use of bearer tokens (also relatively weak).
> does this imply that the capability design only works internally and you cannot pass caps to other applications?
It's best if the applications agree on a capability protocol, but in some cases you can trick applications into behaving in a capability-based way without explicit support. For example, using secret, unguessable URLs can allow for capability-like design patterns -- although at the risk that secret URLs are relatively easy to leak (stronger capability systems can protect against accidental leakage).
There are things you can do: view a resource (in practice a page or a particular field of a page), edit a resource, or create/delete a resource.
Users are assigned “roles” which are then given permission to do certain things on various resources. It looks like a big grid, with all possible actions as rows, and the various roles along the top as columns.
Users are assigned a role, which then limits what they can see/do on a website.
This seems sort of akin to capabilities, but not quite. For one thing, there is no easy way to set up the permissions for a single object, which you could then assign to a given user.
To get that level of granularity you would have to create a new “role” and a new object type.
Capability systems include a way to _pass_ capabilities as part of the basic routine operations.
So what does a capability system look like? Before I describe, note: I don't necessarily recommend exposing pure capabilities in a UI for end users.
Capabilities are an implementation technique, but can be used to build a user interface that behaves like ACLs from the user's perspective. You'll find, though, that once you've built your underlying system on capabilities, it's much easier to add useful features to your high-level sharing UI that go beyond what can be easily represented with ACLs. For example, it's easier to add secure delegation.
With that out of the way, what would a pure capability system look like, if we exposed it directly in a user interface?
Let's say you want to give access to Bob.
In a pure capability system, you don't assign a role to "Bob". Instead, you create a capability for the role, and you send that capability to Bob, via some arbitrary communications mechanism. Bob uses the capability, which grants him access.
Crucially, there is no need for the system to have any notion of how to authenticate "Bob". It doesn't care if the user is really "Bob", it only cares that the user presents the correct capability. This is where capability systems are powerful -- they avoid the need for any centralized list of principals (user identities) and avoid the need for a way to authenticate those principals. This is especially helpful when you need to, say, delegate some responsibility to an automated system that shouldn't be treated as a full user.
But does this mean that when someone accesses the capability, the system actually has no idea who they are, and so can't attribute the changes to anyone?
No. In a capability system, we can take a different approach to auditability.
When you create a capability to send to Bob, you can arrange so that any actions performed using the capability are logged as, e.g., "via Bob". Note that this may be a mere text string. The system still doesn't need to know what "Bob" means, but you can now see in the logs which actions were done by Bob. If Bob further delegates his capability to Carol, he may want to add a second label, "via Carol". Now when you look at the logs, you might see "via Bob; via Carol". This means: "Bob claims that Carol performed this action." No one other than Bob actually needs to know who "Carol" is, much less how to authenticate her. Carol could very well be Bob's imaginary friend. Since the assertion in the audit log says "via Bob" first, we know to hold Bob responsible first. We only care about Carol to the extent that we trust Bob.
Now, again, I don't actually endorse creating a UX like this, because not many users are equipped to understand it. But if you think about it, it does emulate real-life interactions. If I lend my car to Bob, and then the car ends up crashed, I will blame Bob. Bob can say "Oh, I lent it to my friend Carol, she was the one who crashed it," but I don't care, I'm going to hold Bob responsible. At no point in this process do I need to check Bob or Carol's government-issued ID to find out who really crashed my car.
KeyKOS/KeySAFE was a whole, capability-based platform designed for high-security:
http://www.cap-lore.com/Agorics/Library/KeyKos/keysafe/Keysa...
Rees did a security kernel for Scheme48:
http://mumble.net/~jar/pubs/secureos/secureos.html
Waterkin server for web apps done with capability-secure version of Java:
http://waterken.sourceforge.net
DarpaBrowser was written in E language:
http://www.combex.com/tech/darpaBrowser.html
Pony is used for Wallaroo. It presumably uses reference capabilities:
https://blog.wallaroolabs.com/2017/10/why-we-used-pony-to-wr... https://tutorial.ponylang.org/capabilities/reference-capabil...
Edited to add Cap n Proto for IPC. Thanks kentonv for reminder. Should not have forgotten!
Someone can share them with someone else.
ACLs can be thought of as server-side mechanisms to increase the "specificity" of capabilities. The typical capability that ACLs work with is "identity", ie "I have access to this account."
The idea is that people won't be able to delegate fine grained capabilities to someone else without giving the broad capability (access to their whole netflix account etc.)
Now, the downside of ACLs is that they're much harder to do in a distributed computing fashion. So instead of blockchains etc. you have to trust a certain backend, and that backend can lock you out or become compromised.
Think of capabilities as client-side controlled access tokens and ACLs as server side controlled ones.
Delegation is a feature, not a bug, because:
> the downside of ACLs is that they're much harder to do in a distributed computing fashion.
That's because large-scale work almost always requires delegation. So using a system that tries to prevent delegation is really shooting yourself in the foot -- but it seems fine at first, as long as the system stays simple. You only run into trouble when things get complex, and the ACLs become completely impossible to manage, but by then it's too late to make a fundamental design change.
Now, you might insist that in some cases, you really do need to prevent delegation. But it turns out ACLs don't really enforce this: I can always set up a proxy server that has my credentials and provides some API to another user, hence delegating my authority with whatever granularity I wish. No ACL system prevents that, hence no ACL system prevents delegation!
Usually, what you really want is not to prevent delegation entirely, but rather to prevent accidental policy breaches. That is, you trust that people aren't maliciously trying to circumvent the rules, but you aren't sure that your users fully understand what the rules are or why they are in place. This especially comes up in compliance scenarios: HIPAA or ITAR restrictions might legally prohibit delegating some data, but the rules are so complicated that not every employee in your company understands them. So, you want to enforce some basic policy rules that prevent them from doing so.
It turns out this is a UI problem, not a security problem. In either an ACL system or a capability system, it's easy to design a UI that prevents such user error, without the need to fundamentally change the underlying security architecture.
But, we're too used to thinking of these compliance problems as something the basic security architecture should enforce, rather than as a UI problem. So we keep shooting ourselves in the feet.
Or what often happens in practice: users just share their login credentials to delegate access. But then this comes with the problem that it's hard to track who did what on an account, revoke access to specific people, or prevent "delegated" users from revoking access from others including the original owner by changing the password.
In other words the capabilities maxim implies that privacy and DRM should not exist.
Privacy, on the other hand, works fine. The model used by the E family of capability-safe languages, like E-on-Java and Monte, allows for a compute node to be totally isolated aside from The Network, an abstract message-passing I/O interface. The resulting model grants nodes privacy within themselves by default.
If you want your code and data to be private, do not put them on machines which you do not control. If you want certain friends to not reshare private photos, do not show those friends those photos. If you have a secret, and you tell somebody, it's no longer a secret.
Personal computing is the thing that is antagonistic to DRM, where "computer" means "can simulate anything, including a better computer", and where "better" means a computer that does what I want, including ignoring DRM.
You can do the same with capability-based systems, if you really want to.
So how do you do this? Let each friend have a unique capability such as "friend"? And then revoke them on the server side when someone is taken out of the group?
In that case you again have a list of which friend is allowed to access a resource, aka an ACL. Which is what I was saying.
Now you may say this is totally ineffective since that friend could ask another friend to be a proxy. But that's not very convenient usually, and involves violating various trust, which is auditable btw since we know whose capability was used. And also in a chatroom they won't be able anymore to speak in their own name, but only the proxy friend's name.
Of course it's theoretically possible to still do stuff, but according to the capabilities motto, no one can be banned from a chatroom or from editing a document as long as SOMEONE can edit it.
For example, a good capability-based sharing system should have a way for a user to indicate that resharing of some resource is not allowed. The effect of enabling this is that the regular sharing UI will be disabled for those people who "aren't supposed to" reshare. This UI might look exactly the same to the user as an ACL-based implementation would.
This may sound "weaker" somehow than the ACL system, but it actually isn't, because neither system actually securely prevents resharing. Both merely make it inconvenient, forcing the user to go out of their way to break policy.
Capabilities support audit trails: http://www.erights.org/download/horton/document.pdf
I highly recommend that you read at least the first of these papers, "Demolishing Capability Myths", which directly addresses your point.
Whatever happened to them?