Write signatures, not schemas
catern.com
catern.com
- Libraries often CAN'T be written in a separate language without an additional effort -- and often writing with interop in mind means you have to give up nice features like C++ STL containers
- Libraries often CAN'T have separate build environment -- want to switch to C++14 but you have a third-party library which is stuck in C++03? Tough.
- Libraries often CAN'T isolate failures. A segfault or OOM in library will kill your app. Not so with external process.
- For many languages, libraries CAN'T have access to resources that the rest of the program cannot access. Other that few exceptions (browser Javascript, Java with non-default security settings, or exotic execution models), most languages have "escape hatches" which allow underlying OS access, thus bypassing any language-based restrictions there are.
- Libraries often CAN'T have effective resource management. Trying to find out how much memory does library use, or changing library IO priority is very hard. (putting all library in a separate thread might help with CPU usage but not with other things)
There are definitely times when you want a library -- your "priority queue" class should not be out of process. But for things which can tolerate extra latency / interface limitations, consider making them a separate process.
But just to respond quickly to these directly:
- Writing in a single language, not multiple languages which requires you to speak IPC protocols, is the whole point.
- Added a note to the article
- Added a note to the article
- Several of the techniques I listed apply to C - raw, unsafe native code. If they work for C, they work for all languages.
- You can change the IO priority of threads, and track their IO usage separately. The question of separating memory usage is, I think, sufficiently addressed by the other points in the article (if you're using SFI, for example, you're already going to separate memory usage)
> You can express far more sophisticated abstractions [...] without losing any functionality.
but "support multiple languages" is definitely a functionality and a very important one, too. Did you want to say:
"You can express far more sophisticated abstractions with a type signature than you can with a protocol schema, _as long as you are willing to limit yourself to a single language and give up on resource isolation"
----
Re "Software fault isolation", I think you are simply wrong there. The SFI paper you cited is about NaCL sandbox, and with NaCL you are not going to be passing arbitrary structures to the app -- with NaCL programming (and all other call gates), you have manually serialize/deserialize messages -- and once you start writing serdes function, you might as well go full remote protocol to gain support of other languages etc..
And related to this topic, you want to clarify: are we talking about production languages which people write programs on, or are you talking about rewriting existing code in a whole new language in order to get rid of separate processes? Because if my choices are either "put it in a separate process" or "rewrite whole thing in NaCl or Java" I am not going to be choosing the latter option.
>- Libraries often CAN'T be written in a separate language without an additional effort -- and often writing with interop in mind means you have to give up nice features like C++ STL containers
And I said something that wasn't quite right.
What I should have said is: Yes, writing a language-independent library can be harder, because you need to use a language-independent type system to describe the signature of the library. But it's harder still to write a standalone process, because protocol schemas are even less capable than language-independent type systems. And no, you don't need to avoid STL or anything.
>Re "Software fault isolation", ... you have manually serialize/deserialize messages
No you don't. For further reference, also look at the other library privilege-separation techniques I linked.
>are we talking about production languages which people write programs on
Yes. Such as C.
> Yes. Such as C.
Am I missing something? Your page lists the following privilege separation links:
- Google's SFI paper: that only works in NaCL environments, and does not support "regular" C libraries
- Wikipedia link to capabilities - no mention of C there, only Java and C#
- Stack inspection - trivially bypassed in C
- "Capability-safe architectures such as CHERI" - does not apply to common x86 servers.
- "Multics-style "call gates"" - this is Multix specific, no such facilities exists in modern OSes
So, if I have a modern app written in C or Python, which runs on regular Linux machine, and let's say there is a library which keeps running out of memory and crashing - what are my options? It looks like none of the links you mentioned apply.
So let me ask again, are there any modern options for library isolation, something I can actually download and link to a C program? Something with github page I can visit and download the code, and a community that can support/answer questions? And if such thing exists, is this easier to use than your favorite IPC system?
-----
Re NaCl:
NaCL definitely requires some kind kind of serialization/deserialization code. The paper [0] is pretty clear that NaCl-isolated code lives in a separate process (see Figure 1, and section 2) and can only communicate over IMC, which uses "datagrams consisting of untyped byte arrays along with optional NaCl Resource Descriptors”. And you need serialization to send over untyped byte arrays. Moreover, at the end of section 2, the paper says: "It is not appropriate for modules requiring process creation, direct file system access, or unrestricted access to the network"... which rules out a lot of the interesting libraries.
[0] https://static.googleusercontent.com/media/research.google.c...