Pony – High Performance Actor Programming
ponylang.org
ponylang.org
Unfortunately, it turns out they have simply coopted the term to mean something completely different. According to the documentation: "Pony capabilities are completely new, no other language has them."[0]
Please think of a new word for this; don't redefine one (or a pair) that is widely-used already.
We use capabilities in Pony to statically avoid data races - not for security. But we believe that Pony's capabilities can also be developed further to support security - would be great to try and program Sandstorm’s security with them.
Right, capability-based security (my definition) originated in the 60s and has been in continual use ever since. Pony's definition is new, AFAIK (and according to the documentation).
> We would have loved to use a different word, but there are not enough different recognisable words in the English language.
I think "type qualifiers" is the correct term. The Pony docs already describe capabilities as being similar to C++'s "const" and "volatile" type qualifiers, and this term will help people understand that the concept is similar (but more advanced).
The Pony docs spend a whole paragraph explaining that its capabilities are not the same thing as any other "capabilities" I may have heard of, e.g. "You may have heard of capabilities somewhere else, but these are not the same thing." The fact that you need this paragraph suggests that you've heard this objection a number of times, which should be demonstration enough that you really ought to choose a different term. People don't usually need disclaimers like this. English is a language that has entirely too many words -- there are plenty available. Or, consider making one up. :)
> But we believe that Pony's capabilities can also be developed further to support security
Yes, it's very likely that your "capabilities" would compose well with object capabilities in order to effect security. But that's just all the more reason they shouldn't use the same word, since it will be incredibly confusing to anyone who is trying to use both.
You may be interested in the type system paper that covers this:
http://tutorial.ponylang.org/capabilities/introduction/
It seems to state that the type qualifiers are the capabilities, and explicitly claims no existing language support capabilities.
If this is just an issue of misleading documentation then I apologize for raising a false alarm, though the documentation should be corrected. :)
No worries on a false alarm, as I am always eager to discuss capabilities security, in any form :)
If Pony were using "capability" to describe the combination of a variable and its qualifiers, then I'd say, yes, that's a capability -- it designates a thing, and the permission to use it. But they are using "capability" to describe specifically the _qualifiers_ on the variable, which is not correct, and is confusing.
Here's a great paper discussing what capabilities are and the confusion around them: http://zesty.ca/capmyths/usenix.pdf
Before you ask: Yes, Linux/POSIX capabilities (as described on the capabilities(7) man page) are similarly misnamed. This is covered in the paper, and has been a source of a lot of confusion.
1960s-1970s: Capability-based security invented, popularized, widely used, almost makes it into hardware (Intel 432).
1980s-1990s: People incorrectly redefine capabilities in various ways (e.g. as verbs rather than nouns), then declare "capability-based security doesn't work" based on these misunderstandings (see the paper I linked). Capability systems find they have trouble making headway against these misconceptions, as the difference between the misconceived version and what capabilities really are are just subtle enough to confuse anyone who isn't immersed in it. (Meanwhile, the basic ideas of capabilities live on as object-oriented programming, but aren't enforced strictly enough to be called "security".)
2000s-2010s: People start realizing that capabilities as originally defined do in fact work ridiculously well and they become re-accepted.
We really don't want to go backwards again. :/
That was (is?) all about capabilities.
1. Your documentation introducing capabilities specifically says: "Pony capabilities are completely new, no other language has them." If you really mean "capability" in Mark Miller's sense of the word, then this statement is incorrect. Many object-capability languages exist.
2. What your documentation describes as "capabilities" looks to me like type qualifiers. The documentation says "A capability is a form of type qualifier and provides a lot more guarantees than const does!" But this doesn't sound anything like capabilities.
Is it possible that Pony is an object-capability language but you are also using the word to refer to a different concept? If Pony is actually an object-cap language then that's excellent, and I even more so encourage you to rename the type qualifier concept to avoid the confusion!
FWIW, Sandstorm is very interested in adopting a high-performance ocap language, but would probably shy away from one which uses the word "capbaility" to mean something else, for fear of confusing developers.
Sorry for the docs misunderstanding - I promise, bad docs not withstanding, we are using "capability" to mean exactly what you want us to mean :)
"It's very similar to earlier versions of rust. In terms of actor local GC, language provided scheduling of actors, separated function and async invocation system, mixed structural and nominal types, and an overabundance of language supported reference qualifiers. I think they did a good job at it, certainly much better than I did, but I'd be nervous about the cognitive load of the reference qualifiers (capabilities). See this page for example, or the previous few in the tutorial. We had that sort of cognitive load on variables early on and people basically rejected the language because of it, as well as associated compositionality problems. The many years of design iteration had a lot do to with factoring that space into fewer, more essential and general qualifiers. Slow, tedious, world breaking refactoring work. I hope for pony's sake this doesn't happen to them too -- it looks really nice -- but that's my biggest worry for it."
http://www.reddit.com/r/rust/comments/34rszb/pony_type_and_m...
In our experience, programmers can pick up the system fast (e.g. after a 2 hour discussion).
Similarly, programmers can easily write programs which type-check, even if they do not have a deep understanding of the type system (e.g. they need not memorise the table you mentioned). Type-checking is enough - given the guarantees by the type system, (data-race freedom and atomicity).
The use of defaults reduces the annotation burden - typically to only 10%-20% of locations which allow them.
Regarding annotations, the "Fast & Cheap" paper contains the following:
The language uses carefully chosen default capabilities to minimise the required annotations. In addition, the compiler guides the programmer as to which annotations should be used, infers annotations locally, and performs automatic recovery in some circumstances. As a result, when implementing LINPACK GUPS (in app. F) we require just 8 capability annotations and 3 uses of recover in 249 LOC. In approximately 10k LOC in the standard library, 89.3% of types required no annotation.
Syntax may be less important than a language's features, but it isn't until the 4th page of the tutorial that I even have an inkling of what code in Pony looks like.
I'm excited to give this a try!
Once I've understand it was a new language (cool name btw), I've look for the Hello-world example to see if the language was easy enough for me. I've found the link after reading the HN comment: http://tutorial.ponylang.org/getting-started/helloworld/
Then, I've quickly read the part on concurrent programming. But I've not understood how it is easier with Pony. It's very frustrating. So far, my understanding is there is class (called actor) that can discuss together by sending each other immutable object. I would like to see some code in the tutorial that show how concurrency is easily done in Pony.
I must not be understanding that claim correctly. Erlang does garbage collection on a per actor basis (at least that's my understanding). If the claim is that multiple actors can garbage collect simultaneously, then I guess that means Erlang only GCs one actor at a time. If so, why can't Erlang GC multiple actors at a time? It does full SMP for actor execution.
It does. The article is incorrect. Erlang has a fully concurrent garbage collector among actors. One actor's GC running on one CPU scheduler will not interfere with execution of actors running in other CPU schedulers.
The research paper is here:
http://ponylang.org/papers/OGC.pdf
The Pony object garbage collector is fully concurrent, the reachable memory for any actor is GC'd totally independently. At the same time, Pony allows (safely, with no data races) sharing pointers across actors, for performance (ie without copying).
There's a paper on the type system that allows this:
Erlang's method is to form linked webs of processes and then the death of a process "poisons the web" and kills off all processes in the web. By trapping exits, you can put in stopgaps for this behaviour, which is what supervisors do, among other things.
Process handling in Erlang is more akin to "manual memory management" or ARC/RAII style memory management here.
This seems pretty impossible in distributed Erlang. Perhaps the Pid was sent to another node (which may be alive but not currently dist connected), or was serialized and may be deserialized later.
This section in particular: Benchmarks and preliminary comparisons with Erlang, Scala, Akka, and libccpa Table1 & Table2
It seems that you have the exact same numbers for Erlang and Scala in table2, this is very hard to believe. You either accidentally put down the same number twice or you measured the performance of your benchmarking tool, otherwise it is extremely unlikely that two entirely different systems give the the same exact measurement. Similar story in table1. maybe I am missing something terribly obvious but this looks off to me.
The numbers are taken from another source and it seems they didn't have the exact numbers, hence the ~9s figure which is then used to calculate 333,333. They seem to have gotten the numbers by looking up the graphs on this page:
http://libcppa.blogspot.co.uk/search/label/benchmark
Makes me wonder how precise you could be at getting the numbers from just the graphs. Pixel precision at 800x600 is probably not too bad.
If I were looking to compare two things, I would run all the benchmarks on a single machine under my control. I might look at previously published reports to make sure I was getting comparable numbers. But there is no way I would publish a comparison that I merely hoped was apples to apples. I've just had too many benchmarks depend on subtle issues, ones that I had presumed were irrelevant.
For arguments' sake here is what a simple Erlang actor looks like:
loop(State) ->
...
loop(NewState).
...
Pid = spawn(fun () -> loop(InitState) end)http://erldocs.com/17.3/erts/erlang.html#hibernate/3
Hibernate compresses the current memory, save it and then kill the active process. Then if a message is sent to it, it "wakes" up.
I don't recall Javascript supporting or having any library that would enable actor model of concurrency.
The general idea of actor model is that each actor is a process with a queue (mail box) and they talk to each other via msg. Javascript does not have such a thing nor does a server actor?
However, Pony makes a messaging order guarantee that's much stronger than is typical for the actor model. It guarantees causal messaging. That is, any message that is a "cause" of another message (i.e. was sent or received by an actor prior to the message in question) is guaranteed to arrive before the "effect" if they have the same destination.
That guarantee is achieved with no runtime cost, which is pretty fun.
Pony still allows unbounded non-determinism, of course, but causal messaging gives a very comfortable guarantee to the programmer.
But I agree, exactly what guarantees they're trying to provide (and is it opt in?) is an interesting question.
See p558, "Example 3"
But your claim might be stronger, though I'm not sure in what sense it is stronger.
In Pony, it holds for the entire program. So, if A1 sends a message to A2, and then A1 sends a message to A3, and in response to the second message A3 sends a message to A2... in Pony, A2 is guaranteed to get the message from A1 before the message from A3.
On the other hand I could imagine that this could also cause problems: If you have a large program, could then this causality cause some messages to be delayed for a (too) long and not directly visible time in order to achieve the causality guarantee? But the claim was that there is no runtime overhead.
You have an actor A13 that communicates with actor A2. The guarantee allows you to break up A13 into the two actors A1 and A3 that now both communicate with A2 and be done with it.
In erlang, you might not've done so, because you know nothing about message ordering in non-tree-shaped topologies.
Locally, a message pass never fails and always delivers the message directly. It can't be in-flight, so m1 is placed in the mailbox before m2 happens, and thus m3 will always come later.
In the distributed setting, A1, A2 and A3 might be living on three different nodes, and then m1 might get delayed on the TCP socket while m2 and m3 goes through quickly. Thus we observe the reverse ordering, m3 followed by m1. This is probably why the causal ordering constraint is dropped in Erlang. Is pony distributed?
For now, check out some of the included packages, like net/ssl. That calls C extensively (libressl, or openssl if you like). Or regex, which uses PCRE2.
Anyone know what version 6.1 is refering to? Latest Erlang OTP version is 17.5. Version 11 was released sometimes 10 years ago.
Erlang/OTP 17 [erts-6.4]
What the hell am I missing?
> No loadable code. Everything is known to the compiler.
[1]: https://github.com/CausalityLtd/ponyc/search?utf8=%E2%9C%93&...
Also, your 3-clause BSD license only has two clauses.
https://github.com/CausalityLtd/ponyc/blob/975d9f6c60553030e...
Obviously, I have never written anything in Pony, but this absolutely seems like an advanced use case. It exists because 'C' has a concept of null, so it needs to be handled. The manual even admits that all bets are off when dealing with the FFI.
It's analogous to Maybe, in Haskell.
Capability defaulting seems a little dangerous (not that it's not type safe, but it having such information be implicit seems like a good way to encourage people to forget about it; on the other hand, if the compiler is good enough about warning people it shouldn't be a problem).
I'm not sure subtyping is worthwhile for this language (since it complicates the type system without much of a gain wrt constrained polymorphism).
1. How does process failure and recovery work?
2. Are there any plans to add binary patterns? That's one thing I'm sure I'd miss from Erlang if I wanted to write a server in this language.
2. Yes! Pony has first class pattern matching already (along with algebraic data types, i.e. tuple types, union types, and intersection types, to make pattern matching really useful). It allows matching both on type and on standard or programmer defined structural equality. As I mentioned, we're big Erlang fans, so we definitely won't ignore stuff like this.
Bonus: if this language thing doesn't work out you can just pivot to selling the unlimited-resource and failure-free hardware and operating system you've invented...
Snark aside, a system that always assumes "clean shutdown" is going to massively simpler to design. It's also going to be unusable in real-world environments.
http://tutorial.ponylang.org/capabilities/introduction/ is the best concise description why concurrency is hard I read until now.
The question is, what does X is waiting for Y mean? 'Waiting' would imply that there are language constructs to actually wait. Pony doesn't have that. An actor that is blocked (i.e. has no messages) and that satisfies necessary RC invariants will be collected.
If the program can implement a lock between the two actors, then your language is not deadlock free. Even a loop by checking if some variable has changed by the other actor (if possible in this language, I still don't even know by the deep lack of examples), and the other actor doing the same thing, that can be claimed as deadlock.
Anyway, there is only a send construct (essentially a remote procedure call that returns no value) but no receive construct. Here's an example of passing data in a ring: https://github.com/CausalityLtd/ponyc/blob/master/examples/r...
As there is no blocking receive, no immediate way of breaking this came to my mind, although I had the same initial thought.
Is there a specific language you would like to compare with?
I wanted to know if these "end"s are optional (I can write it like Python) or strictly needed (I have to write it like Ruby).
It is unnecessary on the top-level though, since the top-level expression will end where the next one begins. Which means you probably can't have nested class/function definitions.
This technique is not new and is used in OCaml, for example.