Erik Meijer: Duality and the End of Reactive [video]
channel9.msdn.com
channel9.msdn.com
It's a hard real-time language for avionics software, BUT it has type inference and duck typing (everything is "symbolic"). SolScript is supposed to look familiar to anyone with a basic knowledge of mathematics. It's also a literate language, because every SolScript file is a Markdown file (and SolScript code is inside Markdown code blocks).
[link redacted]
Also there seems to be no actual proof of safety of the language's semantics (are the semantics formally specified?), but perhaps I missed it.
The compiler can find out the range of numbers and the lengths of lists using static analysis, because SolScript is such a "restricted" language. Functional programmers would probably dump all that information into a type system. We call it "dataflow analysis", but I think the end result is the same.
Note that, all of the static analysis is done on the dataflow graph underlying a SolScript program. In the dataflow graph there are no "functions" or "classes"... The static analysis is much simpler because it is done on something more primitive than the language itself.
In the future, there will be a website dedicated to SolScript itself. This website will be the "technical" website. We already have a domain [2], but there's no website there.
Right now, we are working on getting SolScript into production. This thesis was really the first time anyone ever wrote anything about SolScript. Part of my job, in the future, will be to write blog posts and create tutorials. I'll definitely keep HN posted!
BTW, there's a broken link on your home page: /71of71 instead of /71objectives
- avionics
- type inference
- duck typing
You like challenges don't you? ;)
I like the literate component a lot, I can see why that would be a very big plus when you're building avionics software.
Bookmarked your thesis.
What was your reason for going for 'duck typed' and 'scripted' rather than strongly typed and compiled? (Which I would assume to be a whole lot more suited for the intended branch of industry, so that has me curious.)
At the surface of it this looks like a 'textual spreadsheet' with the lines replacing the cells of the sheet.
This 'spreadsheet' is then evaluated 'x' times per second to satisfy the hard real time requirements.
How do you stabilize the results from one 'tick' to the next? Do you keep a shadow copy that you work on with all the results posted to the next generation?
There is something funny about a person working for a small software outfit choosing a format for textfiles (for really good reasons) because Microsoft or Apple might not be around when the files need to be read. I get your reasoning and agree with it completely but you have to balance that against the chances of your own company not being around anymore either.
How much of the software of the original Boeing 747 is still in use today? I'd imagine there are no fuselages that have not been upgraded multiple times since the first 747's took flight. So the lifespan of the documentation is roughly the same as the lifespan of the software.
Are there any public sources out there?
Here are some of the oldest 747 airframes known to be still on active duty:
http://www.airliners.net/aviation-forums/general_aviation/re...
But that does not say much about the avionics in them.
Do you intend to release the source to solscript in order to profit from the 'many eyes' out there to reduce the number of bugs in the runtime? (Which can't be written in solscript, so effectively will have to be written in some unsafe language.)
I like that markdown trick a lot, that's applicable to just about every programming environment, see:
http://michael-sokol.tumblr.com/post/13112309135/literate-pr...
I say SolScript is "duck typed" and "scripted", because that's how it feels to program SolScript. However, under the hood, SolScript is a statically typed, compiled language.
About the "spreadsheet" idea, that is exactly what the SolScript IDE (Solide) is going to be. There will be a spreadsheet mode, which displays the source code as a spreadsheet. The last chapter talks about Solide, if you want to know more. (The creation of Solide is going to be my job, and I've been watching Bret Victor videos, all day yesterday, for inspiration.)
We have no "deep copy" of the data. The data is simply mutated in place, for performance. The compiler knows in what order it has to update the datacells, so SolScript users don't notice any of this. If I say `b = a`, then `a` will be calculated before `b` uses it. (If you want the value of `a` before `a` is updated, you say `b = previous a`.)
The whole "plain text" argument is about the fact that companies can die. So can (UN)MANNED, of course. The difference is that plain text files will always be readable, in contrast to Word files. You don't need our software to read plain text documentation.
Releasing the source is not something we plan to do (for now). We are not against the idea, but right now we have other priorities. There will probably be a gratis version, even for commercial use. Only that version won't be certified.
Yes, SolScript is written in an "unsafe" language: C. This makes it extremely cross-platform. We can run on Linux, OS X, Windows, iOS, VxWorks... The coding standards of our C code are extremely strict. Avionics style.
And about Markdown, yes that could be applied to other programming languages. BUT SolScript integrates with Markdown on a whole new level. For example, sections in Markdown are the scopes or namespaces of a SolScript program. Markdown is built into SolScript, not tacked on.
Edit: Circular dependencies really make no sense, in a declarative programming language. Defining `a = b` and `b = a` has no clear meaning, so the compiler will throw an error. However, you can define things recursively using `previous`. For example: `a = b + 1 initially 0` and `b = previous a`. In this case, `b` is 0, 1, 2, 3... and `a` is 1, 2, 3, 4...
Well, you post your thesis, I dig in :)
> Let me try to go over them.
Much appreciated, thanks for all your answers, I've read a good chunk of your thesis by now (that's hard work :) ), good stuff.
> We have no "deep copy" of the data. The data is simply mutated in place, for performance. The compiler knows in what order it has to update the datacells, so SolScript users don't notice any of this. If I say `b = a`, then `a` will be calculated before `b` uses it. (If you want the value of `a` before `a` is updated, you say `b = previous a`.)
Ok. That may cause some trouble when there are long chains of circular dependencies but I'm sure you're aware of that and that you have a solution in place (assuming circular dependencies are even allowed).
Again, thank you for all the answers and much good luck with your project, I'll be following your progress, it's super interesting what you are doing.
Considering Avionics domain, using Coq or ATS to prove the correctness of SolScript compiler will help boost confidence.
Duck typing refers to the ability of making shit up at runtime, as in the ability for functions, classes or code to manifest at runtime and to respond to user queries based on runtime info. It doesn't refer to a lack of explicit types in your code, since in a static language types are still there and still have to be something concrete and solvable at compile-time. It doesn't even refer to structural typing, because again, the types themselves have to be solvable at compile time.
I'm a static languages fan and I've also worked a lot with dynamic languages. The ability to make shit up at runtime is much more than the ability to call a method on an object without being explicit about the interface used. This is not about succinctness as as it about programming style. With an expressive static language used to its strengths, as a developer you want to write code that is as close to provably correct as possible, so even if the static language used is very expressive, the style is oriented around modeling the problem domain with types. In a dynamic language on the other hand, the focus is on the raw data that needs to be manipulated and code is written to make shit up on the fly (i.e. method_missing, eval, classes manifested at runtime by means of meta-programming, etc...).
Hence my opinion that duck typing shouldn't be used outside the context of dynamic languages, because people might get the wrong idea about it.
My paper is in german, but I'll publish it on github and translate it (time given). I've also got a hypothesis on how FRP can be abstracted into a domain model using AI and non-axiomatic reasoning. But I hope I don't run in a dead-end there though, it's just a theory :)
Reactive programming is ill-defined and over-hyped, so let's talk about types and math instead (plus some not-very-subtle in-jokes about Microsoft).
An enumerator is basically a getter with the ability to fail and / or terminate. It might also return a promise rather than a value. An enumerable is a getter that returns an enumerator. We can express all this very cleanly in generic types. And if we take the category-theoretic dual of these types we get the observer and observable types.
So who needs reactive? ;-)
[1] http://dl.acm.org/citation.cfm?id=3988
Rx is a one trick pony that focuses exclusively on the duality between event streams and data streams. Management of time in any real sense is completely missing from the framework. No continuous bindings, control flow must all be buried as data flow, etc...
React is a much better solution for writing reactive programs than Rx.
Are you speaking about React.js? Erik's point about reactive being a buzzword and distorted is proven by that because it's not the same thing at all. React.js is cool and all that but it has nothing to do with guaranteed composition and dealing with side-effects and monadic operators. That's like saying Angular.js is better than list operations and functions that operate on lists.
[1] http://dl.acm.org/citation.cfm?id=37508&CFID=356264587&CFTOK... [2] http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-R...
My point was that if you were building user interfaces, react would get you much further than Rx, which is a hammer looking for a problem where it is actually better.
If you want the ability to deal with side effects, I'm working that in my own work while avoiding burying everything in data flow:
http://research.microsoft.com/apps/pubs/default.aspx?id=2112...
Ironically, I started this work while working for Martin with the goal of making the scala compiler more "reactive" (the ability to responsively react to change); I actually succeeded while making minimal changes to scalac. Like actors or FRP, Rx can't be applied to scalac either, which desperately needs to be reactive to fit into the IDE. Rx's idea of reactivity revolves around event streams, virtual, managed time and frameworks like react attack the problem of change directly. Erik is right about useless definitions, but one based around change really works.
I think the description of events as just another collection, albeit a time based collection flow was nicer ( Observable === Collection + Time ), and also that this collection must have the ability to tell you that it has terminated.
If you want statically typed functional programming, but also functional programming that stands a chance of adoption in many environments, then Scala is about your only choice.
And Rust? Does anyone dare basing production stuff on Rust like now? With breaking changes all the time?
This is the first time I've heard of reactive and read HN everyday.
> Java's enumerable( Iterable?) is broken
anyone know what how its broken?
You're not paying enough attention. There were tons of articles on reactive on the front page, at least 1-2 per week, for the last year or so. See also: "functional reactive programming". See also: "React.js". See also: "Om".
https://www.google.com/search?q=site%3Anews.ycombinator.com+...
Take a look on GitHub at all the projects with reactive in the name or descriptions; it's become a buzzword that people use to describe various things that don't have any relationship to the original.
Iterable is broken, as Erik said, because it has a hasNext() method, and a next() method, which is a problem because hasNext() causes side-effects to be evaluated.
Imagine if the implementation of the iterator has to execute a long running function, or wait for the next value in an observable to be pushed, like an event. So, if call hasNext() twice, then it may cause a side effect so that next() actually has a different result than it would without the side effect. Thus, it is not mathematically, a function, and can't really be used to build monads.
So, unfortunately, due to backwards compatibility, Java classes that implement Iterable/Iterator can't automatically get the benefits of list monads etc..., by simply defining a library of operators that take Iterable/Iterators, like C# could with IEnumerable/IEnumerator.
In C#, IE only has side effects in the MoveNext() method, which returns true or false if there's another element, and the effect is explicit in the name, which helps. However, more importantly, if you get the value from Current, there is no side effect, and you can do that as many times as you like.
With Iterable, if you call next() it moves to the next element. So, it is impossible to get the same value twice with a guarantee of no side effects, and impossible to guarantee mathematically sound composition of Iterables. The interface is broken, and changing it would break nearly every Java program.
The speaker mocks his audience of Microsoft engineers (like not in fun, he is really saying their culture sucks).
He is dismissive of "reactive programming" as nonsense from architecture astronauts, without giving a remotely fair description of what it actually is. His straw-man of "var x = 10; println(x); x = 42; println(x)" is not reactive, because "x" is not a function of other mutable state in the model. I've never heard of reactive programming until five minutes ago, but I can tell this just by reading the Wikipedia excerpts from his own slides (and a little follow-up reading confirms this).
I'm all for rabble-rousing, but if you're throwing punches you should know what you're talking about.
At 17:30 he makes a joke about a Steve Jobs function that says "iPhone", "iPad", "iCloud", and then "terminates naturally". Holy bad taste.
EDIT: others have pointed out that with extra context this is more obviously a self-deprecating and good-natured shtick.
And I don't think he's being dismissive of reactive programming, he's dismissive of that definition, which is too broad. He was one of the main proponents of Reactive programming inside Microsoft.
"In Redmond, like a lot of people talk about code but don't write it, so I think Microsoft can only come back if, you know, they start writing more code."
> And I don't think he's being dismissive of reactive programming, he's dismissive of that definition, which is too broad.
The definition seems pretty reasonable to me. It immediately resonated with me; it described a principle I had been trying to achieve in some code I was writing recently. The idea of having changes automatically propagate through your model as a data-flow network is a pretty noticeable contrast to other ways of programming.
> He was one of the main proponents of Reactive programming inside Microsoft.
Yeah I guess I don't have this context. But without it, the talk just rubbed me wrong.
It's like you've never worked at a tech shop with a sales team. Also keep in mind that it wasn't meant to be taken seriously or literally. If his audience wasn't terribly offended by it, why are you?
My reading and feelings concerning the Manifesto mirror Erik's so I enjoyed listening to his mockery of it. I didn't really notice the MS bashing.
The issue with the Reactive Manifesto is that it is largely waffle. There are styles of programming that account for time, such as functional reactive programming. They tend to have formal definitions, or at least concrete implementations, so one can make objective statements about them. The Manifesto on the other hand is vague and full of platitudes. It's great for the Pointy-haired Boss, but there is little of substance than one can discuss in any objective manner.
[Update] I can understand how it's off-putting to listen to the presentation without this background.
> He is dismissive of "reactive programming" as nonsense from architecture astronauts, without giving a remotely fair description of what it actually is.
That's because his presentation is from React 2014, so he can fairly assume that the audience has some idea of what he's talking about, because "reactive" is the freaking topic of this audience - http://reactconf.com/
> if you're throwing punches you should know what you're talking about
Well, he kind of does. He's one of the architects of Reactive Extensions (Rx) and now he's contributing to RxJava. I don't agree with many of his opinions, for example I think they made design mistakes while architecting Rx, but he's OK in the knowledge department ;-)
> At 17:30 he makes a joke about a Steve Jobs function that says "iPhone", "iPad", "iCloud", and then "terminates naturally". Holy bad taste.
He also makes the same joke about himself - was trying to explain how streams are terminated, either naturally or by error (the joke on himself was that he was fired, i.e. onError).
If he has that much background I would expect him to know better than to give such a poor example when he's trying to make fun of a definition as "dilbert-esque."
Tongue-in-cheek is a fun style and I totally get that, but if you're taking unfair hyperbolic pot-shots I think it should be at your own expense, not other people's. But that's just me.
Curious to hear more.
I think ISubject<I,O> has the interface for a pipe, with IObservable being a Producer equivalent, and IObserver being a Consumer equivalent. So there may be some cunning that could be done with your own implementations [to create an Effect]. (This is just wild brainstorming without looking at the code, so take all of this with a large pinch of salt).
Working within the Rx system, there'd be two ways of doing this:
The Observable has a mechanism for slowing down, but obviously it can't get instruction from observers - so it would have to make a judgment on what is 'too much'; a slow down would affect all observers. Definitely (well probably) not what you want.
The other way is to use the various buffering functions in Rx on the subscription, or roll your own function that has some intelligence. That localises the 'backing up' on a per observer basis, but doesn't slow down the observable itself.
To 'Stop' you could switch your subscription to Observable.Never<T> until you're ready to receive messages. Obviously you'll miss messages generated on the other stream whilst you were not listening.
I think it is relevant, what allows you to ignore the behavior of the system in favor of a component?
Dean wampler in a recent talk spoke of "Reactive Streams" which add this capability, but I have not seen it yet.
I have not seen anyone do this in Rx yet. Also ideally the back pressure is out of band to avoid starvation of that channel
Or do you think it should be an inherent feature of the stream itself? Isn't the point of IObservable to be the dual of IEnumerable, so calling Select on IEnumerable is a projection from A -> B, calling Select on IObservable is also a projection from A -> B, I think having an additional effect breaks that duality. So an Observable that has a side-channel or out-of-band stream wouldn't be an Observable, it'd be another category (which is why I suggested Pipes).
I totally understand why you'd want something like that, I'm just not sure it's fair to suggest Rx has design mistakes because of this though.
It certainly seems possible to use Rx primitives to build more functionality, yes.
Though everyone will have to build this themselves (if, of course, they even need it), and they'll probably all build it differently, where other lib's have this concept baked in (i.e. Pipes).
IMHO, libraries shouldn't try to do everything anyway, and to that end, Rx does it's thing and does it well (unix philosophy), as it's based on these mathematical abstractions.
Ironically, I think you completely missed his point, probably because of an unfamiliarity with his background and history. He's not dismissive of reactive programming at all, quite the opposite! For example, he gave a course on Coursera about Functional Reactive Programming with Martin Odersky and Thomas Kuhn[0].
So my take on it is that he's just making fun of himself and his friends in a typical blunt Dutch fashion. Which of course doesn't work if you don't know the context.
http://en.wikipedia.org/wiki/Erik_Meijer_(computer_scientist...
[1] http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-B...
Yeah I wouldn't know why he would want to have to do anything with MSDN