Development of an onboard application in Clojure for the Boeing 737 Max [video] | Hacker News Reader

Development of an onboard application in Clojure for the Boeing 737 Max [video]

youtube.com

87 pointsraspasov37 comments

Isn't it amazing that they built a real-time diagnostic system for Boeing plane in Clojure with a team that haven't previously used Clojure? They basically were learning on the fly, coming from OO background, about map/reduce, immutability etc. Kudos for being really open when choosing the best tool for the job after initial research (among the reasons behind picking Clojure the guy lists functional reactive style and being able to easily parse DSLs). Code brevity was one of the positive side effects, which made code reviews much quicker. Impressive

> They basically were learning on the fly

I chuckled on this. Pun intended? :)

I'm getting kind of sleepy here at 2 AM and my brain misarranged the headline words into something like "Development of an application in Clojure onboard the Boeing 737 Max". So it seems like an easy mistake to make!

One of the things that is impressive, is the amount of "real life" applications clojure is apparently capable of, compared to other relatively new programming languages. I always was expecting that avionics software would be created using low level languages or something like ADA for example. Really interesting talk that opens a very wide area for clojure!

Bear in mind that avionics software as a whole is a broad category that includes everything from fly-by-wire flight controls to the onboard entertainment system. Each system is classified under an appropriate Software Level as specified in DO-178C:

https://en.wikipedia.org/wiki/DO-178C#Software_level

The OMF system described in the video is Level E, meaning it's failure would have no safety impact on the aircraft. This gives us greater latitude in choice of software solutions. The higher levels, however, have strict requirements regarding latency and memory management that would automatically preclude the use of Clojure or any language with garbage collection.

So, as always, it's selecting the right tool for the job.

Cheers, interesting info

From my experience most of the work I have done in the last 5 years is in C for Level A - C (and D but that is changing). Ada is only used on legacy projects that started out that way. We have one level D project that uses embedded Linux and a technicolor assortment of languages. E projects generally are a free for all with what is used.

SCADE is also used in levels A-C:

> At present, more than 98 system types have been certified using SCADE DO-178B/C certified tools, for more than 53 aircraft programs, by multiple certification authorities, worldwide, such as the FAA, EASA, Transport Canada, and ANAC, for Levels C to A.

http://www.esterel-technologies.com/industries/do-178b-and-d...

Yeah Simulink produces C code too. Doesn't SCADE produce c and Ada code to be compiled for the target? I mean most Control laws (application code) is now modeled and produces C code. I would not even call SCADE and Simulink a language. Custom OS's that run that SCADE code is written in C.

Right, SCADE compiles down to C, but the development and verification are done in SCADE.

I'm one of the leads who worked with Donevan on this project. Feel free to ask me any questions and I'll answer as best I can.

I'm interested in the underlying platform and how the software is being shipped? Do you need to go through a specific process in order to install a new version or a bug fix?

Anything in a regulated industry such as aerospace tends towards the process heavy. Our requirements get tracked in our QMS system that ensures we're AS9100-compliant. There's a final test-for-credit step after which the software part should be considered hermetically sealed. Any software change requires a new test-for-credit round that likely takes an extensive amount of time.

So, in short, developing for aerospace is not agile. :-) It's the Niagara Falls of waterfall methods.

I work at BAE and we do quite a bit of software for Boeing. DO178 C could allow you to do Agile if the PSAC was set up from the beginning for that. The problem would be the cost points and contracts and customer buy in that would be needed on all sides. But alas for now it is the mother of all Waterfall methods.

We developed OMF using agile scrum, not waterfall, but we ultimately were running our agile process within a waterfall

On the video it sounded like it was 16,000 lines of code. Was that the amount or did I mishear it? That was claimed as one of the largest Clojure codebases, which seems unlikely.

That's what I heard in person

That's correct. As for size relative to other projects, we came to that conclusion by combing large GitHub clojure projects and comparing non-blank lines. For instance, LightTable comes in at around 12,000 lines. Clojure is a darn terse language.

Of course, there may be private repositories which are larger (thus the weasel words "one of"). Corrections welcome. :-)

From memory, CircleCI have around 100k LOC. Nonetheless, it's very cool getting Clojure on a plane.

Thanks. Found a reference to the CircleCI code base:

http://www.ccs.neu.edu/home/samth/typed-clojure-draft.pdf

I wonder how their move away from clojure.typed has impacted code size.

I was made aware of a private Clojure project that is comprised of over 30k lines. So we aren't the largest, but probably the first to be on a commercial aircraft.

But as Tim pointed out auditing of open source projects had most far smaller than 16k. Lamina a very feature rich library I think was on the order of 6k.

Perhaps I should have gone with "Clojure on planes" as my title.

Or "States On A Plane" with a photo of Samuel L. Jackson. :-)

I'd consider 16,000 on the small side. For comparison, I've been working on a Clojure roguelike as a hobby project and it's in the ~18,000 LOC range.

Aaron, how are you counting the SLOC? I am not doing a simple `wc -l` on all clj files in our project. I am looking for lines that include either closing parenthesis or closing brackets. It isn't exact. A better count would be to count the # of forms.

I get 17k in your project with find . -name ".clj" | xargs wc -l but 8k if I run 'find . -name ".clj" | xargs egrep "[)\]]" 2> /dev/null | wc -l'.

If I just run `wc -l` on our project I get 35k lines.

8k SLOC is still a pretty impressive size project considering the audit of several open source projects that I did were mostly below 8k SLOC.

I used sloccount-clojure to count lines, but before running it I made one change so that it would include .cljc files.

Then I summed both robinson and zaffre (0.4.0 branch) seeing as they both used to be part of the same project and have only recently been separated.

I develop Cursive - it's closed source, but it's around 31k LOC Clojure, 23k Java and 5k Kotlin, all as counted by cloc. I'm not sure what's up with your find command, but that reports 5k LOC in Cursive, it's definitely more than that. Straight wc -l reports 74k.

I'm pretty sure my code has more bugs than yours, though :-)

Anyway, very interesting presentation, thanks - it's great to see a project like this in a space I wouldn't have expected it. I'm very surprised (and pleased) to know that Clojure is on Boeing's approved language list - I'd love to know how that came about.

BTW this is the largest Clojure project I've heard about: https://twitter.com/hlship/status/631935954039652352. That's a pure Clojure project.

You are right. It's not the largest Clojure codebase(Language Breakdown of Storm [1]). There may be reasons why they think their Clojure codebase is largest.

[1] https://www.openhub.net/p/apache-storm/analyses/latest/langu...

I re-ran SLOC using the sloccount-clojure tool, we sit at about 34k lines using that tool. Instead of my jenky find/egrep/wc.

It's not widely known but Python supports a certain amount of functional-programming idioms. It also supports event-driven data flow. Was it considered at the beginning of the project? (was it even one of the languages Boeing supports?) Why would or why would it not be a good choice for this health monitoring application?

The big difference between Python and modern functional languages is that Python defaults to mutability. Clojure is immutable by default, and that makes it much easier to reason about code in isolation. When you're working on a large project, it's really valuable to be able to make a change in one place and know that you're not affecting something in another.

When things are passed around by reference, it becomes very difficult to guarantee the scope of any change. Even when using immutable data structure libraries in a mutable by default language doesn't really give you many guarantees. As soon as somebody puts a mutable object into your immutable data structure all bets are off. So, you really want a language that defaults to immutability and has different semantics for working with mutable data.

Does Boeing have a policy on using and defining acronyms?

Interesting presentation, although I noticed Donevan used a few acronyms that some people may not be familiar with.

Not sure if there is a policy. Are there acronyms in particular, I tried to limit it to the couple applications we interface with. Which truth be told I don't know off the top of my head what they stand for. More importantly is what the application does. I worked at Boeing for 6 years and people talk in acronyms most the time not knowing what it stands for, just what the system does.

Why Clojure and not Scala?

I'd pick Clojure between the two because simplicity.

In the video he mentions that libraries went EOL during development, given the the predicted lifespans of modern airliners how do you plan on been able to build the software a decade from now?

If needed we or Boeing would take over the lamina library, or migrate to an alternative like core.async or manifold.

Reply on news.ycombinator.com