Hacker News Reader

Most safety critical code in the world, both civilian and military, is currently written in C/C++. Ada/SPARK is also used but it 's below 50%.

These languages have the best practical static code analyzer, verification and proofing tools money can by. I'm personally using Astree https://www.absint.com/astree/index.htm Automatic docking software for the ATV that delivers supply to ISS is written using C code and verified with Astree.

People get stuck into the language, but if it has some features that make it work, you can use it with verifiers. Then you write tests and simulators and tests ...

JavaScript and Chromium seems more suspect to me, but if the code is well tested and has limited run time, I think it can be good to go. The state is stored and prosessed in those radiation hardened RAD750 processors. Chromium/JavaScript runtime image can restart every few seconds if needed.

The UI and controls seem like unnecessarily flashy bullshit to me. Usually if you diverge from KISS principle bad things happen. But before I make any judgements I would see how they made those decisions.

How much is Astrée? I get nervous when you can’t see a price on the website.

In this case, C++ code is written in a different style than regular applications: exceptions are not allowed, allocations are not allowed past a certain point in program initialization, raw pointers are frowned upon, etc. This is common in this type of software and when you pair it with the right development culture it really makes a difference in the robustness of the result.

Any idea if there's a database layer (eg SQLite, PG, etc) on the systems?

On the ground side, yes, there are many databases for different purposes. But not on the actual flight software.

Very curious, how is state persisted where non-volatile storage is needed, and how is that persistence structured?

State is, as far as the system is concerned, is what values the multitude of sensors are reporting at any given moment.

It is never persisted outside of RAM. The flight computers are never powered off during a mission. You rely on two flight computers to always be functioning, that’s why the system is one-fault redundant.

You can think as each string being “functional“: given the same set of inputs you expect the same set of outputs.

What happens if some anomaly happens where the shuttle loses power for a second?

Are you asking about what happens if all the power goes out or are you asking how they get state back? If everything reboots it’s fine. On restart the computers will look at all the sensors add project the state of now. There is no reason to keep the state of the past is there? Why does it matter where the craft was 5 minutes before? What matters is what is what is in front of them and the future right? If you tell the computer I want to fly home it looks at where you are and tells you how to get there. It does not need to remember the past parts of the trip does it?

I'm reminded of the quote from Linus:

"Real men don’t use backups, they post their stuff on a public ftp server and let the rest of the world make copies."

Real programmers don't use persistent storage, they just rely on physical reality in general.

It'd have to know that it's destination was earth, or does it just sit idle after a power on and wait for the command? What happens if the astronauts are asleep? I feel like it needs some persisted data.

None if possible. In the safety critical parts you want to use software that

- don't do dynamic any memory allocation, has bounded memory usage, never runs out of memory.

- has bounded execution times

- has low interrupt latency

- can do checksum on data and internal state an option. (if need to load into memory)

- is tested using the most rigorous standards, 100% verification and code coverage.

If you write great relatively general but customizable database system like this with good indexing, you need only 5-10 customers and you are set. Safety-critical systems are growing in size and there is real need for software that can get easily certified.

SQLite could probably get certified as a part a system at least as read only database. It's relatively solid code. But it will cost big money and I don't think anyone has done it.

Is C++ code formally verified, e.g. with something like Coq?

The fact that it's implemented in JavaScript and Chromium just tells you the fancy display is entirely superfluous to mission success.

Interesting way to see it. For anyone wondering how unreliable it is, think instead that the trade off was well planned.

Is there a backup should that fancy display fail? If not, then I'd hardly call it "superfluous to mission success".

Apparently the physical controls below the screens can perform all of the mission critical functions in the event of a display failure.

If only browsers had a similar fallback.

Once you decide that absolute reliability is unattainable, you can instead aim for sufficient redundancy. It’s not perfect, but with enough safe guards in place it may very well be good enough.

Software is not always like hardware though: the same software failure can cascade across redundant instances rapidly just as easily as it affects the primary.

The same hardware failure can cascade too; redundancy alone is not a panacea, you need some safeguards. Ideally, nothing should fully trust anything else.

Maybe cascade was the wrong word. I meant that if you pop in a hot spare and it's running the same software, there's a good chance it will encounter the same issue, sometimes immediately.

In some cases redundant software is developed independently by different teams to the same spec; I think I read about that with the Space Shuttle.

How correlated are your layers of swiss cheese...?

I was just reading that they design the rocket so that one engine could explode without affecting the others (hopefully) and they test the engines by dropping a nut or bolt or something in, which is not something rocket engines normally can tolerate.

I'm reasonably sure that quite a bit of the complexity is more or less directly caused by the amount of features it provides. I agree with you in that Chromium probably brings in a lot of features they don't need, but I would also say that reimplementing the features they do need is also a huge liability and would incur quite a bit of complexity too... but it'd make it their insular system with no other resources flowing into improving it.

I don't have the data to make that call, but I would think someone at their end evaluated this quite thoroughly and determined the risks of rolling their own to exceed the risks of using Chromium.

(Also they mentioned on the stream that they do in fact have a few hardware buttons for important things. Can't cite this, sorry...)

Why not Javascript? I would argue that a good usable UI in a spacecraft is a good thing, and Javascript is a great language for building UIs

> Javascript is a great language for building UIs

I disagree. It is easy to build something presentable with HTML and browsers, and there have been many JS frameworks that lower the barrier. However, that is not because JS is a good language for it; it is because JS is the only language available in that environment.

In fact, there are at least two other languages that transpile to JS, because you need JS to be able to run in a browser. But JS the language is not good enough, so you want two write your code in something else.

Building a UI in a completely different language, without a browser, is definitely harder, but you might have several advantages in the other, non-UI parts.

You haven’t made any arguments about why Javascript isn’t good for building UIs there. Here’s mine for why it is:

- first class functions (callback style is a good fit for UI programming)

- async/await and single threaded - many components of UIs are async by nature, Javascript supports you well here while keeping things simple with its single threaded model

There's... a large number of languages with both of those features.

Could you name few better than js?

In the context of writing reliable software for a spacecraft? Pretty much anything with a reasonable type system, for a start. Maybe C#, or some reasonable subset of C++20, or Rust. Maybe Reason or OCaml.

I mean very clearly the good people at SpaceX have heard of these languages and decided against it. I think JS and Chromium are the best UI kit I can think of in terms of man hours to get X done. If you're careful the only thing I've ever had issues with is latency, but in space if you're in a situation where missing a frame or two is important you're already beyond fucked.

From my experience, Qt/QML is way better for things like that. I mean, the UI does not need to be responsive, does it?

Maybe they couldn't afford the Qt license.

Many people are using TypeScript these days, it works very well.

I like C# and Rust too. I've built UI's in C++ and MFC/OpenGL/DirectX. It remains my fairly strong opinion that JavaScript is well suited to UI development.

Sharing the UI thread with logic is something to be mitigated when building UIs, it's not a feature.

In theory, maybe. In practice, your UI thread very rarely has to do the kind of heavy lifting that would block it so badly you actually have problems.

I have encountered this -- for example parsing very large incoming JSON payloads -- but very rarely. And if you really need to mitigate it, there are webworkers.

Javascript is a bad language period. For anything. It's used for UIs for historical reasons, not because it's good.

Javascript has also evolved quite a bit and there is also TypeScript that improves on that. The language is not perfect by any means, but it gets better.

I would expect the UI in a space ship to be soft realtime. Chromium and JS are anything but.

The only thing that's self-evident to me is that Javascript is a popular language for building UIs.

I don't see why.

Chromium is being used in the UI. Not on critical flight software.

> maybe even manual docking

It isn't done manually

> C++ and linux do not strike me as mission critical piece of software / language either

A lot of flight software is done on C++, as you even mentioned. In the end, your best reliability partner is not using a language that's only know by fewer people.

What would be a good "mission critical piece of software"? QNX (with no memory protection?). Linux is fine. I suppose they're not firing Ubuntu out of the box and running that with no customizations.

Separation of concerns and redundancy usually beat correctness on the long term (which doesn't mean they didn't test the correctness of the system).

  QNX (with no memory protection?)

Its probably a misunderstanding, but QNX has a very strong memory protection model that extends to its driver stack. Given that its a true realtime/microkernel, which has its own set of problems, but memory protection isn't one of them. For a number of cases QNX is going to be a large step up from linux.

http://www.qnx.com/developers/docs/6.5.0/topic/com.qnx.doc.i...

Unlikely that they are using javascript in the actual flight controller. Also under the screen they have a small amount of manual buttons that is likely to be seperate from the touchscreen interface logic.

>javascript in the actual flight controller.

zero chances that it's the case. I meant that in aerospace applications UI is mission critical too.

> I meant that in aerospace applications UI is mission critical too.

It is, but usability of the UI is mission critical as well. The humans in the capsule need to be able to quickly and effectively consume information in a way that doesn't make important facts hard to get at, and then quickly and effectively perform commands to address issues.

I don't think a touch GUI is a bad choice in this regard, and if I'm building a touch GUI I'd rather use some existing framework. My personal choice would've been something like Gtk or Qt though :)

It should be most likely be WebGL and some glue js code. I don't think they're running everything in just js for the view code.

In the case of SpaceX, it seems that they use embedded Linux and C++ with x86 processors, same as standard PC processors. A reason for using embedded linux is that, it allows using standard C++ or even scripting languages for controlling the hardware from the user-space by just reading and writing files. Linux device drivers (aka kernel modules) maps the hardware to special files on /sys or /dev. For instance, it makes possible to control a GPIO (General Purpose IO) which the device driver maps to sysfs special file system, by just writing 1 or 0 to the file a like /sys/class/gpio/gpio4/value which would enable GPIO 4 and turn on a LED attached to it. Another practical example about this feature is that, on Linux, it is possible to turn on or turn off the keyboard capslock LED by writing to some /sys/class/leds file such as "$ echo 1 > /sys/class/leds/input7\:\:capslock/brightness" which turns on the Capslock LED. By writing 0 to it, the LED is turned off.

This feature of Unix and Linux allows controlling the hardware with any programming language capable of reading and writing files, including Python and standard C++. I guess that they may be using standard PC hardware with industrial IO card. They may also use a single-board computer or custom board with x86 low-power variants processors built for embedded applications as SOC system-on-chip. One example of low power x86-SOC based processor is: https://www.cnx-software.com/2015/04/09/vortex86dx3-is-a-new...