Hacker News Reader

Comment by tpoacher | Hacker News Reader

Depends on your definition of "controlled". The ambiguity behind the term is doing a lot of heavy lifting here.

Boeing's MCAS system was also "just software". Which in principle can be "controlled", i.e. changed, updated, audited or whatnot.

But then people died precisely because pilots found themselves unable to override or "control" the systems precisely when it mattered.

View on HN

> Boeing's MCAS system was also "just software". Which in principle can be "controlled", i.e. changed, updated, audited or whatnot.

> But then people died precisely because pilots found themselves unable to override or "control" the systems precisely when it mattered.

Wasn't it designed to do so? Also works as a counter example, that sandboxes can limit AI if just operators want to do so.

based on my limited memory and understanding of the incident, when MCAS kicked in, it would make the plane controls more "rigid" in the intended "corrective" direction, as a tactile way of "encouraging" the pilots to comply towards the correction. If a pilot really wanted to override, effectively they could do so by using brute strength to overcome the imposed rigidity, and force their preferred direction.

But then the bug made MCAS kick in on false positives repeatedly and in a prolonged manner, causing the pilots to tire out and no longer be able to overpower the AI to take control of the plane controls.

(lay understanding, and oversimplification of a complex issue, possibly wrong, take with a huge grain of salt)

Reply on news.ycombinator.com