Hacker News Reader

Comment by mynegation | Hacker News Reader

At least one of the planes in this situation was equipped with the system that lands the plane automatically in almost zero visibility. It’s not a stretch of imagination to see how that can extend to the system that tracks planes on approach to the airport and steering on the ground and extrapolates their trajectories at least for the typical braking/abort time forward. These are not general aviation, these are planes carrying tons of cargo and hundreds of people.

View on HN

What you’re talking about is called a collision avoidance system and it exists and both of these planes have it.

It doesn’t alert you to planes on the ground because it would never stop going off. How is the system supposed to know that the pilot approaching the threshold as you land is going to stop? It can’t so it would issue an immediate correction alert and you have to go around.

Sounds like a job for... A.I.?

No, no and no. Safety critical systems must be validated to be safe in the operational domain and for that they must be deterministic. AI is anything but that.

AI is entirely deterministic. ChatGPT and StableDiffusion and friends are fed endless amounts of random seeds along with every input to keep them from just saying always the same thing.

The output for a given input (absent random seeds) may be constant, but is the output deterministically correct?

What does that question even mean?

Roughly, they are asking whether you can take an AI system and effectively reduce it to an analytic function and tell, without actually running the AI, what the output is going to be with a particular input.

A generative AI may well be deterministic and generate repeatable output for a given inpiut, but that doesn't mean the output is correct for every input. It may merely generate the wrong answer consistently.

"Deterministic" only insofar as it's repeatable, but their behaviour is not predictable. If the behaviour is not predictable, how can it be validated as being correct?

If for a given set of inputs there is a deterministic output, then the overall behaviour to a series of inputs is just as deterministic and predictable.

I'm not sure what you mean with "is not predictable" when you also admit that it's repeatable.

I think the point is that reality is essentially a chaotic system. That is, yes, you can repeat a failure from an input once you've seen it, but the search space is too big to enumerate beforehand.

That entirely depends on what exactly you implement here, it's entirely possible to implement an AI with continuous & linear properties, meaning that you can extrapolate it's behaviour between a set of inputs with decent accuracy and it won't suddenly change it's behaviour between continuous & linear inputs.

But AI isn't different than existing software systems either. Both will take an input from reality and take actions upon it.

Predictable as in, can say in advance what it will do.

Repeatable means you get the same output a second time, given the same input. Predictable means you can say in advance what it will do, and can then check the output against your prediction.

If you can't predict the outcome, you can't validate the process, and can't guarantee its performance.

That much is clear but any remotely continuous AI model is very predictable.

That's useless in this case. You need to be able to prove that it will work with all inputs, and there are too many combinations of inputs to exhaustively enumerate.

That would be true of any software system we put into an airplane, yet we have deployed software into airplanes.

If your AI has linear / continuous output, testing it should be no different than any other software.

There is no way to determine that a non-trivial neural network won't drastically diverge in output due to small changes in input (eg one pixel attacks on image classifiers). This is true for all current models I know of.

Almost all neural network implementations have continuous outputs (ie the nodes in the output layer produce a value between 0 and 1). That doesn't change the above issue at all.

This is much less of an issue with traditional methods

I was thinking A.I. could be useful as an assistant for ATC. But yeah, it would inevitably be over-relied-upon and over-trusted.

ATC controller doing a bad job and almost killing a bunch of people is anything but great

skip the buzzword, why couldn’t a computer program of any kind helped avoid this

Reply on news.ycombinator.com