Hacker News Reader

Comment by sorenjan | Hacker News Reader

The last time I was reminded of the bitter lesson was when I read about Guidance & Control Networks, after seeing them used in an autonomous drone that beat the best human FPV pilots [0]. Basically it's using a small MLP (Multi Layer Perceptron) on the order of 200 parameters, and using the drone's state as input and controlling the motors directly with the output. We have all kinds of fancy control theory like MCP (Model Predictive Control), but it turns out that the best solution might be to train a relatively tiny NN using a mix of simulation and collected sensor data instead. It's not better because of huge computation resources, it's actually more computationally efficient than some classic alternatives, but it is more general.

[0] https://www.tudelft.nl/en/2025/lr/autonomous-drone-from-tu-d...

https://www.nature.com/articles/s41586-023-06419-4

https://arxiv.org/abs/2305.13078

https://arxiv.org/abs/2305.02705

View on HN

But I have also seen people trying to use deep networks to identify rotating machinery faults, like bearings, from raw accelerometer data collected a high frequencies like 40 kHz. Whereas the spectrum of the data from running FFT on the signal contains fault information much more obviously and clearly.

Throwing a deep network on a problem without some physical insight into the problem has also its disadvantages it seems.

Yeah, we're shouting into the wind here. I have had people tell me directly that my ideas from old school state estimation were irrelevant in the era of deep learning. They may produce (in this case) worse results, but the long game I'm assured is superior.

The specific scenario was estimating the orientation of a stationary semi trailer. An objectively measurable number and it was consistently off by 30 deg, yet I was the jerk for suggesting we move from end to end DL to trad Bar Shalom techniques.

That scene isn't for me anymore.

> That scene isn't for me anymore.

They will learn. At least when the competition beats their solution with a hybrid approach they can't begin to understand.

I’m working on this sort of thing right now in a SaaS product that previously didn’t have support for vibration data. One competitor is all ML-d up to the hilt but customers don’t like the black box and keep finding it gives false positives with no explanation. I think one problem is those buying not understanding problem; they just want to plug in a sensor and insights to happen, but without information about the machine that’s never going to be able to provide useful insights

>It's not better because of huge computation resources, it's actually more computationally efficient than some classic alternatives

It's similar with options pricing. The most sophisticated models like multivariate stochastic volatility are computationally expensive to approximate with classical approaches (and have no closed form solution), so just training a small NN on the output of a vast number of simulations of the underlying processes ends up producing a more efficient model than traditional approaches. Same with stuff like trinomial trees.

Interesting. Are these models the SOTA in the options trading industry (e.g. MM) nowadays?

This is really interesting. I think force fields in molecular dynamics have underwent a similar NN revolution. You train your NN on the output of expensive calculations to replace the expensive function with a cheap one. Could you train a small language model with a big one?

> Could you train a small language model with a big one?

Yes, it's called distillation.

Can you elaborate on this? I am curious what processes are being simulated to feed the network.

There are also extremely misleading research articles out there promising good results with deep networks in the area of anomaly detection, without adequate comparison with more classical techniques.

This well-known critical paper shows examples of AI articles/techniques applied to popular datasets with good-looking results. But, it also demonstrates that, literally, a single line of MATLAB code can outperform some of these techniques: https://arxiv.org/pdf/2009.13807

> an autonomous drone that beat the best human FPV pilots

Doesn’t any such claim come with huge caveats — pre specified track/course, no random objects flying between, etc…? ie. train & test distributions are ensured same by ensuring test time can never be more complicated than training data.

Also presumably better sensing than raw visual input.

> "MCP (Model Predictive Control)"

^ that's MPC. (MCP = Model Context Protocol)

Reply on news.ycombinator.com