Ask HN: Embedded NN Libraries

Dear HN,

Do you have recommendations for libraries for running NNs on embedded devices?

- must compile using a C or C++ compiler

- must be efficient (runtime speed on embedded obv matters)

- must support conv, lstm

- easy export from PyTorch

- bonus points for rfft / torch.stft support

Any advice on how to choose/evaluate libraries would also be welcome. So far I've had a look at tinygrad, ONNX, and ExecuTorch. It seems ONNX is the most supported, stable option, but I'd very much welcome advice from people who have actual experience running NNs on embedded devices.