essexedwards

I haven't read the code, but I would expect simpl.cpp to have something to do with SiMPL https://github.com/dohyun-cse/mfem/blob/simpl2/miniapps/simp...

According to the paper (https://arxiv.org/abs/2411.19421) the code for SiMPL is implemented in MFEM and is available here: https://github.com/dohyun-cse/mfem/tree/simpl2.

Yes, there is hope for a high-level heuristic understanding. Here's my attempt to explain in more familiar terms.

They train a new neural network from scratch for each problem. The network is trained only on the data about that problem. The loss function tries to make sure it can map the inputs to the outputs. It also tries to keep the network weights small so that the neural network is as simple as possible. Hopefully a simple function that maps the sample inputs to the sample outputs will also do the right thing on the test input. It works 20~30% of the time.

> No one actually knows what their thresholds are (including library authors)

If low-level numerical libraries provided documentation for their accuracy guarantees, it would make it easier to develop software on top of those libraries. I think numerical libraries should be doing this, when possible. It's already common for special-function (e.g. sin, cos, sqrt) libraries to specify their accuracy in ULPs. It's less common for linear algebra libraries to specify their accuracy, but it's still quite doable for BLAS-like operations.

I've also been surprised many times by issues in numerical libraries. In addition to matrices with simple entries, I've found plenty of bugs just testing small matrices, with dimensions in {0,1,2,3,4}. Many libraries/routines fall over when the matrix is small, especially when one dimension is 0 or 1.

Presently, I am working on cuSPARSE and I'm very keen to improve its testing and correctness. I would appreciate anything more you can share about bugs you've seen in cuSPARSE. Feel free to email me, eedwards at nvidia.com