If I understand correctly that's sort of exactly what this is. The geometry of the diffraction gratings encodes a forward propagation model trained as classifier of the number "2".

I don't quite understand the mathematics of how it was trained, but they were able to discretize the geometry of those layers somehow into little 0.4mm pixels of "trainable diffractive neurons" and they simulated light transmission through the layers to compute a loss function.

I'm really surprised that this was computationally feasible. Simulation of light through the gratings must have been cheap enough as a function evaluation to train the network.