I'd love to see SSMs replace transformers but adapting them to non-causal, 2D+ inputs doesn't seem that straightforward.

Is there a non-autoregressive future?