Look up autoregressive model. There are already forecasting models that do something like this. I don't know if any use attention, it seems like a pretty obvious thing to have done. Overall this isn't a new idea.

Deep learning generally brings a lot less to tabular data because the underlying thing being modelled is much simpler compared to language models that effectively are modeling the human mind + culture so there's a lot more to fit the model to.