Other than stating there was one, they didn’t show a benefit of this over something like a Wide and Deep model, DCNv2 model, or even a vanilla NN. Transformers make sense if you need to use something N items ago as context (as in text) where N is large. But in their example, any model which takes the last ~5 or so interactions should be able to quickly understand contextual user preferences.

A transformer may also be larger than their baseline, but you still need to justify how those parameters are allocated.