Ask HN: Tranformer Models in Fintech?

Any more details on uses here? I read a post from Stripe on this. Any other companies or any of you using these successfully?

FYI... has anyone here considered how to represent background information (background context) at the beginning of a transformer sequence? Specialized tokens that refer to user attributes (a token preamble, if you will...).