It looks like a Transformer encoder post-trained on classification and regression tasks. The encoder-only model is less noticed in recent years, but this product finds a nice application for it.