In a case like this it still seems more appropriate to encode that data in tabular form and use a tabular foundation model