Am I misreading this code? It looks like you don't have precomputed table representations and search, instead you scan, embed, and compare on each run?
https://github.com/Dataherald/dataherald/blob/main/services/...
https://github.com/Dataherald/dataherald/blob/main/services/...
They are then retrieved and injected based on relevance to the query.