Sorry if this is a different sensd of that same word…

I can understand almost everything in the explanations of how ChatGPT, neural networks and fine-tuning works

But one thing isn’t being explained…

How do you embed the words as vectors????

Do you just make up your own scheme?

I understand about vector databases being used for search, to retrieve snippets and stuff them into a prompt. I don’t mean that.

I mean how did people embed the words WHEN THEY USE THE EMBEDDINGS API?

I think this whole vector database and pinecone stuff will be phased out once the LLM windows get larger, like with the new Claude from Anthropic. And the ability to fine tune the model on your data. Then the LLM can make its own lower-rank model of your entire corpus of data, rather than having to do vector lookups and stuffing it into a prompt.