They are based on llama2 architecture which performs a lot worse.
1,073 karma · joined October 23, 2019
It is all financially-driven. All of the AI companies are working on models constantly are are purposely putting out releases so they can be ahead of the game or have some nice news for investors in order to make sure their stocks grow.
OpenAI probably know they are ahead with GPT-5 as well and they are planning to drop it Q3/Q4 after Llama3 (which will maybe trump GPT-4), so they can be ahead again for the EOY earnings.
we only need a vector database if we want to do semantic vector-embedding search AND our dataset is too large for memory
long answer:
RAG is just a pattern of working with LLMs, independent of particular database technologies.
Basically it means you inject some context data or domain specific data into the prompt to achieve the following:
1. Nudging the model into the right direction so it will use the "correct parts" of it's "global knowledge". We want to nudge it into the correct domain or a domain very similar to our problem. We do this to generate better answers and reduce hallucinations or to adhere to a particular style we want to achieve. This is basically just a prompt-engineering technique.
2. Achieving in-context learning by providing some context to the model. This is also a prompt engineering technique used to make the model reason about things it does not know from the training and to allow it to reason about a particular text you provide at runtime.
What I just explained is basically the "AG"-Part of RAG: Augmented Generation. It boils down to putting some external data into the context.
The R (Retrieval), on the other hand, is about finding the correct data to put in the context.
If you copy an email and paste into the prompt of ChatGPT, you are basically already doing RAG. The R is copy pasting the Email from Gmail, and the AG is putting it into the Prompt.
So, no, generally speaking you don't need a vector Database.
In the real world you will probably have a large context (some files on the disk, a PDF file, etc.) and you don't want to manually select the relevant bits of information, of course. We want to automate the Retrieval part of RAG. This is mostly what people mean when they talk about "RAG". Essentially it is a information retrieval, or search problem.
A lot of programmers and projects which come from the LLM community will use RAG with a semantic vector search. Semantic vector search has shown to be quite good at selecting relevant context chunks from full text and it also feels natural because it will use embedding models, which are very similar to LLMs. By using vector-embedding search we simulate with a manual step what we think will happen when we put all the data into the LLM context: Selection through semantic interpretation.
In this case we need some form of "vector search", but we do not need a database for that per se, we can also do that in-memory. However, if our dataset is too large to be efficiently processed in-memory, we would need a vector database.
However, we can also use any other means of information-retrieval techniques or search techniques, like keyword search, full-text search, knowledge graphs, sql queries, and many more, to retrieve the relevant bits of information we want to pass to the model.
In that case we also don't need a vector database. Maybe some other database, maybe a search tool like elastic.io, maybe no database at all.
We cannot simply state that at some point in time RAG will not be necessary. Like everything in the computer science world it always will depend on our data size and the resource constraints we have.
Unless of course we can process a corpus the size of the whole internet in <1 second. However, I doubt this can be achieved in the next 20 years.
Instead of computing every token every time on the whole context, we can grab a cache to make some shortcut. We do the same in software development all the time. Of course it's a performance issue.
With RAG tools that exist today, we can already do things like
- providing summaries
- hierarchical summarization
- generation of questions / more prompts to nudge the model
- caching
- using knowledge graphs, function calling, or database connectors for non-semantic data querying
etc.
Regarding the index usually a mix of BM25 and vector index seems to perform best for most generic data.
As far as I know .env file is the main supported way of setting environmental variables in VS Code.
- compiling from source
- need to install build-essentials on docker images which takes up lot of space and takes a long time
just install the different python versions by downloading the binaries or through your package manager and run python3.<whatever_version_you_want> -m venv and the virtual environment will automatically, forever, use that python version when you activate it.
Python really makes things difficult :D
This could be easily implemented with langchain for example.
Sometimes I use HN search to look for good threads about a topic.
Of course all of them feel like a Frankenstein compared to actual ChatGPT. They feel similar and work just as well until, sometimes, they put out complete and utter garbage or artifacts and you wonder if they skimped on fine-tuning.
1. Listen to what smart people say and accept their belief about the purpose of life. (by the way most will agree with the above paragraph and accept there is no inherent purpose in life other than what we decide it should be).
2. Listen to what the majority of people say and go with it.
3. Define for yourself.
It is quite clear that number 3 is the one we would recommend, because only this way can you guarantee the purpose of life which you have defined is something that fits to your view in life.
In the end you will maybe see that the question about the "purpose" of life is a paradox. Life exists on itself, without us, and without us reasoning about it. So why would it need a purpose? A purpose implies that there is a design, a creator.
For this reason I dislike the terminology "purpose" and I rather like to say that we may choose to take on a life task.
Due to contrasting and negative experiences in my life I chose my task to be liberation of or from the mind. The basic idea is to liberate ourselves from constant pondering, worrying, and analyzing which our mind is doing constantly because we needed that to gain an evolutional advantage. But mostly we are unharmed today and not in peril, so our mind is underutilized and we develop psychological illnesses. This is my personal take, maybe yours will be different in the end, maybe it will be similar, who knows.
If you are interested to learn more, I can tell you that there are many many variations of it throughout history, some may call it religion, spirituality, meditation, zen, etc. In the end all of these things boil down to looking outside of yourself at the wonders of the world and inside yourself at your own feelings at same time in order to stop our attachment to the mind which is solely concerned with trying to reason about things which are inexplicable. And most religions, be it hinduism, buddhism, taoism, christianity, or whoever have similar takes on this. The idea is living in the present moment instead of in the future or past, where our mind mostly spends it's energy.
Nobody can tell you what the purpose of life is, you have to chose for yourself. If you choose to take on something which has stood the test of time and was formed over thousands and thousands of years, an idea forged by millions of people, you may chose to engage also in the art of letting go, of freeing the mind, of stopping to worry. Or as Patanjali the writer of the famous Yogasutra called it: Stopping to identify with the ebbs and flows of the mind.
There are so many ways to dive into this topic, like Yoga, meditation, or books like Eckart Tolle's "The power of Now", or even a lot more in popular literature. It has helped millions or billions of people in human history to engage in some form of meditative praxis or spirituality, for a good reason.