ghostextension.com
For compiling information or getting an immediate yes/no it's likely correct - but I found ctrl+f generally gets me there faster albiet with slightly more reading.
At least in the context of this lease agreement which does have everything well organized and uses carefully chosen keywords already.
Consider the example question of "I won't be able to pay until the 9th of this month, will I get a fee?" - are you going to search for "fee"? There are 66 occurrences.
Modify the question to "If I pay on the 4th of the month, will there be any late fee?" and you get the correct answer too.
For the question "What restrictions are there on parties?" it appears to get that correctly answered. If you search for "party" you'll get 19 results that appear to be legal entity parties rather than the possibly noisy type.
In my testing the biggest challenges with using for example OpenAI embeddings with cosine similarity or something are A) figuring out the section breaks or right chunk size so that information stays in context and B) retrieving enough chunks to get the correct hit for a query without having too much extraneous information that confuses it.
I think that it's hard to make a parser that most optimally slices up arbitrary documents.
Since you have some larger documents preloaded I assume for those you have the embeddings search. But for user uploads you are skipping that now and just feeding all of the text extracted from the PDF into the prompt along with the query.
It also seems to be one of the most important limitations of ChatGPT, and a lot of people/teams are looking for solutions.
Which option seems to you to be the best alternative? And where do you see the future of this?
https://www.konjer.xyz/the-alchemist (disclaimer: built by me)
What specifically is missing from the answers in your opinion?
I want it to understand a complete fiction book and tell me about how a character grows throughout their journey from chapter 1 to chapter 12 over 350 pages.
Then each excerpt could be fed to the LLM asking it how this part relates to the question you’d want an answer to.
Then ask for each what it shows about the character and it’s personality, weaknesses, etc.
And finally recursively summarise them, asking for the summary to show how the character has grown through the summarised content.
Basically ending up with a map-reduce.
Bigger sources, or lots of content related to the character, would lead to less accuracy, and increase the likelihood of hitting the window’s limit.
It would also be highly specific and quite brittle, although one could probably turn it into a more generic process / pipeline (ie what dust.tt enables).
I might have completely missed your point or overlooked some glaring flaw though, in which case please do let me know what you think.
It's byok. Keys are not persisted. You can choose chat-gpt-turbo or text-davinci.
Limit is 2.4M tokens per call, working to get higher too.
How could one ever trust the output of ChatGPT?
This feels to me a bit like non-L5 autonomous driving: If I have to assist at all, it'd be easier to do it myself. In the same vein, for this project (and ChatGPT generally): Can I actually trust that the output from ChatGPT in answering my question about the document is factually correct?
e.g., If I hand it a home rental agreement legal document and ask "What is the late move out penalty if I am 10-minutes late in dropping off the keys?", it may give the correct answer. Or it may generate a plausible-sounding answer using the words in the document that is completely (or perhaps even just slightly) incorrect.
How could I possibly know without reading it myself?
Yours is not a good example though because "10 minutes late" is never going to be in a document like that.
This seems similar to ChatPDF.com (with a 200 page limit though, instead of the 5 page limit that you have, it seems) which I suppose we'll see a lot more competitors for as the ChatGPT API expands.
Have you thought of even larger knowledge-bases? like entire legal systems etc...
Anyway, amazingly executed, nice work!
You then put those vectors in a vector database (e.g. pinecone, pgvector, chroma).
To run searches, you generate an embedding of the search term (could be the raw user search, could be something a model like ChatGPT was asked to transform the user's search into), then query the vector database for the n closest vectors. The trick is getting a model that generates good vectors for search (and transforming the user's query into some text that'd be useful vector(s) to search against). If feeding that into an LLM context, the next step is making sure that you get your prompt right, and don't overload the model with unrelated information (i.e. bad search results).
The key is that the vector representation embeds language concepts in how close vectors are to one another. An easy way to gain a feel for this is to look at single-word embeddings. Computerphile have a great episode on it[1]. You can take a vector for 'King', subtract the vector for 'Man' and add the vector for 'Woman' and the closest vector in that search will likely be 'Queen'. Scale up this idea to whole paragraphs (and larger vectors as a result).
LangChain has an example of searching a database of facts[2] (although I find their documentation pretty inaccessible - they explain their library, but don't step back from inside the weeds of what they're doing to really explain why / what's going on). Many of the features LangChain implements are distilling (or sometimes simply lifting and providing a toolkit to directly apply) LLM papers.
1: Computerphile Word Embeddings https://www.youtube.com/watch?v=gQddtTdmG_8
2: https://langchain.readthedocs.io/en/latest/use_cases/questio...
It's like, imagine there's a complex machine with large panels full of buttons and levers - and then, someone covered the panels with tapestry. Beautiful tapestry, showing artistic interpretations of things mundane and holy, trivialities of everyday life next to impossible dreams. And then, people were told the machine is to be operated by touching that tapestry, and that the artworks are the guide to understanding it and using it effectively. And then a whole religion formed around studying patterns in the tapestry. To me, prompt engineering is that religion.
There's an actual interface to the machine hidden under all the clever wordplay. A precise, formalized one. An interface that eats tokens and spits out probabilities. I just don't get why most talk - even seemingly specialist talk - about LLMs is ignoring it entirely, and focuses on the tapestry that's just obscuring the nature of the model, effectively making everything more difficult.
I was on a call this morning and heard someone refer to two of their team members as "Prompt Engineers" as if that were an actual role.
Which would make sense if we were talking about humanity discovering magic is real and trying to reverse engineer it based on ancient spell books[0] - but we're not. We're talking about deep learning models made by other people, using publicly available knowledge and techniques, and often with source code and training set being publicly available too. Prompt engineering feels like people purposefully trying to treat technology as magic.
--
[0] - Or any of the scenarios equivalent to it under Clarke's third law, such as finding a crashed alien starship with a working black-box AI in it, built on a computing substrate we can't even identify, much less prod with a signal generator.
Or, put another way, using text/conversation as user interface in a model is turning a normal engineering problem into a much harder reverse engineering problem. Why would anyone want to make life difficult for themselves this way, and ultimately turning engineering into voodoo?