GPT4All Chat – Locally-running AI chat application powered by the GPT4All-J
gpt4all.io
gpt4all.io
Edit: Also no info about the author, seemingly no affiliation to openai (despite being called gpt4all), no license info on the page. Click on the github link and find some "nomic-ai" with no info other than an email address. Is it made by a Nigerian prince or something?
Remind me, how many domains do you go through again when authenticating with online version of Outlook? Last time I looked, they send you through something like 4 different domains. And emails from Microsoft seemingly are on purpose sent from as many different domains as possible.
That, of course, sounds extremely convincing and instantly trustworthy.
You are making this sound even worse.
Valuable information for me
The entire thing is fishy.
1. This is not based on GPT4. If you're going to include the words "GPT4" in your name and not base it on GPT4, what else can I not trust you about?
2. There's no direct link to the actual model used, just a reference to GPT-J.
3. I searched for GPT-J, huggingface docs describes it as a GPT-2-like model, the repo for it was started in august 2021, and hasn't really had significant updates in a long while.
Honestly, I don't want to spend time figuring out if this is worth it, trust is already out the window.
It's honestly quite sad :(
At least we know there they've painted themselves into a corner because when GPT 5 comes out, it wouldn't make sense to call the project GPT5All.
Another thing that could help make clear is to not mimic the UI style of ChatGPT.
For example, if I don't like what Microsoft is doing with Windows and I name my own OS WindowsPro while it being a Linux distribution, I'm clearly trying to mislead people into believing it's something it's not.
*(not Coca-Cola)
How about calling it GPTJ4All which is more honest, or if you think you will transcend GPTJ, GPTForAll? Or GPT”4”All?
If you want to target people who care about Apache 2.0 licensing, you should listen to the same people who tell you the name is offensive and scammy-seeming. Otherwise you alienate half your initial target market before you even start. It is not too late to accept feedback and change the name. My 2 cents.
> if you think you will transcend GPTJ
The comment you're replying to already addressed the possibility that you don't want to be tied to GPTJ in name. That's fine and understandable. The comment suggested other ideas to do a slight name change that will help you sound less deceptive while still getting to keep a similar idea in the branding. You haven't responded to the main point of the comment.
My 2¢: I saw the title on the front page and thought this was some version of GPT4 running locally. A lot of people are going to think that based on the name. I get why the name is clever but I agree with those saying you should consider a variant that is less misleading. Or, you know, just go with a brand new name. If Leela Zero was called Alpha Zero 4 All it would be weird.
Your name is much more misleading than analogous projects like "Open Office". It's clear Open Office has nothing to do with Microsoft Office just by looking at the name. "GPT4All" does not provide a clear enough level of understanding that it is not GPT4.
It wouldn't have even occurred to me to read it any other way.
“Three weeks ago”
So uh… that’s total BS, unless there’s more history to it than is obvious.
"Running on your local computer, this model is not as powerful as those GPT models you can chat with over the internet by sending your data to large powerful servers and is not affiliated with them."
Vicuna-13B v1.1 is arguably though.
It mentions it in the Github README.
>Building and running
The name could also age well after GPT5 comes out.
It stood out to me as “for” more then the current GPT model.
It might not be 100% clear, but the user of "4" as part of naming something is not uncommon in common vernacular.
The timing isn't permanent to me, the meaning of using GPT4 will prevail in the long run when GPT5,6,7 goes.
I suspect people will stop using gpt4all and Devs will come up with some inexplicable reason to name their junk project gpt7.
It'd be like Three calling naming themselves after 3g but offering 2.5g then continuing to offer than while everyone moved on to 5g.
I'm doing my best to always be constructive and I'd suggest that what we really need is something that could work as an edge processor.
That annoys me as well that everything AI is GPT something even if nobody but open ai has access to GTP models, but that's how it is.
We can't stop that train.
There is a clear difference between the GPT family and the mimicry.
Maybe it's best if the open source sticks with LLM or some other term, and popularize that instead
I mean, you could have called it gpt5, because it's 5x better. But you didn't.
Granted, maybe OpenAI will decide not to go after you, it’s up to them in the end. But if their lawyers feel these projects threaten to genericize the “GPT-4” trademark, they will be obligated to issue takedowns or they’ll risk claims that they aren’t defending their mark.
Concretely ?
I've been running Vicuna locally for several days now using llama.cpp (i.e. CPU only, because my laptop lacks a good GPU). It's not that hard to set it up yourself from scratch. Compiling llama.cpp is straightforward under Linux. The model (13B parameters, 4 bit) can be downloaded from HuggingFace.
There are several difficulties, however:
1) Vanilla llama.cpp doesn't appear to have a simple-to-use interface to interact with it from another process (your app), such as REST API (an idea found in Fabrice Bellard's text-synth) -- you want your main app to be decoupled from a process which consumes a lot of CPU and RAM and can crash. I solved it by using llamacpphtmld project which provides HTTP access to the model. It's a pretty simple project and I think of making my own Go wrappers.
2) Vicuna was trained on ChatGPT output so it often responds with garbage such as "As an AI language model..." and refuses to discuss "controversial topics". I solved it with prompt engineering where I add things like "%bot_name% never moralizes", "%bot_name% is rude" etc. (it's not always rude but that somehow stops it from moralizing)
3) it's pretty slow, at least on my laptop (1-3 minutes to process a phrase -- although I include dialog history as well)
So far I'm very pleased with the results (aside from the fact that it's very slow) -- the bot feels GPT3-level when you use it as a chatbot, or as a story generator. In fact, in my tests, it feels like it actually exceeds GPT3 in that regard. I run two bots on my IRC channel, one is Vicuna and one is GPT3-based, and I feed them same prompts to compare. Vicuna feels better as a general-purpose chatbot which can talk pretty much about anything, and it's pretty imaginative. GPT3 often refuses to talk about things, and it loses track of our dialog more quickly.
Although it's not that hard to set it up, it takes time to get things right, so I thought maybe open-source my findings as some kind of middleware to quickly integrate with other projects (localhost REST API + read-to-use weights and preset prompts). So far most projects I've seen are either very low-level (like llama.cpp) or very high-level (web-based chat).
Try this model, which uses excludes some of the "ethics" filtering.
Also, IDK what you mean by a "small" GPU, but I am getting ~3.5 tokens/sec on a laptop 2060, and I think I can squeeze more out of it. And there are methods for splitting up the load in small VRAM pools.
See the section "CPU Installation (GGML Quantised)"
You need Python to download the model from HuggingFace using the official API. After that, all you need is the binary file with weights and a compiled binary of llama.cpp
P.S. The author seems to have renamed their repo to "eachadea/legacy-vicuna-13b" on HuggingFace
Dear Customer,
We have a legal obligation and a responsibility to ensure that our subscribers are not able to reach sites which are barred by law or that are likely to cause malicious damage to their online reputation or devices.
You have tried to access a site at https://gpt4all.io/index.html which we consider to be harmful.
If you consider this to be incorrect and need access to this site, please contact us on address team@internetty.uk or call us on 0333 102 0000 or visit https://www.internetty.uk and we will review our policy for the site.
We apologise for any inconvenience that this restriction may cause you but wish to assure you that we are constantly reviewing threats on the internet to ensure that all our subscribers remain safe.
Text generation wise, it works pretty well. The text is not as fast as GPT3.5 or even GPT4. I noticed it is a lot slower when generating code.
I asked it to write a code to search Google and return the first 10 links.
Here is the conversation:
Prompt: Write a python code to search Google and return the links for the first 10 results
Response: Here is the python code to search Google and return the links for the first 10 results:
```python
import requests import re
url = 'https://www.google.com/search'
# set the number of results to display num_results = 10
# set the search query query = 'python'
# make the request to Google response = requests.get(url)
# extract the links from the response links = response.text.split('\n')
# extract the links from the response and sort them in descending order sorted_links = [link.split('\n')[0] for link in links[::-1]]
# print the first 10 links print(sorted_links[:num_results]) ```
Prompt: Rewrite the same code in javascript
Response: Here is the rewritten javascript code to search Google and return the links for the first 10 results:
```javascript
function searchGoogle() { var query = 'python'; var num_results = 10; var url = 'https://www.google.com/search'; var response = ''; var links = '';
// make the request to Google
var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.onload = function() {
if (xhr.status === 200) {
response = xhr.responseText;
links = response.split('\n');
links.sort();
links.forEach(function(link) {
links.push(link.split('\n')[0]);
});
console.log(links[:num_results]);
}
};
xhr.send();The code above uses XMLHttpRequest to make a request to Google's search engine and extract the links for the first 10 results. The links are then sorted in descending order and printed to the console.
edit: formatting
GPT also excels at translation
The abstract: "This preliminary technical report describes the development of GPT4All, a chatbot trained over a massive curated corpus of assistant interactions including word problems, story descriptions, multi-turn dialogue, and code. We openly release the collected data, data curation procedure, training code, and final model weights to promote open research and reproducibility. Additionally, we release quantized 4-bit versions of the model allowing virtually anyone to run the model on CPU."
"We already have GPT at home”
GPT at home…
GPTFORALL OR OPENGPT OR GNUGPT
I really hope it manages to catch up eventually. In terms of privacy, a local LLM is lightyears ahead of a SaaSS LLM controlled by Microsoft.
I am a little wary of the opaque installer binary that requires root permission...
Edit: Other then offline functionality.