rodoxcasta

Wait, this 'Agents' thing seems to be just a way to couple a system prompt and temperature to a model, that's it?

What's the difference from sending the system prompt in the api call, as usual?

Edit: Oh, missed that: "We’re working on connecting Agents to tools and data sources."

> Additionally, when people use CriticGPT, the AI augments their skills, resulting in more comprehensive critiques than when people work alone, and fewer hallucinated bugs than when the model works alone.

But, as per the first graphic, CriticGPT alone has better comprehensiveness than CriticGPT+Human? Is that right?

> Notices: Apple's rights in the attached weight differentials are hereby licensed under the CC-BY-NC license. Apple makes no representations with regards to LLaMa or any other third party software, which are subject to their own terms.

Wait, they can do that? Assuming weights have copyright, shouldn't the finetuning be a modification of the original work and so have the same license?

The title is wrong: this is about ChatGPT Plus, not GPT-4.

Specifically, the author is investigating (possible) changes in the system prompt and tools available to the model in the chat interface of ChatGPT Plus. That tells nothing about the model (GPT-4).

That, I always have difficulty with these types of replies.

What should you do when someone, in a serious discussion, says that the Earth is flat, with a straight face?

Are they mocking you? Perhaps mocking the debate at hand? Are they trolling? Or maybe they are just naive and don't know that they are embarrassing themselves?

Maybe just treat them as a troll? But the thing is that when they appear just as some naive, maybe young person, the generous take would be to explain the ridiculous thing they are saying.. but I always feel so fool when I do that. Plus, I don't have the patience anymore.

on my firefox it didn't work, just keep asking to connect a security token, with no other option. Firefox 116, Ubuntu 20.04, had an Yubikey in the past, never used passkey.

I think it's asking for the old yubikey, but neither the site nor firefox give me other option or a link about what to do.

AFAIK the main difference is in certain provisions designed to block the use of patents to restrict the fundamental freedoms that GPL allows to the user. The main drawback is that it's incompatible with GPLv2-only software, but for new software it doesn't seem to be a problem.

Also, maybe A-GPL could be a good license here. It adds a provision that if the user accesses the code remotely (as on a server), you should share the code too. The default GPL only requires that if you distribute the binary.

PS. not a lawyer, would be happy to be corrected if something I said was wrong

While the article has an obvious point of view, it cites other people defending their view , with plenty of links, and even the response from Harvard ('we don't comment individual investments'). The person that Harvard hired to be their ethics monitor resigned over this. It's not some invention from the head of someone.

Is someone using some old phone, but with a recompiled kernel, to support standard linux features, like docker? It's possible in theory, but still didn't found someone doing it.

Seems that running docker in an old android repurposed as a server is still too much niche, unfortunately.

Bonus if it can run KVM with hardware support, but I don't know if it's possible yet.

If your loss goes from 5% to 1%, you have to deal with 80% less heat. So you can make 3x smaller motors. All the powertrain of these machines will be hugely simplified.

That's no small deal, but in the grand scheme of things that a hot superconductor can give us.. I mean, this can (possibly, with decades of research) give us fusion, quantum computing, etc.

For inference, at least locally, the bottleneck is usually the memory bandwidth (and quantity, of course).

I hope that AI hype lead us to more memory and more memory bandwidth, because they are really lagging behind computer power increase from like 15 years already.

That's an interesting math. I don't think they are using 4 bits, or even 8. My bet would be with 16 bits. (Bear in mind that's just speculation, for "math's sake").

So we are talking about 4x your numbers per specialist model:

180GB * 4 = 720GB. If you count the greater context, let's say 750GB.

Anyone remember how many specialists they are supposedly using for each request?

If it's 2, we are talking about 1.5TB of processed weights for each generated token. With 4, it's 3TB/token.

At 0.06 for 1k tokens we get

3TB*1k/0.06 = 50 petabytes of processed data per dollar.

Doesn't seems so expensive now.

The thing is, you (we) only identify chatGPT generated content when it has that generic voice. Maybe there's a lot more generated content here, but it isn't so obvious. It's a selection bias, we see mostly what's easy to see.

If the weights are not copyrighteable, you don't need a licence do use them, they are just data. There's not a right to infringe if these numbers have no author. Of course, to use openAI API you must abide to their terms. But if you publish your generations and I download them, I have nothing to do to the contract you have with openAI since I'm no part of it. They can't impede me to use it to improve my models.

> The sequence of these two threads is just too perfect. Almost likely someone is trying to make a point.

Exactly! Almost every weak point that Knuth commented is fixed in GPT4 answers.

Maybe OP feed Knuth's observations to the model?

If that ins't the case, I'm really impressed.

> There's no underlying theory of mind here.

Actually, there's some experimental evidence that GPT4 have a Theory of Mind as good as humans, maybe better.

https://arxiv.org/abs/2304.11490

> GPT-4 performed best in zero-shot settings, reaching nearly 80% ToM accuracy, but still fell short of the 87% human accuracy on the test set. However, when supplied with prompts for in-context learning, all RLHF-trained LLMs exceeded 80% ToM accuracy, with GPT-4 reaching 100%.

They focus the immune system on the tumor by teaching it to identify some "neoantigens" in the tumor. Turns out that tumors express various strange proteins in their cell wall, I suppose because of the inherent genetic instability.

Anyone know why this can't be a silver bullet for most tumors? Like, a way to target an intrinsic characteristic of cancers that in theory could be very effective on most of them.. sounds too good to be true. So, what's the catch?

They record the brain language processing via fMRI and the activations of some AI models during language tasks, then create a linear map between them. Then they use that map to try to predict how the brain will process a language task using the AI model activations for the same sentence. This holds true for different imaging techniques and different language tasks.

Transformers perform qualitatively better than other architectures, and GPT2 (the most advanced public model at the time) shows near 100% accuracy. The best correlate of performance in the experiment is the next-word prediction accuracy of the model. Other AI performance metrics don't appear significant.

The conclusion is that this is strong evidence that the brain processes language using the same predictive algorithm as transformers. And GPT2 may have an architecture very similar to the language processing areas of the brain.

Abstract The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a systematic study taking this approach to higher-level cognition: human language processing, our species’ signature cognitive skill. We find that the most powerful “transformer” models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (functional MRI and electrocorticography). Models’ neural fits (“brain score”) and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

We are all crafters, and I admire the attention to these presentation points of OP. This is clear in the documentation and explanations at the repository too.

That said, I don't think the questioning of GP was malicious, just a natural curiosity. Yes, a little suspicious, but, well, we are in the internet after all. In the least, it's good to point when someone does the extra work to make a great presentation.

Anyway, great work riter!

One thing to keep in mind is that for most of the war, it had strong public support. There was a continuous defamation campaign against the anti-war movement. Days after the shooting at Kent, a poll showed 80% agreeing that it was the students' fault.

I'm speculating, but I think that the few who refused to serve were met with the same outrage that someone like Weinstein receives today. It's no surprise that most eighteen-year-old boys could not do that, even if they were strongly against the war.

Actually, replication is very important. If no one can make new llamas, that would mean that facebook used some secret sauce in their training. Understanding publicly how to train these 'enhanced' models that shows performance of much greater models is a very strong motive.

And getting hid of the NC clause of the original llamas too, of course.

As of right now, there's trouble replicating the eval results of the paper, for example.

But, this isn't a bad ideia when you don't know even the basics? Because you wouldn't be able to separate genuine information to subtle or not so subtle hallucinations.

It's like generating code in a language that you know nothing about. You should check for bugs, but you can't.

I do that with two tablets and a laptop. On the notebook I run some vnc server with an X built-in, then some x or vnc tool that links the different X instances running as if they are other monitors, so you only have to move your mouse to some lateral to get focus on the other X. Then on the tablets just use some generic VNC client.

There are multiple options for ever one of the three programs os this setup, but it takes some work to make all run well together. I can check the exact binaries that I'm using or send the scripts, if there's someone interested.

Edit: to make it work well enough with wifi, I had to make some tricks to deal with the tablet's wifi powersave behavior. On usb it works perfectly though.

With 2-shot and chain of trough prompting, GPT4 got 100% right answers in the Theory of Mind tests of the researchers. Humans got 87%.

Some points addressed in the paper:

- Is the model emulating the reasoning of the 2-shot examples?

> The Davinci-3 and GPT-4 models experienced increases in ToM performance from all of the classes of CoT examples that we tested: Photo examples, Non-ToM Inferential examples, and ToM examples. The mean accuracy increases for each model and each type of CoT example are shown in Figure 4, while the accuracy changes for individual ToM questions are shown in Figure S.1. Prompting with Inferential and Photo examples boosted the models’ performance on ToM scenarios even though these in-context examples did not follow the same reasoning pattern as the ToM scenarios. Therefore, our analysis suggests that the benefit of prompting for boosting ToM performance is not due to merely overfitting to the specific set of reasoning steps shown in the CoT examples. Instead, the CoT examples appear to invoke a mode of output that involves step-by-step reasoning, which improves the accuracy across a range of tasks.

- Is the test data included in the training?

> The LLMs may have seen some ToM or Photo scenarios during their training phase, but data leakage is unlikely to affect our findings. First, our findings concern the change in performance arising from prompting, and the specific prompts used to obtain this performance change were novel materials generated for this study. Second, if the model performance relied solely on prior exposure to the training data, there should be little difference between zero-shot Photo and ToM performance (Figure 2), as these materials were published in the same documents; however, the zero-shot performance patterns were very different across Photo and ToM scenarios. Third, the LLM performance improvements arose when the models elaborated their reasoning step-by-step, and this elaborated reasoning was not part of the training data. Therefore, although some data leakage is possible, it is unlikely to affect our conclusions concerning the benefits of prompting.

Other highlights:

- With the prompting techniques 3.5-turbo got human level performance.

- In zero-shot GPT4 is already near human performance (90% of the human score)

- In zero-shot regime 3.5-turbo is worst than davinci-3, but much much better than it with prompting. This happens because turbo is too much cautious by default and often refuses to draw conclusions.

First orbital attempt of Startship got halfway there before exploding