Hidden Changes in GPT-4, Uncovered | Hacker News Reader

Hidden Changes in GPT-4, Uncovered

dmicz.github.io

193 pointsdmicz140 comments

I never really understand why people assert “tell me what I just wrote” or “tell me your system prompt” etc are going to give you actual results. How can you know it’s not just making something up?

Correct. Every single time these people are incorrect and we keep having to explain this

To say it can't “tell me what I just wrote” is to say it can't copy parts of the context.

We know it can copy parts of the context and the system prompt while a special part of the context isn't immune to being copied.

You can test it yourself by adding random strings to the system prompt, you can consistently have the model copy them over. Is that not enough to have a reasonable belief that the model can copy system prompt instructions in the web interface?

Or, get this, that you're wrong and refuse to admit it.

Those that experiment with local models like myself can demonstrate to you that leaking the system prompt is not difficult at all.

It's some strange kind of neurosis to harbor such incorrect and strong beliefs on matters you have zero expertise in.

perhaps because it gives the same answer, verbatim, for many different attempts to figure it out?

and because we know it well enough to be sure that it's not smart and devious enough (yet) to conspire like this? (nor has any clear reason to)

Its well known to spit out nonsense.

Conspire? It's one system controlled by one company.

It's a valid concern that ChatGPT could be hallucinating this, but there are a few things that strongly suggest this isn't what is happening in this case:

1. ChatGPT reliably produces the same output across several different instances, and other people have independently found identical versions of this text with different prompts.[1][2] This typically wouldn't happen with a hallucination, which may change with each time the model is prompted.

2. The instructions accurately describe the capabilities and restrictions of ChatGPT's function calls. When making custom GPTs, the browser tool and DALL-E image generation tool are options, and the "system prompt" given by the custom GPTs reflect whichever tools you've selected.

3. ChatGPT has reliably changed with the changes noticed in the "system prompt". I document a recent change made to the prompt on my post from yesterday.[3] On the older instances of ChatGPT I have, the model will suggest it has no idea what the "guardian_tool" is or make any attempt at stopping a discussion of U.S. elections, while newly made instances of ChatGPT will discuss the "guardian_tool". The "system prompt" repeated then must give at least some sense of what updates are being made under the hood.

I certainly think we should still take the idea that this is the "system prompt" with a grain of salt. There is a good discussion about this here: https://news.ycombinator.com/item?id=37879077

[1] https://www.reddit.com/r/ChatGPT/comments/18494zo/what_are_t...

[2] https://medium.com/@dan_43009/what-we-can-learn-from-openai-...

[3] https://dmicz.github.io/machine-learning/chatgpt-election-up...

You can also play around in the playground or with the API with your own system prompts and see that you can jailbreak them to report the prompt.

In (1) you imply that hallucinations are strictly due to nondeterminism in GPT computation. A hallucination happens (IIUC) because of the numeric imprecision, model regularization and various thresholds. In short, the hallucinations can be reliably reproducible (but, they can also happen due to non-deterministic computation).

There is empirical research showing that hallucinations in LLM tend to vary enormously from one answer to the next compared to accurate answers.

Of course, it could be that GPT-4 has been instructed to lie about its prompt, but failing that, you should expect any answer that stays the same across multiple wordings and prompting methods to be accurate.

That's mostly intuitive.

An accurate answer is often driven by a concrete and highly confident fact in the training dataset (e.g. structured data fact, like a birth date from Wikipedia etc.).

The hallucinations are derived facts of (hopefully) low confidence. Nondeterminism is more common if you have low scores. Only a few facts can take high score (in a usable system), while many can take a low score -- then numeric instability can make a mess.

I'm not very familiar with LLMs, but I do have experience with the traditional ML models and content understanding production system. But, LLMs are not far from them.

I'm not sure if I'm reiterating what you're saying, but isn't an element of it that the model doesn't have the storage capacity to literally contain all the information it's trained with, and so it's necessarily extrapolating from it's representation of knowledge, which is going to sometimes miss the mark?

If it "leaks" the exact same text multiple times, to multiple users, using different prompts to elicit it, it's a reasonable bet it's not just a hallucination.

Especially since (I'm fairly sure? though this is apparently not trivial to find today) it doesn't use temperature=0.

Count the tokens, use different prompts. It's fairly reliable to do this from API but I haven't tried with the web app.

True, there's an easier way to get that information. Download your chat history and look in the conversations.json file. It's all there.

I downloaded my chat history just now and system prompts are not present in it.

Look for TOOL in the conversation.js The system prompts are empty but the tools aren't. Ask ChatGPT something like "What is the price of IBM stock", then download the history and you will find a bunch of TOOL sections there.

I don't have ChatGPT Plus to check that out. Could you share the latest DALL-E 3 system prompt?

I asked it to create an icon for an app and there are two Tool entries:

1. DALL-E generation metadata: 1. A stylized chat bubble integrated with an abstract brain design, representing AI and chat. 2. A sleek, modern avatar that looks like a digital assistant, with elements like a clipboard or list to represent organization. 3. A microchip or circuit board pattern in the shape of a speech balloon, merging AI technology and chat. 4. A robotic hand holding a pen or stylus, positioned over a notepad, symbolizing AI's role in organizing chats. 5. An organizer folder or agenda book with digital or futuristic motifs for AI integration. 6. A light bulb with chat bubbles around it, showing the concept of generating and organizing AI conversations. Seed 8893527578

2. DALL·E displayed 1 images. The images are already plainly visible, so don't repeat the descriptions in detail. Do not list download links as they are available in the ChatGPT UI already. The user may download the images by clicking on them, but do not mention anything about downloading to the user.

Right, so none of the instructions for GPT to call the DALL-E function, none of the prompt expansion rules and none of the censorship rules. Clearly the system prompt is not present in the file OpenAI gives us.

I see what you mean, yes correct.

Cause asking is immediately verifiable?

I give those prompts when I want to confirm or verify the input prompt is what I expect so, that's kind of the point.

I've never gotten back a result that was different than my recall and then I can verify it with a previous conversation stored.

Are you worried that the system will gaslight you into believing you gave a prompt that you didn't?

Copyright was never conceived to apply to technology like this and the onslaught of copyright suits (like the NYT one) underscore its fundamental rent-seeking nature. No doubt these latest changes to GPT-4 are in response to the suits they’re presently fighting. However these cases are ultimately resolved, the end-user will be the biggest loser.

Because ChatGPT users are the only people that are worth considering.

The artist or author might end up being the loser, and the multi billion corporation harvesting their work might make an unearned profit off it.

To me personally it's crazy how many people think that we would be better off without any kind of copyright protection. Copyright solves many real world problems and protects people against having a company profit off their work... but as soon as AI is involved so many people start to advocate for throwing it away.

I don't think most people believe we are better off without copyright. I think people believe that copyright protects specific concrete expressions and that fair use exists to allow others to build on ideas in transformative ways. It's not clear where building a learning model from this work sits in this context, hence the court cases.

Also, it's a subtle difference, but copyright is not intended to solve the problem of companies profiting off of artist's works, it is intended to promote the progress of science and useful arts. It attempts to do this by giving creators limited exclusive rights.

How does locking away most of the knowledge, research and learning materials in the private vaults of a few publishing houses for their personal profit promote the progress of science I wonder?

Even scientists are tired of the predatory and rent seeking behaviour of the publishers they have fallen prey to and are looking for any way out.

This is not promoting progress this is the opposite of it

I think it grossly mischaracterizes what copyright protects to describe is as "most of the knowledge, research and learning materials". Still I agree, that the extensions of copyright length and the behavior/incentives of publishers works against the original intent of copyright. Having said that, publishers only have control of copyright because authors give it to them. Copyright rests with the creator — the system where people are compelled to sign this over to publishers is a different (but of course related) problem. Scientists who are tired of the predatory behavior of publishers have other choices today. It's not clear what alternative you are proposing.

> vaults of a few publishing houses for their personal profit

Because they made it, it wouldn't exist without them, and others value it. If this data wasn't objectively valuable, we wouldn't be having this discussion.

If companies are required to purchase licenses for everything they train on, it will guarantee that only huge corporations with deep pockets can produce powerful models. Microsoft will be slightly inconvenienced, Stability AI will be destroyed. Some artists might get a payday, but most of the money will go to companies with large copyright libraries like Getty. The general quality of all models will decrease. I don't see any other possible outcome.

Almost a year ago, I made¹ the following prediction:

It looks like to me that many companies want to use the new generative tools, and many others want it not to impact their stake in the copyright system. I’m pretty sure they will both come to a compromise which will leave most users without any benefits, either from reduced copyrights or from availability of generative tools. It’s what would make both powerful parties satisfied (if not happy), and will impact the status quo the least.

Say, for instance, that they instituted a mostly mandatory licensing scheme, so that an individual artist had no choice but to allow use of their art as input when creating generative tools. People using art in this way have to pay a rather high licensing fee, but it is not paid to the artist, but to some sort of central copyright office. Huge copyright holders can also pay an exorbitantly high fee (to the same recipient) to opt out of licensing. Win-win-win; Existing copyright holders keep their existing copyrights, only large-ish actors can create new generative tools, new political positions and institutions are created with lots of money flowing in. Of course, artists then get screwed by being co-opted by generative tools which they can never afford to create themselves, and the general public get robbed both of the opportunity of using and creating new generative tools, and of any less restrictive copyright law.

1. <https://news.ycombinator.com/item?id=35191112>

I hope you’re wrong, but I think you’re right.

For music there are already similar mechanisms in place in many countries - in Poland it's ZAiKS, in US it's ASCAP. They collect fees from organisations playing copyrighted music publicly.

(I agree that it would be terrible if they began enforcing other copyrighted content and for training purposes, because it would lead to centralisation)

Sacem in France.

They're the worst, eg they will notoriously come after you if you play public domain music as well.

For many artists this is not about "getting a payday" and is instead about "not being replaced by AI". So the outcome you describe would probably sound great to those artists.

How did dock workers feel wen containerized shipping starting gaining popularity? Should we have let them all continue putting things on ships piece b piece and stacking and unstacking each shipment by hand?

How did portrait artists feel when photography was gaining popularity? Should we have let them control the industry so that if we want to record a memory of a person we must have them stand or sit for hours while someone draws them?

etc.

Man there's always someone in these discussions who will smugly tell us that this is all inevitable and our empathy for the creatives in our economy is misplaced. To you I give a hearty fuck you.

No, I am describing what happens when technology makes the market for certain jobs and talents change. The stevedores may have had a bad time for a while but our modern society only exists because we can ship things quickly and efficiently.

I feel bad for copy editors and people who write corporate blog posts or design logos or come up with ad jingles, but their niche is gone now and they need to adapt.

Thanks for being respectful and cordial though.

I often see these processes described as passive economic mechanisms that we are subjected to and not as decisions that we all make collectively and actively accept, making excuses based on the neoliberal understanding of our time as to why those people deserve to have their jobs made redundant and their livings wrenched from them.

To me, it's a kind of cowardice that people like you shrug your shoulders at and sigh and say "that's just the way things are". You can say that's just how the markets work. I don't have to respect you for it.

I am not saying that artists are going to stop being a thing. We will keep buying books written by people and watch movies directed by people and people will still make music and what have you, but it will be different. The music industry was completely different in 1900 when there no available mass recordings, different again in the 1950s with popular radio and records, and the 2000s brought the internet and MP3s.

Things change -- people's jobs will be different. It isn't going to mean artists will stop making art or machines will make everything bland, it is just a new tool that will change industries and make things easier for people to do well and thus make more art. Some people won't be able to live well doing the same thing they do now, but what they do now wasn't what they would have been doing if they were in their grandparents time.

I'd say you are creating a bit of a straw man there. The commenter you are responding to didn't say that's just the way things are. It feels like you are making their argument for them.

They showed some examples in the past and showed that society adapted.

We could try and improve our society and systems to have a safety net, education that allows us to adapt to rapidly changing technologies, etc sure but that's a whole discussion in itself.

If you give people freedom (good thing, right?) and tools exist to perform a task in a variety of different ways (some faster/more efficient than others) people will naturally gravitate towards using the most efficient tools to gain a competitive advantage, and other people will prefer work produced with those tools because it's better/cheaper. As long as better tools exist and people are free, this is just the way things are gonna play out.

If you're angry that independent artists are being fucked over by bigcorp, AI tools aren't the battle you should pick, because it's a guaranteed loss for a lot of very logical reasons, and it's just another example of a pattern of oppression enabled by our social and political systems. Even if by magic you managed to change something there'd just be another inequality coming down the pipe shortly after.

The good artists are already using AI, just like they photobashed, traced templates and used camera obscuras to produce better art faster down through the ages. A true artist transcends medium to focus on message.

AI is a tool. different artists use different tools. some good artists use ai. many good artists will not be interested in that particular tool.

In agreement with your "slightly inconvenienced": The world's dozen or so largest publishers have market caps averaging below $10bn range each.

"Even" just OpenAI alone could pocket a few of them if they need easy sources of acquiring content.

This includes the largest educational publishers. And while these publishers do not own all their content, the reality is most authors earn so little, that a "allow AI training on my work for $x extra" would give them vast amounts of content.

As for Getty, Getty has a market cap of "only" $2bn. The big players will easily afford to build or buy libraries like that.

But of course it will be the end of decent open models.

> it will guarantee that only huge corporations with deep pockets can produce powerful models

It will also guarantee that the financial means to continue making that data, that is clearly so important, would be preserved. Someone has to pay for the crafting of the data.

How does it protect a small artist against a large corporation profiting off their work?

I don’t even have the means to start litigation, let alone see it through.

It only protects those who are already moneyed and/or famous enough to negatively impact a large corporation’s reputation - and even in those cases it’s mostly for the benefit of the lawyers and bureaucrats who make a living off it.

If you register your work, which requires some effort, but is not prohibitively expensive or difficult, you can sue for statutory damages, which are substantial enough (up to $150k for willful infringement) that lawyers will work on contingency. There are many individual artists how have been successful here. The law actually has some real teeth that individuals can use to protect their work.

It would be nice if there was a preventative concept, where the role of the creator being a predator, seeking and suing, would be mostly reversed, so that others would instead ask for permission, and maybe get the rights to copies through a fair exchange of money, like a license. We could call this "copy rights".

> but as soon as AI is involved so many people start to advocate for throwing it away

No, it's been years I've heard it.

Don't try to portray some people opinion as they are some AI zealot.

It's brought up on discussions about torrent, Disney, streaming platforms, music, etc...

Yes. I've been aware of the intellectual property debate at least back to the great crackdown on sampling around when Paul's Boutique was released. And following it in depth from around the time Lawrence Lessig made arguments to the Supreme Court.

A large chunk of the tech community was following that case and most on HN seemed to be highly sceptical of the current status quo.

> The artist or author might end up being the loser, and the multi billion corporation harvesting their work might make an unearned profit off it.

Exactly like before AI you mean then? Except instead of OpenAI it was Disney, Universal and other large corporations on that same seat.

>to me personally it's crazy how many people think that we would be better off without any kind of copyright protection.

Why should I care that the old billionaire copyright corps are dying exactly? What would I benefit defending them for me as what they did was privatizing culture as far as I remember for their personal benefit and even had a large negative influence on tech.

The copyright system being so unequal and skewed towards multi billion companies dug its own grave by itself.

OpenAI is the force to cut slice from the copyright pie which the big copyright hoarders have. The hoarders will not strike back to try to kill the OpenAI business. Because in any case they will not be able to kill the technology itself. So, obviously, it's better for them to have OpenAI as a partner and share some profit with them to control the AI field than to kill this one and wait for another AI menace to raise.

OpenAI is not the one who would kill a copyright. They just want their cut.

The copyright issue seems unchanged. Anyone taking wholesale quotes from another entity is likely in violation of copyright law. If someone uses AI, and posts the output from it as their own work, and that work contains copyrighted material, the person who posted it is in violation of copyright. AI is just a tool they chose to use and they remain responsible for remaining in compliance with copyright law.

What we need is a reasonable way for people using AI to determine which parts of the text or images they have are subject to copyright.

Just a tool that required billions of dollars worth of copyrighted material to be created.

How can you possibly argue that taking a bunch of text and creating an application that creates text isn't transformative?

The tool itself unambiguously is fair use.

Whether something is transformative is one of 4 tests for fair use.

> Anyone taking wholesale quotes from another entity is likely in violation of copyright law

What do you mean anyone?

Is Sony liable when you play an entire movie on their TV? Is Nuance liable when you use their Dragon screen reader to cerbalize an entire NYT article? Is Google liable when you display an entire webpage in Google Chrome? How about if you switch to Dark Mode, is that a transformative use?

Why would AI be any different? It’s just a tool at the end of the day!

I'm sure you'd feel the same way if it was your life's work these systems were hoovering up and regurgitating.

The problem is people at large companies creating these AI models, wanting the freedom to copy artists’ works when using it, but these large companies also want to keep copyright protection intact, for their regular business activities. They want to eat the cake and have it too. And they are arguing for essentially eliminating copyright for their specific purpose and convenience, when copyright has virtually never been loosened for the public’s convenience, even when the exceptions the public asks for are often minor and laudable. If these companies were to argue that copyright should be eliminated because of this new technology, I might not object. But now that they come and ask… no, they pretend to already have, a copyright exception for their specific use, I will happily turn around and use their own copyright maximalist arguments against them.

(Copied from a comment of mine written over a year ago: <https://news.ycombinator.com/item?id=33582047>)

> these large companies also want to keep copyright protection intact, for their regular business activities

Care to share an example? I didn't hear of OpenAI or anyone else arguing or trying to sue anyone for abusing the copyright. If anything, their business decisions rely on an assumption that copyright will not help them protect their work

Don't they have an explicit T&C that says you are not allowed to use their output for training other models?

T&C yes, but not copyright.

Prime example for you right here:

https://nypost.com/2023/12/18/business/openai-suspends-byted...

100% pure unadulterated hypocrisy from "OpenAI".

T&C yes, but not copyright. This is fully consistent with them opposing copyright and not opposing paywalls/api limitations.

If their model was leaked, you can be sure they’d claim copyright protection on it.

I wanted to say that they are to smart to expect dmca to protect them.

But then, I think that surely they would use copyright to block competition from using their model directly.

I was mostly thinking of large companies also creating their own AI, like Google, Microsoft, etc.

People generate all of the data going into the system and then the middle-men (OpenAI, Microsoft, Google, Big Tech middle-man of the week) reap a disproportinate centralized benefit. That causes a bigger problem than the so-called rent-seeking behavior of copyright holders in this case, as this has the net effect of leveraging human creativity, etc. to devalue it and continue the erosion of the middle class.

Bad things happen when you let middlemen get the upper hand, like the American health care system, or big finance disconnected from the real economy. I'll vote against the middleman every time in favor of the original value creator, because society goes down the toliet when middlemen win.

Yes, when I read "rent-seeking" i assume OP meant OpenAI.

Google search at least was just a link to content we wrote. OpenAI just steals it.

OP was obviously referring to the copyright holders whose data he feels so entitled to.

Yes. If you think about it, the individual is being subjected to a man in the middle attack, cleaving a creator from their creation via the use of consent agreements for providing a platform. Rent seeking.

What is the alternative though? I agree with the feelings and sentiments of the anti-ai people that want it to pay copyright, but I never hear any considerations for what comes next.

This is going to end up being the music industry all over again. It's going to be impossible for any individuals or small companies to get the rights needed, and instead were going to get massive content labels selling the rights, or only giant corporations being able to hop through all these new hoops.

We don't want a repeat of that as a society, creating yet another leeching middleman and horrible industry favoring only the incumbents.

I don’t see it ending like that. LLMs will just be taught not to emit copyrighted content verbatim. Whatever the courts end up deciding, they’ll be trained to stay just this side of legal. I’m certain it’s already being worked on.

Open models are a thing. Rather than attacking the technology (which is great) with litigation to hurt a few bad actors, we should attack the capitalist rules that enables rent seeking middle man parasites to flourish.

The data is not the technology.

Why should 'big tech' corporations be allowed to use AI to remix/mash-up human-generated content all of a sudden when creative individuals have generally been prohibited from doing it for so long?

Wow, I didn't know creative individuals have been banned from remixing copywrited material in their own private works.

We must tell the millions of kids who doodle characters in their notebooks that this prohibited.

The title is wrong: this is about ChatGPT Plus, not GPT-4.

Specifically, the author is investigating (possible) changes in the system prompt and tools available to the model in the chat interface of ChatGPT Plus. That tells nothing about the model (GPT-4).

I thought ChatGTP Plus was the choice between GPT3.5 or GPT4… At least it is in mine.

The important and overlooked distinction is that the choice of underlying model in the product ChatGPT is not the same as calling gpt4 via the api.

Sending a prompt into one vs the other, the API sends through the model and back out, the product has censorship watchers and other unknowable bolt-ons.

ChatGPT plus lets you choose between 3.5 and 4 behind the same web client.

As a backend that ChatGPT is built on. There's layers of prompts, and other stuff on top, compared to the raw 3.5 or 4 models.

I find it a bit interesting that posts and comments that praise GPT's abilities often rank high. But those that critique its current capacity do not perform well and have people quickly assert that 3.5 must be being used. The reason I find this interesting is because criticism (as opposed to complaints) is a requisite step in improving systems. Shouldn't we, HN, be the first to hack away at and find weaknesses in these systems? Should we wait to communicate our findings until we find solutions or should we be open in our discussions so that we can facilitate collaboration? How can I accurately critique a flaw in GPT without some OAI fanboy confusing a critique with the system from saying the system is a useless piece of garbage? I've struggled with this latter one despite being open about how impressive I think these systems are and how I use them frequently but that I recognize that they frequently hallucinate and often in subtle ways. There's so much to discuss about these systems but I find it difficult due to existing priors.

The people who still care enough to discuss after this long are mostly the optimists, the sceptics mostly got bored and stopped discussing half a year ago since no significant advancement has happened since then.

Reading this, it really makes me wonder how content creators will adapt in the AI era. Even if LLMs can't reproduce their content word for word, I find myself going to ChatGPT more and Google less.

> I find myself going to ChatGPT more and Google less.

My use is a perfect example of many of these types of points made in the NYT Lawsuit [1].

[1] Great summary by Hoeg Law: https://www.youtube.com/watch?v=ETDBDZoRJJU

If I were to guess, this is a result of the NYT lawsuit.

gtp4 powered chatgtp, not gpt4

Why not limit AI training to scientific subjects and leave the arts to humans?

Because we're aiming for AGI

$$$$$

We wouldn't be humans if we did that.

Or artists, for that matter.

Or particularly good scientists, unless blindly (over)fitting models is what constitutes science.

It doesn't work like that, not with text. If you make a text model smart enough it will be able to generate whatever you want.

Maybe because machine art encourages humans to think more laterally? Have you seen how tired art has become? It's mostly a cynical scheme to make or launder money. AI can force artists to actually take risks to outdo the mediocrity of generic art.

Because art is less rigorous and therefore lower hanging fruit.

What else do you want to leave out? It is an endless list, and nobody out there will actually enforce it -- even if someone follows that voluntarily, another company is going to ignore it.

Reply on news.ycombinator.com