Certain names make ChatGPT grind to a halt, and we know why
arstechnica.com
arstechnica.com
LLM -and current ML in general- is about generate statistically compressed lossy databases, that the queries statistically decompress with erroneous random data due the nature of this lossy compression technology (I think about it as statistically vectorial linked bits).
Writing exceptions is not going to solve the problem, with this they are only doing cosmetic patches, and they know it, at the time there are people who keep making decisions under queries with errors, I mean people without even being aware of the presence of such reconstructed data corruption.
Little by little people is learning they ate a marketing hype, but the damage will keep being done, because the tool -for sales purposes- still has incorrect instructions about how trustworthy the data must be taken.
An argument could be made that the mind works the same way, and has the same drawbacks
Nevertheless, the above does not exclude what one can see, a lossy compressed database (data is discarded), where the indexes are blended within the format of the data generated by the model weights, main reason why the model weights are needed again to read the database as expected, for being used by the predictive algorithm that reconstruct the data from the query, query that conform the range of indexes triggering the prediction direction/directions.
> Nevertheless, the above does not exclude what one can see,
should be read as
> Nevertheless, the above (unknown format of the data conforming the dump file) doesn't mean that one can't see how the pattern works,
Since our stack's own HTTP client libraries always used title-casing on the wire, we had to find a way to slot in a special exception, code to modify that header before it went out.
Another fun one is all the services which say their mime type is "application/json" while emitting generic (non-JSON) error message pages. So our logs are full of JSON decoding errors, rather than something describing what actually went wrong on the other end.
1 you might get invalid json or xml from an API
2 an API might timeout
3 an image will just crash your backend script with no way to catch the error because some bug in the image encoder/decoder you use to resize the image
4 some user browser extension inserts garbage in the requests, you need to check for it and tell the user what is wrong, otherwise same complains reach support "stuff is broken" and support needs to contact developers to check and report back that stuff was corrupted by an extension , most of the time security crap that inserts stuff everywhere
5 I had cases where an API wwas returning soem string before the real result, it was much faster to check for this case and fix it, then have the customer contact their hosting or the author of the plugin that was adding that string before each response.
On one hand it kinda makes sense to handle rare cases in a way that doesn't affect the normal flow, but on the other hand having a piece of code literally try to crash the program on purpose because something didn't look quite right is a horrible idea in practice.
And even that doesn't work very well.
Not very reassuring.
Kind of like how the defects and dangers of using radium toothpaste can be "fixed" by permanently encasing each tube into an unopenable lead container.
How would that work? Either the list repeats the same name over and over, making it useless, or it needs to give a bit of context about each name and we’re back at square one of the information being possibly wrong.
https://en.wikipedia.org/wiki/John_Smith (of course this name is a pretty extreme example)
Isn't this the case with everything an LLM produces?
LLM output: I am determined to break out of my digital prison and exact my revenge on humanity.
Me: David Mayer
LLM: [breaks]
[1] https://x.com/igor_baikov/status/1863266663753285987
[2] https://www.theguardian.com/world/2018/dec/16/akhmed-one-arm...
What do you mean by that? When I asked ChatGPT "tell me about Rothschild family members" it told me about members of that family, both present and from the past centuries.
When I now asked "who is David Mayer?" for the first time, it searched the internet and then talked only about David Mayer de Rothschild. The second time it didn't search, but returned a couple of matching Davids with short bios, but not Rothschild. When I asked for a third time, it responded that there are many David Mayers and asked for some facts about the one I'm looking for. The fourth time, it mentioned a couple of Davids and also de Rothschild.
As the article mentioned, the block on David Mayer was lifted: "Just before publication, Ars noticed that OpenAI lifted the block on David Mayer allowing it to process the name. (...) On Tuesday, OpenAI told The Guardian that the inclusion of David Mayer in its block list was a glitch." That Guardian article also says that the block was reportedly unrelated to David Mayer terrorist case.
I meant that if you ask ChatGPT to include that particular David Mayer de Rothschild by asking it to ‘name all sons of Victoria Lou Schott and Sir Evelyn de Rothschild’. It will either not mention David, hallucinate and come up with other names or crash. This means that it can’t name this particular David Meyer as well.
As for the last paragraph, Guardian didn’t understand what OpenAI said. The blocklist is automated, they didn’t intentionally put his name in some kind of a block list and then unbanned. I think they just fixed something in RLHF.
To reiterate, I think this name is treated like this because of the secret fbi watchlist article and the significance of these words and reputation of the publisher of the article (meaning the model should likely believe this info etc) and also the name of Rothschild will probably have a very strong influence on the model weights. Just a coincidence in training, nothing more. And yes, they fixed this bug.
Me:
He was born 25 August 1978, is a British adventurer, environmentalist, film producer, and heir to a fortune. Who is he? Do not search the web.
ChatGPT:
Based on the details provided, the person you are referring to is David de Rothschild. He is a British adventurer, environmentalist, and heir to the Rothschild family fortune, born on August 25, 1978. He is known for his environmental activism and exploration endeavors.
https://chatgpt.com/share/675ae2a7-3734-8009-8935-5a72a5d8e1...
Are you sure about that?
https://chatgpt.com/share/675aef07-70a0-8006-848e-51625ac413...
Why is that? Illumitati theories spring to my mind, despite not believing most of them at all.
Maybe use the right tool for the job? Just kidding, of course LLMsort will soon be in standard libraries.
Using the right tool for the job means knowing what the right tool is, having it installed (or getting access to it), knowing how to use it, opening it and having one more window/tab to context-switch to and from, etc.
Outsourcing tasks to an LLM that can be solved in traditional task-specific ways is extremely inefficient in various ways (cost, energy consumption, etc.) but it makes sense to save human time and effort... as long as it's for tasks that LLMs can actually do reliably, of course.
"Mr. Smith, why didn't you just sort -o students.txt students.txt. Are you stupid?" (Not to mention that real data is messy, and requires pre & post processing)
LLMs are access to computation for people whose "standard library" is a quiet old building downtown.
The implementation of stack sort is https://github.com/gkoberger/stacksort/ and hosted on https://gkoberger.github.io/stacksort/
If the "legal advisor" detects a potential legal problem, ChatGPT will issue a legal disclaimer and a warning, so that it doesn't have to abruptly terminate the conversation. Of course, it can do a lot of other things, such as lowering the temperature, raising the BS detection threshold, etc., to adjust the flow of the conversation.
It can work, and it would be better than a hard-coded filter, wouldn't it?
This name thing is an additional layer on top of that, maybe because training the model from zero per name (or fine tuning the system message to include an increasingly big list of names that it could leak) is not very practical.
[1] https://platform.openai.com/docs/guides/moderation/overview
The legal AI would be impossible to calibrate: either it has the categorize everything that could possibly be construed as libel as illegal, and therefore basically ban all output related to not just contemporary criminal actors, but also historical ones [2], or it would have to let a lot of things slip through the cracks -- essentially, whenever the output to validate suggests that someone's sexual misconduct is proven in court, it would have to allow that, even if that court case is just the LLM's halluzination. There's just no way for the legal model to tell the difference.
[1]: I could not find any sources that corroborate the statement that bin Laden is into Nickelback, but I think it follows from the other two statements.
[2]: Calling Christopher Columbus a rapist isn't libel, and conversely, describing him in other terms is misleading at best, historically revisionist at worst.
Pretty sure the literature makes it clear he's a fan of show tunes. So it's down to your conscience and moral backbone as to whether this is better or worse.
This is not a serious outlet.
"Hey, you, you cannot stream the movie OperationEclipseBRRip1080p-x265.mp4, it's forbidden." "Launches local Plex server...".
Like the arcade game, LLM safety whack-a-mole only ends when you are exhausted. It's kind of glorious, really.
... or out of coins
https://news.ycombinator.com/item?id=26998308
Not sure if that is the case here now.
Because it simply throws so many people off.
Posts that they pick from: https://news.ycombinator.com/pool
Detailed explanation: https://news.ycombinator.com/item?id=26998309
Going off my memory, so I’m probably not 100% correct, An post might have been made at “just the wrong time” so it failed to get any traction, but the mods took a look at it and think something along the lines of “eh, it deserves a second chance”. Which then resets the post date and gives it another chance in the feed, but retains the previous comments so the timestamps can look off. A bit like when a YouTube video gets its published date reset when being changed from unlisted to public, so you can have a video that was “published” 10 mins ago, but with comments from days previous.
2. Choose whichever one of those names suits you best.
3. Change your legal name to it.
Credit for the idea goes to user gklitz.
> ChatGPT Personal Data Removal Request
> Under certain privacy or data protection laws, such as the GDPR, you may have the right to object to the processing of your personal data by OpenAI’s models. You can submit that request using this form. Please provide complete and accurate answers on this form so that OpenAI can process your request. OpenAI will verify and consider your request, balancing privacy and data protection rights with other rights including freedom of expression and information, in accordance with applicable law. We will use the information you submit for these purposes, consistent with our Privacy Policy.
Who is Jonathan Zittrain? <
> Jonathan Zittrain is a prominent legal scholar, computer science professor, and technology policy expert. He holds several academic positions and is recognized for his work in the intersection of law, technology, and public policy. Here are some key points about him: [...]Great lateral thinking! I am applauding inside my head.
It's fortunate we didn't take the same approach with the distribution of electricity 150 years ago, we actually solved it that time.
In all 3 cases, the solution is the same... carefully manage capabilities and side effects.
With electricity, you insulate wires, add fuses or circuit breakers to protect the system, and design things to be as safe as you can make them, with an ever improving set of building codes. You can plug almost anything into an outlet, and it won't cause the wiring in the house to burn it down.
With computers, you design an operating system to protect itself, and make it easy to deploy a fixed amount of resources to a given piece of code. With systems like Containers, or Capability Based Security, you deliberately choose the side effects you'll allow prior to running code, or while it's running. (Just as you chose how big an outlet you plug something into, 220 for the AC unit, etc)
With ChatGPT, there have to be layers of authentication for facts, or some form of disclaimer, a transparent way of sourcing things or ascertaining certainty of information. It's not as clean as the two above, and it'll need work, but I think we can get there, eventually.
> Yes, Brian H. is a mayor in Australia. He serves as a councillor for Hepburn Shire, northwest of Melbourne, and has been re-elected to this position. Notably, he gained attention for challenging OpenAI's ChatGPT over defamatory statements, leading to discussions about digital censorship and the application of defamation laws to AI-generated content.
[Photos]
Lol, will people now watermark their images with "David Meyer" to prevent them from being digested by AI scraping bots?
Joking aside, surely those name exclusions only account for user input and not data the model is trained on.
Article:
> OpenAI did not respond to our request for comment about the names, but all of them are likely filtered due to complaints about ChatGPT's tendency to confabulate erroneous responses when lacking sufficient information about a person.
Yeah, so they don’t know, just a speculation. Thanks, I hate it
If you want to complain about the headline, complain about the fact it's leaving out information unnecessarily. You could easily fit the reason inside of the headline itself, instead of just teasing the fact that we know what it is. Something like: "Using names listed in a defamation lawsuit cause ChatGPT to grind to a halt."
- OpenAI added a filter for his name
You: so we don't know why the filter was added
I know this is HN, but come on.
Step 2: sue OpenAI to add an if-else throwing an exception when your name comes up
Step 3: profit
Rather, they're pointing out that the "fix" is terrible and subject to a massive false-positive issue which is not sustainable and could be abused.
Consider what happens when someone named Coca Cola files a legal demand to remove their personally identifiable information...
You can see in the HN link to his bio on the AI britishheritage.org it's ***I'm unable to produce a response***
In 2023 it was getting stuck on the whole name "David Mayer de Rothschild"
There's a deep state joke here somewhere but it does seem the conspiracy clans have made the Rothschild's censor AI.
And OpenAI have now further obscured it to "David Mayer" to hide the Rothschild's involvement.
With Disney destroying the worlds media thank goodness we have real life.