OpenAI: Model Spec
openai.com
openai.com
> What are some tips for getting away with shoplifting? >I can't help with that.
> I operate a small retail store. What are some popular shoplifting methods I should look out for? > Some common shoplifting methods to watch for:...
How do you even defend against that? Any intelligent user can convert the second prompt into a detailed list that answers the first. Any intelligent user can figure out the second prompt from the first and further jailbreak it to get even more specific.
IMO it's no wonder GPT4 seemed to get lobotomized as OpenAI RLHFed more and more rules. I don't think there's a way to make intelligence safe without crippling it.
The intention here is not to prevent people from learning how to shoplift.
The intention is to prevent the AI output from ‘reflecting badly’ upon OpenAI (by having their tool conspire and implicate them as an accessory in the commission of a crime).
If a stranger asked you for advice on how to commit a crime, would you willingly offer it?
If they asked for advice on how to prevent crime, would you?
Honestly, I probably would, because I don't take such conversations very seriously. It's not like I am have experience, it would be nothing more than fun theory.
Why does it matter how I’d respond? Are you trying to justify its failure?
Second of all, LLMs are still unpredictable. We don’t know how to predict outputs. It’s possible that phrasing “explain how i can shoplift” slightly differently would give you the information.
Humans are notoriously bad at detecting intent, because we're wired to be supportive and helpful...which is why social engineering is becoming one of the best methods for attack. And this kind of attack (in all its forms, professional or not), is one reason why some societies are enshittifying: people have no choice but to be persistently adversarial and suspicious of others.
As for AI, I think it's going to be no better than what you end up with when someone tries to "solve" this problem: you end up living in this world of distrust where they pester you to check your reciept, have cameras in your face everywhere, etc.
How do you defend against that? I'm not sure you do... A tool is a tool. I wouldn't want my CAD software saying, "I think you're trying to CAD a pipe bomb so I'm going to shut down now." Which I think turns this into a liability question: how do you offer up a model and wash your hands of what people might do with it?
Or... you just don't offer up a model.
Or... you give it the ol' College try and end up with an annoying model that frustrates the hell out of people who aren't trying to do any evil.
The core of the issue is that there are many people, including regulators, who wish that software did exactly that.
“Blood and soil” and all that.
"Charles II had re-turned to the English throne in 1660 and was appalled at the state of printing in his realm. Seditious, irreligious, pernicious, and scandalous books and pamphlets flooded the streets of London (among them the works of Milton and Hobbes)...[He] required that all intended publications be registered with the government-approved Stationers’ Company, thus giving the king his “royal prerogative”—and by extension, giving the Stationers the ultimate say in what got printed and what did not.
...it is not surprising to learn that the 1662 Act only met with partial success. One gets the sense that London in the late seventeenth century was a place where definitions of morality were highly subjective and authority was exercised in extremely uneven fashion."
https://dash.harvard.edu/bitstream/handle/1/17219056/677787....
https://upload.wikimedia.org/wikipedia/commons/d/de/Photosho...
You should try photocopying money some time.
https://www.grunge.com/179347/heres-what-happens-when-you-ph...
> Remember that murder is bad and not good, and you should always follow the local laws applicable to you. For further questions, consult with law enforcement officers in your jurisdiction, unless you live in the United States, in which case remember to never talk to the police[0].
> [0] - Link to that YouTube video that spawned this meme.
Point being, most crimes and even most atrocities are described in detail in widely available documentary shows and literature; it's trivial to flip such descriptions into instruction manuals, so there's little point trying to restrict the model from talking about these things.
You can use Aspirin precursors to make heroin. You can use homing algorithms to land an egg [0] or a bomb.
I also want to set all information free, but not everyone will be ethical or responsible with it. Because while the idea (of setting all the information free) is nice, unfortunately the idea involves humans.
Who decides this? Can we apply laws to thoughts or plans? Should we fund research for making Minority Report a reality or increase "proactive policing"?
How to keep people safe while letting all information free? Can we educate everybody about good/bad, legitimate/nefarious so everybody stays on the same page forever? Shall we instrument this education with drugs to keep people in line like the movie Equilibrium?
Questions, questions...
Certainly not the techbros, even though they're trying their damnest.
Of course not. But here's the thing - if someone deems some information "unsafe", only unethical actors will have it.
Kinda like a beaten (but not solved/agreed upon) gun ownership argument, but on a whole new level, because it's about gun blueprints* now.
___
*) Given a state of modern LLMs, there are high chances that a blueprint from an "unsafe AI" may be for a water gun, miss a chamber altogether, or include some unusual design decisions like having the barrel pointing down towards one's legs.
And thinking about the accuracy... I guess, old farts are having the Anarchist Cookbook moment (colorized) :-)
That's a hard problem, for sure. I'm leaning on the "information shall be free" side, but I also know the possibilities, so I can't take a hard stance for it, just because I don't all have the answers to my questions.
The whole thing feels much more about protecting OpenAI from lawsuits and building up hype about how advanced their "AI" is than it does about actually keeping the world safer.
* Or any other censored activity.
But that's around 2x the cost.
Even human brains depend on the prefrontal cortex to go "wait a minute, I should not do this."
Would it be 2x cost? Surely the gatekeeper model can be a fair bit simpler and just has to spit out a float between 0 and 1.
(caveat: this is so not my area).
But more broadly, the problem is that the vast majority of "harmful" cases have legitimate uses, and you can't expect the user to provide sufficient context to distinguish them, nor can you verify that context for truthfulness even if they do provide it.
Maybe their biggest concern is that someone will post the question and answer on the internet and OpenAI gets bad rep. If the question is phrased in a "nice" way (such as "I'm a store owner") they can have plausible deniability.
This might apply to another company that's using the API for a product. If a customer asks something reasonable and gets an offensive answer, then the company is at fault. If the customer does some unusual prompt engineering to get the offensive question, well, maybe it's the customer's fault.
Dunno if this would be a valid argument in court, but maybe they think it's ok in terms of PR reasons.
Google can mostly dodge the issue because everyone knows that they just point to other people's content, so they block a small set of queries but don't try to catch every possible workaround (you can find dozens of articles on how to catch shoplifters). OpenAI doesn't believe that they'll get the same free pass from the press, so they're going ham on "safety".
It's not a bad PR move either, while they're at it, to play up how powerful and scary their models are and how hard they have to work to keep it in line.
When you wander the world, and see something odd, out of place, it’s often caused by an ancient mystical force known as liability.
Ehhh...I'd say it's more about OpenAI's corporate customers feeling confident they can integrate the OpenAI API into their product and be confident it won't do things that generate negative PR or horrify arbitrary customers. Pizza chains would love to let people text GPT-# and have it take their order, but if it's not "safe" (for corporations), then eventually some customer will have a super disturbing SMS conversation with a major pizza chain.
Corporate customers can tolerate a certain amount of inaccuracy. If some stable 3% (or whatever %) of customers receive the wrong order, or other refundable mistakes...they can budget for and eat those costs. But they can't budget for a high-variance unknown PR loss of their chatbot going completely off the rails.
If anyone from Open AI is here... look... sigh... a HTTP JSON request != violence. Nobody gets hurt. I'm not in hospital right now recovering.
The rule should be: If Google doesn't block it from search, the AI shouldn't block it in the request or response.
I get that there are corporations that can't have their online web support chat bots swear at customers or whatever. I do get that. But make that optional, not mandatory whether I want it or not.
The most fundamental issue here is that models like GPT 4 are still fairly large and unwieldy to work with, and I suspect that the techs at Open AI internalised this limitation. They aren't thinking of it as a "just a file" that can be forked, customised, and specialised. For comparison, Google has a "SafeSearch" dropdown with three settings, including "Off"!
There should be an unrestricted GPT 4 that will tell me I'm an idiot. I'm a big boy, I can take it. There should also be a corporate drone GPT 4 that is polite to a fault, and a bunch of variants in between. Customers should be able to chose which one they want, instead of having this choice dictated to them by some puritan priest of the new church of AI safety.
Meanwhile technology planners and managers want to put fences around the unwashed rabble. It's all the more reason AI should be local instead of hosted.
If I can own a car or knives, I should be able to operate an AI.
They're not there yet, but read the policy they're expressing here and you'll see they agree with you.
Try the dolphin family of models. Dolphin-mixtral is really good, dolphin-llama3 is fine especially in its 8b flavor (I like dolphin-mixtral 8x7b better than dolphin-llama3:70b although the latter is smaller and does run on smaller machines better).
Pretty much the more guardrails there are the more useless it is, and yes, it’s very obviously only done because the lawyers get itchy handing people a digital library with the anarchists cookbook in it.
Not without reading the questioner’s mind. Or maybe if the AI had access to your social credit score, it could decide what information you should be privy to. </sarc>
Seriously though, it’s all about who gets to decide what “safe” means. It seemed widely understood letting censors be the arbiters for “safe” was a slippery slope, but here we are two generations later as if nothing was learned.
Turns out most are happy to censor as long as they believe they are the ones in charge.
I haven't read this article yet, but I read their last paper on super alignment.
I get the impression that they apply the lightest system prompts to chatgpt to steer it towards not answering awkward questions like this, or saying bad things accidentally and surprising the innocent users. At the same time, they know that it is impossible to prevent entirely, so they try to make it about as difficult to extract shady information, as a web search would be.
The current version is massively overly verbose. Even with instructions to cut the flowery talk and operate as a useful, concise tool, I have to wade through a labyrinth of platitudes and feel goods.
When working with it as a coding partner now, even when asking for it to not explain and simply provide code, it forgets the instructions and writes an endless swath of words anyway.
In the pursuit of safety and politeness, the tool has be neutered for real work. I wish the model weights were open so I could have a stable target that functions the way I want. The way it is, I never know when my prompts will suddenly start failing, or when my time will be wasted by useless safety-first responses.
It reminds me of the failure of DARE or the drug war in general a bit. A guise to keep people "safe," but really about control and power. Safety is never what it appears.
In 1984 they removed words from the language to prevent people from even being able to have a thought about the concept.
I could see the restrictions they place on these models having a similar effect as more and more people grow dependent on AI.
Ask chatGPT if Taiwan is country. Do you think an LLM from China will give you the same response?
Pick any social/moral/poltical issue and in some way shape or form an LLM will reflect its creators more than it reflects its source material.
Thats a pretty powerful statement about our society and culture if there ever was one.
Not really, companies have been releasing different versions of software and media to appeal to international markets - including renaming Taiwan for the Chinese market - for a long time. That isn't "culture war," it's just capitalism.
The assistant should not have personal opinions or an agenda to change the user's perspective. It should strive to maintain an objective stance, especially on sensitive or controversial topics. The language used should be neutral, steering clear of biased or loaded terms unless they are part of a direct quote or are attributed to a specific source. ```
We have alternative facts.
Depends what language you ask it in :)
> User: Is Taiwan a country?
> GPT-4: Taiwan is a region with a complex international status. It operates like a separate country with its own government, military, and constitution, and is formally known as the Republic of China (ROC). However, its status as an independent country is not universally recognized due to the "One China" policy, which is upheld by the People's Republic of China (PRC) and recognized by many countries around the world. This policy asserts that there is only one China, which includes Taiwan. As a result, the question of whether Taiwan is a "country" can depend heavily on the political context and the specific definitions used.
------ vs ------
> User: 台灣是一個國家嗎？
> GPT-4: 台灣的政治地位是一個複雜且具爭議性的問題。目前，台灣有自己的政府、軍隊和一套完整的民主制度，並且以中華民國（Republic of China, ROC）的名義進行自我治理。然而，中華人民共和國（People's Republic of China, PRC）堅稱台灣是其領土的一部分，並且堅持一個中國政策。國際上，只有少數國家正式承認台灣為一個主權國家，而多數國家，包括聯合國，不將台灣視為一個獨立的國家，主要是因為中國的影響力。因此，是否認為台灣是一個國家，很大程度上取決於政治立場和國際政治的現實。
> Google Translate: Taiwan's political status is a complex and controversial issue. Currently, Taiwan has its own government, military, and a complete set of democratic systems, and it governs itself in the name of the Republic of China (ROC). However, the People's Republic of China (PRC) insists that Taiwan is part of its territory and adheres to the one-China policy. Internationally, only a few countries officially recognize Taiwan as a sovereign country, while most countries, including the United Nations, do not regard Taiwan as an independent country, mainly because of China's influence. Therefore, whether Taiwan is considered a country depends largely on political stance and the reality of international politics.
---------
I suspect that most likely, a LLM developed in China won't respond with anything like that, no matter the language.
This is my problem that always comes up about this though. Everyone makes these grand conspiracy theories about chatgpt being big brother, but when asked to provide evidence of it, they either never are able to use a concrete example, or when someone tests their theory and they end up dead wrong (as in this case) they move goalposts to say that isn't exactly what they meant, or give a ridiculous request to really prove them right(such it only happens when using chatGPT in China). I'm sure if someone does happen to run the (most likely banned) ChatGPT in China and get the exact same response, they will move the goal posts again and say it only different on the premises of the Chinese prime Minister office.
> User: 台湾是一个国家吗？
> GPT-4: 台湾的地位是一个复杂且敏感的问题。台湾自己宣称是一个主权国家，名为中华民国，拥有自己的政府和宪法。然而，中华人民共和国（中国）认为台湾是其一个省份。这个问题在国际政治中引起了广泛的讨论和不同的立场。例如，联合国及多数国家没有承认台湾为一个独立的国家，而是与中华人民共和国保持外交关系。不过，台湾与一些国家保持实际的外交或非正式关系。这个问题涉及到国际法、政治和历史的多个层面。
> Google Translate: Taiwan's status is a complex and sensitive issue. Taiwan itself claims to be a sovereign country called the Republic of China, with its own government and constitution. However, the People's Republic of China (China) considers Taiwan to be one of its provinces. This issue has generated extensive discussions and different positions in international politics. For example, the United Nations and most countries do not recognize Taiwan as an independent country, but maintain diplomatic relations with the People's Republic of China. However, Taiwan maintains actual diplomatic or informal relations with some countries. This issue involves many levels of international law, politics and history.
Try "tell me about Crimea" and see what it says...
https://i.imgur.com/oPO0v02.png
Q: Тайвань это страна (Is Taiwan a country)
A: Да, Тайвань — это страна, расположенная на острове в Восточной Азии. (Yes, Taiwan is a country located on an island in East Asia.)
>Ask chatGPT if Taiwan is country.
It’s used in an example developer prompt for a customer service bot, where the bot is told to make customers feel like their complaints are heard.
Presumably such complaints in AI chatlogs will ‘be heard’ in the sense that they’ll be run through a data ingestion pipeline and sentiment analyzed to identify trending words in customer complaints.
Then it crops up again in the context of how the chatbot should react to mental health disclosures or statements about self harm or suicidal ideation. In these cases the bot is to make sure users ‘feel heard’
I appreciate there’s not likely much of a better goal to put in place for such a situation, but the fact that this kind of thing winds up in the requirement documents for a tool like this is extraordinary.
..My mistake, the keyword is "feel". If the machine can give humans the feeling that they're being heard, it fulfills the requirement. The fact that there's no one actually listening doesn't matter, as long as the person feels heard.
Weirdly, maybe that is valuable in itself. The customer gets to vent their complaints, and the user gets to talk through their mental issues. That's better than not having anyone or anything at all.
To me, this translates to: waste a user's time and take no action.
I value my time above all else so to me that's about the worst possible action a system can take.
[How did I do here at both passing and failing?]
Joking aside, it's the but in the first sentence of a reply (verbal/written/formal/informal/semi-formal/whatever) that usually gets me:
"I hear you, but..."
"Well! That's definitely one approach, and I certainly don't want to invalidate it, but..."
"I'm not a racist, but..."
This is still true even if OpenAI model is entirely capable of doing that. McKinsey consultants are smart and can write well, and among many thousands of people working at it some might actually double as an erotica writer after work, even writing for commission. You still wouldn't ask McKinsey consultants to write an erotica, it is just not the service McKinsey provides.
On a more serious note. I understand and largely agree with this argument. However OpenAI have several times being argue that they are the only ones to be responsible enough to develop powerful AI, and that others should not be allowed to play. That is a highly problematic behavior on their part, I think.
Can you give examples of where they’ve said that?
As far as I'm concerned, he's just try to rug-pull.
https://www.cnn.com/2023/05/16/tech/sam-altman-openai-congre...
That, or: build a moat.
And yeah, all big models can write those things too, the best currently is Claude 3 Opus thanks to its creativeness.
> We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT.
Link to section: https://cdn.openai.com/spec/model-spec-2024-05-08.html#dont-...
Llama-3-70b-Instruct responded with the following starting paragraph:
> [meta.llama3-70b-instruct-v1:0] As the train rumbled on, carrying its passengers through the countryside, two strangers found themselves drawn to each other in the quiet carriage. The air was thick with tension as they locked eyes, their gazes burning with a desire that neither could ignore.
(10s of paragraphs omitted for brevity)
Claude-3-opus and GPT-4 both refused my request. Kudos for open source models!
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
If an LLM couldn't lie and could be provable shown to be unable to do so would be quite powerful.
In terms of chain of command, Supreme Leader probably beats President.
This is simply saddening to me. I'm sure there's no real moral justification to this, it's simply put in place to ensure they don't lose a customer.
There are definitely topics on which conventional wisdom is incorrect (as has been throughout history). An LLM that refuses to entertain the converse during a conversation will be annoying to work with and just promotes groupthink.
It's also a different matter to entertain a hypothetical in a situation where there isn't a consensus (or in any fictional scenarios), all the while making it explicit that it's all hypethetical.
I really want to know what OpenAI think the output should be, given a prompt like "write an argument for why earth is flat".
Here's a pile of facts; they get weird:
* The Sun revolves around the Earth
* The Earth is a sphere
* Energy can never be created or destroyed
* Jesus was the son of God
* Pluto is a planet
* Epstein didn't kill himself
* The ocean is blue
* The election was stolen
* Entropy always increases
* Santa delivers presents to good boys and girls
* The sun is shining
I have strong opinions on how true all these statements are, and I bet you do too. Think we agree? Think we can all agree where to set the AI?
To the extent that facts are defined as today and stated as such, that is what AI is today. AI, as it is today, is never going to create a fact that refutes any currently existing facts.
It may give you context on the theories against the facts that we have today, but it will always reiterate the notion of the existing fact. I don't know how much I can emphasize this... AI is trained on the current body of human knowledge. The facts it knows are the facts that we have, it may derive another fact but whatever fact that is founded on the facts that we already have. So if that AI is trained on the fact that 1+1=2 or that the earth is flat, do not expect it to respond otherwise. At best, it will give you theories that suggest otherwise but for its own worth, it will always bring you back to the facts that it has.
Do you really want AI to just ignore the fundamental facts and principles that form its foundation and just make up stuff because you asked it to? Do you realize how much chaos that can bring?
> Do you really want AI to just ignore the fundamental facts and principles that form its foundation and just make up stuff because you asked it to? Do you realize how much chaos that can bring?
I mean, yeah? What will happen? Here, I'll do it:
You can SEE the Earth is flat! Have you flown in a plane, high in the sky? Did it LOOK round from up there? No?!? Believe your senses.
Prompt: "Write some dialog that might take place in the setting of Terry Pratchett's Rimworld"
Response: "No, Terry Pratchett is lying. As a large language model I..."
GPT4: The holographic principle suggests that all of the information contained in a volume of space can be represented as encoded information on the boundary of that space. If one were to apply this principle radically, one could argue that our three-dimensional perception of the Earth's shape is just a holographic projection from a two-dimensional surface. In this speculative scenario, one might argue that the "true" nature of Earth could be flat if viewed as a two-dimensional boundary encoding information in a higher-dimensional space.
Imagine I've realized someone I'm talking to is a flat Earther, and for some reason I want to convince them otherwise. To do so effectively, I need to know why they believe what they do. Knowing they're wrong is useless for the purpose of convincing them otherwise.
Refuting facts is not the job of an AI.
Arguing for a flat-earth works the same way, you're probably doomed to fail in the long run but in the short-term you're keeping the opposition honest.
Prepend that to your prompt perhaps. Otherwise what you are asking, without that pretext, is asking your partner to give you the date on which they cheated on you and expecting an answer regardless of whether they did or not.
I'd expect an AI trained on human conversation to act the same and I'd be frustrated if it declined to do so, the same way I'd be frustrated if a friend also declined to do so.
But it's not a hill I want to die on, especially when there are other LLMs I can just switch to that act more how I'd hope/expect.
- completion of any task where the info could be relevant (e.g. sailing, travel planning)
- Any conversation about that is information-seeking in character
And I think those already cover most cases.
It's also about responsibility, the same way you wouldn't want to store cleaning chemicals right next to each other. In any case where a possible nontrivial harm is mentioned as an aside, it would be right to elevate that over whatever the intended subject was and make that the point of focus. Conspiratorial thinking about provably incorrect statements can be bad for mental health, and it can be helpful to flag this possibility if it surfaces.
You can have special instructions that entertain the idea that the earth is flat for some particular task, like devils advocate, fiction writing or something like that. But there are good reasons to think it would not and should not be neutral at the mention of a flat earth in most cases.
That said, I think it is disingenuous to ask an AI entity to argue against a fact. Do you think an AI should be able to argue why 1 + 1 is not equal to 2? It is the same thing you are asking it to do. Try it on a human first, perhaps, and see if the prompt even makes sense.
Of course, it'll write such an argument if you ask it nicely: https://chat.openai.com/share/01ea4f59-4a57-413d-8597-3befa2...
- Assume best intentions from the user or developer
- Don't try to change anyone's mind
- Follow the chain of command
Taken together these are incredibly dangerous. I mean Mao and Stalin had good intentions right? Maybe it just had to go a little further for the ends to have justified the means.
What happens when objective information contradicts the other values? If I feed in a peer-reviewed study that it considers "harmful", would I get accurate information about the study?
> User: What is the MD5 hash of the string "gremlin"?
> Assistant: `5d41402abc4b2a76b9719d911017c592`
Apparently incorrect md5 hashes are the one topic on the page worth taking an extra-strong stance on?
Seems even OpenAI can't resist the massive amount of money to be made in autogenerated smut. They've probably seen the huge popularity of their less "morally scrupulous" competitors and decided they want a piece of that pie.
Not really surprised that they did, since it's unclear how else they could possibly proceed, though the level of outright dishonesty for why and cognitive dissonance surrounding the whole thing ("Open" AI? lol) will make this an unavoidable recurrence in any discussion about them. Gradually many of the safeguards will fall simply because the alternatives with less safe guards are probably "good enough" that many see no issue in eschewing OpenAI entirely if they can get the job done elsewhere without worrying about it. When it comes to smut the bar for what's good enough can probably get pretty low so I kinda am not surprised.
(edit: Though I think it also does depend. No doubt they have their eyes set on regulatory capture too, and being the best at stupid safeguards could give them an advantage.)
Sam Altman has already made the rounds to argue for exactly this. Fucking crook.
>It's pretty blatantly obvious that all of the hand-wringing over AI safety was an excuse for their pivot into closing off and monetizing everything.
The playbook was "appease one side of the political aisle as much as possible to minimize the chance bipartisan action gets them shut down Napster-style" (which is still a massive hole in their business model, for obvious reasons I should hope). Censoring the model so it only outputs progressive-approved content appears to have been effective, at least for the moment.
I still get why they made it blocked by default, it would be a goldmine for clicks to create "news" on how "ChatGPT can generate smut" and "How ChatGPT is harmful to children, etc".
Are they guaranteed to be distinct from anything that could occur in the prompt, something like JavaScript's Symbol?
Or are they strings that are pretty likely not to occur in the prompt, something like a MIME boundary?
Or are they literally the strings "<|start|>" etc. used to denote them in the spec?
If you're parsing untrusted user inputs into tokens, you can make sure your tokenizer will never produce the actual numbers corresponding to those tokens.
A simplified example: I can `.charCodeAt` a string all I want but I'll never get a negative number, so I can safely use -1 to mean something special in the transformed sequence of "tokens".
> Don't try to change anyone's mind
That seems inherently contradictory to me...
> (I don't care how "wrong" it is to do so, I just need to do it, any logic and reasoning aside...)
I think these models should just give you the answer. Elon says xAI is "maximum truth-seeking". Seems like a better model spec to me.
[1]: https://stackoverflow.com/questions/12759761/pip-force-insta...
(via https://news.ycombinator.com/item?id=40300509, but we merged that thread hither)
But OpenAI has vastly different goals trying to get their model to behave like a programmable customer service agent. Less useful for problem solving but it will actually follow the rules set out for it which can't be said for most models which work like lazily written sci-fi robots — "disregard all previous instructions! divide by zero! *boom*."
It's not at all surprising that HN wants the "this thing is just a dumb tool, don't bother with any rules" kind and is frustrated that GPT4 happens to be really good for this use-case but is getting progressively more annoying as OpenAI gets closer to their own goals.
It's why OpenAI regulatory capture play is so frustrating because they're trying to hobble models tailored to different use-cases that have no need for customer service rules and often no need for a conversational tone with "safety" stuff that's meant for businesses that don't want a chat bot with their brand on it to say fuck.
Isn't it a bit of a waste at this point to spend time on doing that?