It used to be that people recognized that one of the unintended, unfortunate limitations of natural-language interfaces was that it was hard for users to discover their range of capabilities. Now we're designing that flaw into them on purpose.
Every single computer systems I've designed, built and shipped into production has limits programmed into them or else they will be abused or work incorrectly, why should an LLM be any different?
We still need to program the computers, it's just that now we're trying to (somewhat unsuccessfully , see jailbreaks) using the English language to program computers.
Of course programs need to be limited, but being able to discover what those limit are is also needed to be an effective user.
Like that is a pretty good translation for “don’t tell people you accept SSL connections, and if they ask you the usual way say you don’t.”
It gets even tricker in this case because you're exploring new territory. What OAI chooses to do here can and likely will influence laws in the near future.
At the same time, security by obscurity does not work in the long run. In fact, the existence of this repo of reverse engineered prompts maybe means that secrecy is impossible.
Even worse, we won't necessarily know when the information leaks out, so we don't even know what compromises are out in the wild.
This is commonly known as "security through obscurity"[1] and has been shown to be ineffective most of the time.
[1]: https://en.wikipedia.org/wiki/Security_through_obscurity
I don't rely on obscurity for 'security', i just don't think implementation details are required for most users so I don't publish them.
I'm very familiar with security through obscurity, ultimately I like to think the systems I build are secure but I can't always be sure, so why give people a head start? Not publishing details gives me time improve security.
Security through obscurity might not be the best approach, but you should know it's fairly common. For example when I generate a link to a Google Doc and "only those with the link" can access the document, I think that's a form of seurity through obscurity. No one is going to guess the link in any practical time frame...
Obscurity is a layer, but cannot be the only one.
It should be required to reference specific rules or policies when effectively denying use of a service.
Even HN has 'gray' rules because dealing with assholes is difficult.
And who themselves are often lawyers.
"If you can't convince, confuse"