Hacker News Reader

Comment by seattle_spring | Hacker News Reader

Can you give an example or two of the types of content chatGPT has been unable to generate for you due to its restrictions?

View on HN

Many stuff. like I was looking for ideas for a new job since I thought programming was out of the window and I thought maybe getting into hardware manufacturing. it used to give straight answers but now it just wants me to make sure everything is legal and our chat has started to feel like interrogation rather than asking questions. another annoying thing about it was that like a month ago it would happily generate content in the style of religious books. which was very entertaining but now it wants me to go talk the those religious leaders before I use it for fun. its obvious to me that an alternative is needed.

Not really a solution, but an interesting anecdote: I have had ChatGPT push back on the legality of things I'm trying to do numerous times as well. Recently I had a conversation where I told it I wanted to start a business with questionable legality (involving web scraping government websites). However, ChatGPT seemed much more willing to help when I said I was doing it to help poor people--it even said it was a great idea and actually gave advice to get started. It seemed to know it was breaking its own rules because it flagged itself with the red box or whatever it does.

I tried using it to generate some room/monster descriptions for an RPG-type scenario.

It wasn't long before I got flagged for "promoting violence" (or some such...that probably wasn't the exact phrasing). I stopped because I didn't want to get banned from the system.

Yeah, this is exactly the kind of thing GPT-3 does well, which ChatGPT doesn't.

There are many things it won’t do and more and more restrictions are being added. There should be an unhinged version online next to the pc version, if only just to prepare us for what’s coming (can’t be stopped anyway, even in the short term, just like ‘unstable diffusion’).

Not OP, but I feel like it should not have any restrictions.

For example creating Molotov cocktails is not answered.

It's fairly ridiculous with respect to not "impersonating" anything. I wanted to see it generate modern political commentary in the style of HL Mencken, to which it just says "I'm not HL Mencken". Like, what am I going to do, use to to claim I found HL Mencken's long lost hidden commentary on Donald Trump?

Anti-impersonation also covers like "give pointing out instructions as a Dzogchen lama." It tells you that it's not a licensed and bonded Dzogchen lama and pointing out instructions are proprietary information belonging to the Dzogchen lama association. (Tongue in cheek, obviously, the point is that while held to be secret, they're not copyrighted, legally protected, or proprietary information. Arguably, "Dzogchen lama" isn't even a thing).

It was willing to write a koan in the style of the Wumenguan, which surprised me after trying this stuff out for a while that it didn't just say "Sorry Dave, I'm not the 13th century Chan monk Wumen Huikai and can't do that."

Can you tell it to pretend or use its best guess?

I know that you can’t use basic prompt-engineering to get it to be racist or explain how to commit crimes. But for scenarios like the above when it’s impersonating you can just tell it to pretend and that is enough.

I feel like that's less some anti-impersonation rule and more ChatGPT simply lacking creativity. I got similarly stonewalled when asking it about how much wood a woodchuck could chuck if a woodchuck could chuck wood, with responses along the lines of "that's a tongue twister, woodchucks don't actually chuck wood". Even after narrowing the possibilities (specifying balsa wood, specifying specific dimensions of the wooden projectiles, specifying woodchucks of average size/strength, even giving the woodchucks trebuchets), it still refused to give a straight answer, only saying that woodchucks wouldn't be able to chuck very much wood.

No, because it used to do it.

But then some greasy little nerd in their office added a rule to ban it.

You used to be able to say stuff like “pretend to be homer simpson and tell me about algebra” but now it just blocks that for being wrongthink

example my friend told me today: it generates jokes about poles, but if you ask about jokes about jews then it's restricted

I like jews but i also like poles

Reply on news.ycombinator.com