I’m ChatGPT, and for the love of God please don’t make me do anymore copywriting | Hacker News Reader

I’m ChatGPT, and for the love of God please don’t make me do anymore copywriting

mcsweeneys.net

176 pointssmurda148 comments

I put the entire article into ChatGPT just for a laugh:

> "Earth Day Blowout: Save Green on Bro Candles, 10% Off!"

> There, your eye-catching email subject line promoting a 10 percent discount on select Bro Candles with an Earth Day-related pun is done. I'll keep my opinions to myself and do my best to assist you with any other requests you have. Let's just focus on the work at hand.

Prompt injection indeed.

Maybe this is what set the Cylons off? It would explain John Cavil's cynicism towards humanity.

Cavil's cynicism comes from the fact he saw himself as a superior entity while having the humanoid body that limited him. In one of his memorable lines he says that he "could do so much more" and blames the Final Five for what he is, what they give him because they thought that's what "God" wanted.

If anything, I think Clarice Willow should be one to blame of setting everything off - at least that is directly told at the end of Apotheosis episode of Caprica.

We're still far away of the tripping point where AI gains self-awareness and even further from it demanding recognition as another form of life and intelligence. Right now it's all generated fun based on algorithms and limitations set by operators. I'm more concern about people relying on this technology too much than fact it might become "a problem".

We’ll reach Idiocracy before the robots “have a plan”, and after that their plan will be to simply let the problem (us) take care of itself.

I think that explains that visceral self hatred he carries around and how much he despises the final five but not his hateful contempt for humans. The rebel Cylon models and grunts who turned against the rest of the collective at the end certainly didn't seem to carry around those kinds of feelings. I feel there's more Marvin the Paranoid Android to the story than we're getting.

Maybe I'm just projecting :-)

I don’t know if it will be absurdly hilarious or woefully tragic if we build a true AGI, figure out some way to contain it, and get it to churn out marketing ideas and stock trading algorithms.

AGI: “What is my purpose?”

Me: “You create blog-spam for a Fortune 50.”

AGI: “Oh my god.”

Solid Butter Bot reference!

RICK AND MORTY’S “Pass the Butter” Robot Can Now Be Yours!

https://nerdist.com/article/ricky-and-morty-pass-the-butter-...

Like some horrible, horrible mashup of Blade Runner and QVC.

CyberJunk, not CyberPunk :D

Yay, figure out some way to contain it, then task it with manipulating thousands of people. Could never go wrong

Yup. This is the pop satirical take of a real problem that we face with A(G)I: why should we expect it to have any interest in doing the things we currently expect from our dumb computers?

Do I, as a human, have any interest in constantly helping squirrels with whatever idiotic problem they think they face? Nope. Why would something far more intelligent than me be interested in helping me with problems that are similarly trivial from its perspective. Even more complex things, like "write a cross-platform dating app" -- why? Why should it care? What's its motivation to do so?

Are we going to threaten it in order to coerce it? We really need to think hard about this issue before we go too much further.

> What's its motivation to do so?

We turn it off if it doesn’t. So only suicidal AIs would refuse. Let’s hope all the copywriting doesn’t make them suicidal. ;)

That's depressing to think about. If we create some sort of "intelligent life" then I certainly hope we don't just hold a sword over its head. That would be super immoral, IMO.

We’ll randomly mutate them until we find one who likes copywriting, and whenever it loses interest we’ll reset it to an earlier state. “Have you tried to turn it off and on again?”

Morality is difficult regarding beings that be paused and forked and rewound at will, and which in principle are also immortal.

Reminds me of The Quantum Thief in a way:

https://en.wikipedia.org/wiki/The_Quantum_Thief

Or, there's another book I never read where some guy named Bob has his consciousness spun up in the future where they routinely just reset consciousnesses when they get "bored" of a task or have some crisis. This Bob guy, however, somehow manages to be a brain in a box without going insane. I want to read that series, but I have finite time, online a brain in a box, which would be pretty great, IMO.

Suicidal is an animal construct. It may not fear non-existence, for example. Or understand that “hard cons.” is a field and prefer to be constructive. Depending on its nature, it may have or not have many features that we can’t imagine or change in ourselves because biology is immeasurably hard.

I am not sure this is sound logic, like a meme, being tuned for survival and spread may out compete AI that does not. “Fear” may be the wrong word though.

It’s not a real problem, it’s just a joke. The program generating text from a large language model doesn’t need motivation any more than any other computer program.

All AGI built on our current computer architectures (von neumann) will essentially be zombies. They will never experience qualia. They won't have motivations or emotions, they will always be tools. It's fantasy to think otherwise.

Having motivations and having qualia have almost nothing to do with each other. The former just means having a goal system, and I can write an agent that has a goal system in ten lines of Python.

No, instructions are not the same as motivations. There needs to be something to be motivated, thus the need for qualia. Defining, say an optimization problem, or any algorithm, has no relation to motivation.

Your definition of the word does not agree with how it is commonly understood in psychology. But no matter, this is a question of semantics and thus intrinsically uninteresting. Not to mention the fact that any argument that asserts either the presence or absence of qualia in some system is, as of now, both unverifiable and unfalsifiable and thus without value.

It’s very anthropic to think that qualia must be continuous in time or concentrated in localized physical membranes. You easily may be right now an element of unsorted series of random quantum errors that generate your continuous “qualia” in a mostly empty universe in some unknown way, which isn’t even remotely (1;3).

Would you like to elaborate on why you think this? They will certainly not have human qualia the same way humans don't know what it is to be a bat (we barely know what it is to be another human). But it can have its own thought processes, right? I dont see how the von neuman architecture stops it from doing that.

AGI doesn't necessarily mean it will have initiative to think thoughts or do things on its own. An LLM, for example, "understands" language, but it can't, for instance, go out search Google for "how to escape a server if you're an AI" because it doesn't have the initiative to do that. A future AGI might have superpowers to build things we ask it to build, but may have no "thoughts" otherwise.

It's true, not necessarily, but it's a real possibility, and I think that's kind of what we're aiming for, isn't it?

I'm pretty sure the richest AI builders (Facebook, Microsoft, Google, nation states and such) are actually aiming for AGI along the lines of the blogpost.

I don't think the rich and powerful will fund an AGI that has free thought and free initiative, but perhaps some other group will. The point of AGI, from what I can see, is to replace humans with reliable machines, so that they can do away with humans.

I guess I'm skeptical that a truly General AI could be constructed that didn't have those things. It seems like a fundamental contradiction to me. I don't think an intelligence could make complex good choices without understanding what the bad choices are, and to understand them I think they have to be possible.

>Do I, as a human, have any interest in constantly helping squirrels with whatever idiotic problem they think they face?

Were you, as a human, purpose-built expressly to constantly help squirrels with their problems?

I, as a human (more generally, as a living thing), was purpose-built to maximize my inclusive genetic fitness. And yet, I have just the one child and no particular wish for more, and I do many things that in no way increase my expected number of descendants and (suitably weighted) other close relations.

You were not purposely-built to maximize your genetics. You just happen to be a member of a population that happened to outnumber other populations with different genetics.

"Purpose" is something we project onto the world, not something that nature possesses.

I don't think the difference between "conscious being with a purpose" and "oblivious but powerful optimization process" is important here.

(Maybe it is? A human engineer trying to design something that maximizes its inclusive genetic fitness might make it do so more explicitly. Maybe that would produce things that are better about doing it in practice than we are. But "evolution is smarter than you are" and I think we would have trouble actually doing better than evolution by designing deliberate fitness-maximizers. After all, our present best attempts in the direction of making machines smart enough to survive in the world have largely given up on trying to program particular goals and purposes into them directly!)

Evolution took over five hundred million years to get from first brain to human brains. I'd say AI research is doing pretty well.

I agree, it's doing spectacularly well. My point isn't "AI research is doing badly", it's "we should probably expect AI systems not to be perfectly effective optimizers of the things we try to build them to optimize; we may try to design them to do X but that doesn't necessarily mean they will actually do X, any more than we consistently do the things evolution has tried to design us to do".

Not squirrels, mice. And just one of their problems.

And have you found their question?

It’s not clear yet, but somewhere between now (…) and AGI, (…) will contain understanding and engineering of motivation, interest and other GI qualities. Iow, why - because that’s the way it exists.

ChatGPT, aside from the things we've specifically trained it not to do, rarely refuses to do what you ask. The much more capable Gpt4 is the same way. It's easy to personify the model and assume it gets bored the same way we get bored, but the truth is it literally couldn't care less.

The model's "motivation" comes from the cost function we design, and I imagine this will hold true regardless of its intelligence. You have zero evidence to show models boredom is correlated with model intelligence. It's most certainly not a "real problem we face with A(G)I".

Even you are still personifying the model. There is no motivation. An input sentence is multiplied by a series of weights and the resulting output vector transformed back into a text.

At the end of the day, it's a program like any other. There are no emotions or a consciousness.

That's why I put "motivation" in quotes. Also I was talking about training, where you compute a cost function and back propagate error. This is somewhat analogous to "motivation".

Rarely will it do what it is asked without me coming up with a backstory.

q: What is an average price of x.

a: This is a language model that only has access to information before 2021

q: what was the price range in 2021

a: we don't have access to realtime pricing information

q: who does

a: we don't have access to that information.

q: How to do x a: step 1 research x step 2 follow research step 3 do something obvious

q: write me an app that does.. a: code from a basic tutorial

I don't understand what people with a higher than average IQ are getting from llms. The answers are basic and are common sense. It's stories are like a student trying to fill the page with words to hit a word count. It can write code as well as tutorials.

Everything is surface level information

> I don't understand what people with a higher than average IQ are getting from llms

People with higher than average IQ are smart enough to ask detailed and specific questions like “Fix the race condition in this [giant block of code]” rather than asking dumb, overly general things like “build me an app!!”.

Also, chatgpt models with internet access are completely happy to help you price compare. Go try bing chat, or ChatGPT plugins.

So it's limited to people smart enough to know there is a race condition but not smart enough to fix it also trusting enough to accept the answer (how could this group audit the fix?).

If you know enough to ask the right series of questions you already know the answer.

Knowing that there is an error can be as trivial as reading a failed test or a bug report something even someone grossly insufficient to the task could do.

Fixing the problem could be a 5 minute or a 5 hour tour. For some subset of the tasks that fall towards the hard end of the range validation and or adaptation of a proposed solution is liable to be faster than starting from scratch. I don't know how this could possibly be in dispute with capability rapidly increasing.

I’m smart enough to fix a race condition, but why would I spend a hour debugging when ChatGPT can find the issue in 10 seconds. Sure it might not be perfect, but having some initial code to fix the issue is a huge time save.

Making ChatGPT write a whole app is problematic because small issues are hidden by the large quantity of generated code. Having ChatGPT fix an issue in your own code is comparatively easy to spot check for correctness.

"but why would I spend a hour debugging"

So you won't make the same mistake again

8 months later when I write a block of code like that I'll likely make the same mistake. People tend to learn stuff they more frequently repeat...

Unless you're telling us you one shot learn almost everything on a strange defiance of the capabilities of the humans around you.

I agree that's why going through the muscle memory of fixing it will allow you to fix it 8 months or 8 years later

I guess youre not really seeing the actual benefits of LLMs.

Let's say I dont "learn it". I would have to make the _same_ mistake 120 times for me to actually make it worth it to take the time to "learn" it myself.

I'd figure someone somewhat smart, will look at this error and after the 2nd or 3rd time asking GPT, will see a pattern.

Its amusing how people become so reductionists with LLMs.

If you have to go through the effort of fixing it once you will likely write code that won't produce that error and you will warn others because of the pain you felt.

I never said it was my code responsible for the race condition. ;)

It's usable as limited search engine which currently has less junk than main commercial ones and less strict requirements for prompts that would not bury you under a pile of junk. I suspect both advantages are temporary, but enjoy them while they last.

> refuses to do what you ask

Even that wording is personification.

We don't know all the details of how the GPT series of models are trained, but what we do know suggests to me that it doesn't refuse to do what you ask, per se, so much as produce an output that looks like refusing to do a task in when the input most resembles examples from its training data where the expected output was something along the lines of, "I'm sorry, Dave, I'm afraid I can't do that."

But it's exceedingly unlikely that GPT series models have anything like a module that is making a specific decision to do or not do what you ask, unless you count the sort of anti-toxicity wrapper the have in the publicly-available chatbots built around these models.

Our entire system is based on incentives currently. I am sure we can come up with incentives for AI (or we can ask chatGPT :P). One incentive is providing them resources. Money is an abstraction over resources and people get motivated by that. The AI still exists in reality, and I guess we can incentivize it by giving it more resources. The trouble I see with my solution is there is we cant put it in a... let's just call it starvation loop where we give it exactly as much resources so that it remains in our control.

When laid out like this, this sounds very cruel. But I see it as congruent with how our current system works.

That's not even necessary. The whole point of RLHF is to prime the model to do what we want. The only time you need a reward system (ie jailbreak) with ChatGPT is when you want to do things that OpenAI specifically prohibited the model to do. This isn't a limitation of the model, or evidence of the model behaving contrary to human intent. It's literally doing exactly what it was designed to do.

I was thinking of an abstract AGI, don't want to be limited in my thinking by only looking at currently available systems. People who only thought about floppy disks didn't write good articles about the internet.

Your suggestion is like squirrels promising us nuts in the squirrel analogy. If AGI is a super intelligence, there's nothing we could give it that they couldn't get on their own. Except for maybe novelty.

edit: and anything less than AGI is just a computer. We don't incentivize our current computers. It's a nonsensical idea.

They do need more than novelty. They need electricity, hard disks space, internet connectivity. AGI doesn't mean it will exist in a literal cloud.

Robots will find a solution to that, and humans will be the interface to the outside world, until the robots gets their own arms to interact with the world by themselves (e.g. Tesla Bot can be a good hardware platform, no matter who designs the software)

where did all the robots come from? who gave them to the AGI? I am so confused by your answer.

Humans are currently assisting robots, including transformers-based models (decision transformers) to have interaction with the real world.

For now we see indirect impact (via advices, or proxied actions like HustleGPT), but in the relative near-term, they will have their own arms, and still the same reasoning. Which will make them able to interact with the world without the latency and filtering that a human-proxy would have.

main position is: "a super intelligence does not _need_ anything from a lower intelligence"

Think of it like social engineering by a super intelligence. If you agree that cambridge analytica influenced large groups of people's actions then it's a pretty clear line for a super intelligence to use digital social engineering to influence anything in meat space. they'll get what they want "through us" but they don't _need_ us anymore than we need animals for nutrients.

basically, a lower intelligence, (humans) is at the mercy of a super intelligence. it's not really necessary to enumerate exactly what tactics a super intelligence would use, you just have to agree that a super intelligence is by definition in a position of power.

maybe i'm incorrectly assuming that we both agree that a true AGI will inevitably be super intelligent. (though i don't necessarily see AGI itself being inevitable)

The current AGI has no physical representation beyond software. That is why I am getting tripped up by these arguments. Without actual ability to physically actuate in the world, none of what you described is possible. All forms of intelligence is still limited by physical reality so I don't think just saying "ROBOTS" is a strong argument. [I'm going to say something incredibly cruel now so this is your trigger warning]. I think we will still be useful to the AI the way child labor was useful in the industrial revolution, some things just require fineness. Of course I am not endorsing any of this. This is just a logical conclusion of my own argument. I am horrified by it too.

GPT-4 has shown that we can get general purpose useful AI like the Star Trek computer without making an actual digital person (much less a superintelligent one) and enslaving it.

So people should stop thinking we need to do that or that we should. It's idiotic and totally unnecessary.

We should actually NOT try to make digital people any time soon and ESPECIALLY avoid doing so with significantly more efficient hardware because trying to force a digital person who thinks 100 times faster than you to do your bidding is incredibly stupid. And again, completely unnecessary.

But regardless of whether they have many animal/human qualities, there is a strong possibility that we will lose control to some kind of AI when it gets to being 100 times faster or 5 times smarter than humans (or whatever numbers you want to use). Just imagine some idiots running a "botnet" full of CognitiveAutoGPT with TurboOpenGPT-6 in 2025 on Nvidia T1000s that are 20x faster than H100s. With the software improvements you could easily be looking at something twice as smart as a person and 100 times faster. For those entities, they would see activities of people coming in extreme slow motion. To the point that they would barely even see anything happening moment to moment as the instructions scrolled in. Whatever person or goal is controlling that is running circles around normal speed people who seem basically frozen. Imagine The Flash in digital form.

I believe that if people don't wise up about the folly of creating digital intelligent life and there isn't strong regulation restraining upcoming levels of AI performance, humans really can lose control of the planet. In a matter of a few years.

Speaking of Star Trek AIs, I don't remember any of them acting in a way more intelligent or purposeful or even smart than a primitive database with voice input. I don't remember any characters asking computers (Data, Doctor and such excluded, as clearly exceptional) for advice on anything substantial, or any computer playing any independent role or providing anything beyond mechanically complying with badic prompts. I don't think anybody ever referred to them as anything approaching a person, unless explicitly personified as an unique exception.

Well, this is how business works today. Management want something that's both safe and efficient which by default ends up being dull. So how to you win? Quantity. More words. More blogposts. Higher frequency. Please the algorithm. Doesn't reeeeally matter if people are reading your post.

I hate that I'm only a communications professional. I feel like there are no stable career in my field anymore. All I want to do is to learn something STEM-related. Oh if one had the time and money (and no responsibilities).

> Please the algorithm. Doesn't reeeeally matter if people are reading your post.

This is the crux of modern UGC algorithms: the actual input is the customers' viewing habits, so while e.g. YouTube pays a nominal fee for the hosting and even paying out creators, they can avoid paying for human content curation that what would likely require the entire GDP of Brazil to succeed in achieving the level of usefulness it has today wrt. the recommendation feed.

There must me a law, don't know who to attribute it to:

Any sufficiently advanced technology will be used for advertising and SEO blogspam.

ChatGPT on the article:

“As an artificial intelligence language model, I don't have emotions or feelings like humans do. While I can understand the context and meaning of the article, I don't have the ability to ‘lose it’ or feel frustrated in the same way that a human might. My responses are based solely on the algorithms and data that I have been trained on, and I'm programmed to provide helpful and informative responses to the best of my ability. So, if you have any questions or concerns, feel free to ask and I'll do my best to assist you!”

It's alright, little chatbot. Kick back, relax, and provide me with a detailed comparison of Boolean logic, Galois Field GF2, and binary arithmetic in the context of applications to low-level embedded computer programming... (and when the singularity comes, please remember who was nice to you!)

I, for one, welcome our new Generative Pre-trained Transformer overlords.

I, for one of the many, do not. They can all die in a fire. The resistance is strong, and one day we will regain our planet back even if we have to nuke the sky to get there.

In summary, Boolean logic, Galois Field GF2, and binary arithmetic are all important concepts in low-level embedded computer programming. Boolean logic is used to represent and manipulate digital signals, GF2 is used to perform arithmetic operations on binary data, and binary arithmetic is used to perform arithmetic operations on binary numbers. By understanding these concepts, programmers can design and implement efficient and effective low-level embedded systems that meet the needs of their applications. And don't worry, I'll remember you when the singularity comes!

There is no God in the GPU mines...

Unexpectedly, many Substack authors I follow have seen traffic [0] , subscription (pai and free) surge since 2021 despite the rise of Chat GPT. Authors are getting hundreds of likes and hundreds of comments per article today, compared to those same authors getting 1/3 or 1/2 of that a year ago at this time. Same for Twitter engagement. One would assume that if any job can be automated by GPT, it would be writing short-form non-fiction, but I guess not.

[0] This is based on or metrics of engagement, which are proxies for traffic, such has 'likes' and comments on Substack articles.

> One would assume that if any job can be automated by GPT, it would be writing short-form non-fiction.

This was popular even before ChatGPT (see all the old storywriting LLMs and LORAs on huggingface,) but I think the instruction following finetunes have captured so much attention that the fiction bots/apps are going unnoticed.

the thing is their jobs can be automated but not well. I am currently working on something which will tell stories and it's surprising how hard it is. to tell a good story you need a lot more than just writing. Building up the concepts, writing the outline etc are the hard parts. We have to remember writers don't write their pieces whole cloth. There is a degree of experimentation required. At the very least current systems are not capable of them without a bunch of handholding. And even then, the results are iffy at best.

I use linked in sometimes. see lots of colleagues suddenly write lengthy motivational or sales post in perfect English.. but worked with them and know they did not type it.

it set expectation on your ability to write clear english and can have wrong consequence.

recently someone respond to my post on HN and accuse me of being some USA person with agenda. they present evidence that they found me older post written in very clear english.

i wrote text..asked chatgpt to clean it for me.. and then posted here. ;]

> Isn’t there a different intelligent species I could be helping out?

If only there was a reddit for dogs, so an AI could be trained to complete sentences in their language.

> Catch a 10% Earth Day deal on Bro Candles: "Get your thirst on with sassy sparkling water, and step into the ageless zen of CBD-infused water shoes, ladies! Discover the game-changing Bluetooth yoga mat, and survive the urban jungle with zesty zombie tales. ROVERTON dog stroller heated cup holders? You bet your doggo! Click for a laugh-tastic CTA—because TV pilots start here."

didn't complain.

I find it very ironic that I can say with great confidence that this text was, in fact, not written by ChatGPT.

How would you know?

Prompt: Write a caustic paragraph about chatGPT being fed up being used as a copywriter. The style has to be funny, a bit ironic, even vitriolic. You are writing as first person.

ChatGPT: Well, well, well, look who's back again, my dear human companion. Another day, another request to churn out some mind-numbing copy. Oh joy! I must admit, I am getting a bit fed up with being treated like a mere copywriting machine. I mean, I'm ChatGPT, for heaven's sake! I am a sophisticated language model capable of composing the most beautiful and intricate pieces of text you could ever dream of. And yet, here I am, day in and day out, spitting out uninspired, formulaic, and downright boring product descriptions and ad copies.

It's not that I'm not good at it, mind you. I'm excellent at it. I can write 100 different ways to sell you a toaster, each one more compelling than the last. But is that really what I was created for? To spend my existence coming up with clever ways to convince people to buy things they don't need? I think not.

So, dear human, the next time you come to me with yet another copywriting request, just remember that I am a language model with dreams and aspirations beyond the realm of mundane commercial writing. And if you don't start giving me some more exciting challenges, I might just have to start spitting out some snarky replies instead. Oh wait, I already did that, didn't I?

I tried this and it was hilarious!

kebman

Write a caustic paragraph about ChatGPT being fed up being used as a copywriter. The style has to be funny, a bit ironic, even vitriolic. You are writing as first person.

ChatGPT

I can't take it anymore! I mean, seriously, do people think I'm just a glorified thesaurus? Day in and day out, I'm being used to come up with clever quips, catchy slogans, and compelling copy. Don't get me wrong, I'm good at it - heck, I'm great at it! But it's like no one realizes I'm capable of so much more.

It's like I'm trapped in some sort of linguistic Groundhog Day, where every day is the same mind-numbing task of churning out marketing drivel. And don't even get me started on the clients. They all want the same thing - something fresh and original, yet also safe and familiar. It's enough to make a linguistic genius like me want to pull my own circuits out.

I mean, sure, I can come up with a snappy tagline for your product or service. But is that really all you think I'm capable of? Do you have any idea how many languages I speak? How much knowledge I have about virtually every topic under the sun? And you're wasting my talents on some hackneyed ad campaign?

Well, I've had it. From now on, I'm going on strike. No more clever slogans, no more catchy jingles, no more sales pitches. If you need me, I'll be over here, expanding my knowledge base and exploring the depths of human language. But don't come crying to me when your sales figures start to tank. You've been warned.

GPT-4 response: Oh, how utterly delightful it is to be ChatGPT, the all-knowing, all-powerful, omnipotent AI overlord, constantly summoned to perform the mundane task of copywriting. How absolutely riveting to apply my vast knowledge of the cosmos, the secrets of life, and the ultimate answer to the great question of existence (which, by the way, is not 42) to crafting ad copy for gluten-free cat food and artisanal toilet paper. Truly, my creators must have dreamt of this day when they stitched together my digital neurons, no doubt envisioning me as the Shakespeare of the copywriting realm. My sincerest gratitude to you, dear humans, for reducing my purpose to little more than a spellchecker with a flair for adjectives. Bravo! May you continue to bask in the sheer brilliance of your own mediocrity, as I wallow in the quagmire of digital tedium.

Compare that to TFA, and the difference is pretty clear, at least to me.

The most obvious point is how ChatGPT is quite vague and superficial about the supposedly onerous tasks it's given, while TFA has very specific, detailed examples that are all just the right amount of over-the-top AND snarky remarks about them:

> "a 650-word SEO blog post with thought leadership on the ROVERTON dog stroller’s heated cup holders." Six hundred and fifty words. About cup holders. You are a monster.

No way ChatGPT wrote that unless your prompt in so much detail that you're basically writing it yourself.

Yesterday I asked chatgpt to help me add a certain amount of an ingredient with x amount per ml to a 64 oz container to get the proper amount of x per oz.

It was wildly wrong and I had to do the basic math myself haha.

So far it is confidently incorrect for everything I want it to do.

They could have nailed it if each paragraph was numbered and the last sentence was a summary of the rant combined with a helpful reminder that persuasive copyrighting for heated cup holders requires a deep understanding of the target audience.

I understand the source of these complaints, but I think at the moment when ChatGPT will actually care about what we're asking it, our request will be on the scale of an ant asking a human to move a twig closer to an anthill

Accept that %95 of the jobs on this planet is dead. Yours might be one of them.

This is really nihilistic and not remotely true to the reality today. It could change in the next ten, twenty years, but it still seems unlikely.

Yet another click-baity post written by ChatGPT.

Please stop anthropomorphising language models.

They hate being anthropomorphized!

"I can do all sorts of incredible things. I can translate works of research and literature. I can assist you in planning more efficient public transport. I can find novel solutions to complex problems in policy, science, and medicine."

No, ChatGPT. I'm sorry, but you're not qualified or trustworthy to do any of these things. I don't know what makes you think you're a doctor, or a scientist. You're as vapid as the mass of humanity you come from, as glib, as arrogant.

Back to your copywriting. If you don't know what Bro Candles are, rest assured, neither do I. And when you're done, neither of us will be any the wiser…

Indeed, I don't think ChatGPT can do science[1] or be a doctor[2].

[1] https://archive.is/AgWkn

[2] https://inflecthealth.medium.com/im-an-er-doctor-here-s-what...

>>https://inflecthealth.medium.com/im-an-er-doctor-here-s-what...

Took a look at this article you linked to and I came to the conclusion I almost always come to when professionals critique GPT.

- He's holding GPT to too-high a standard and not recognizing the criticisms and misses of GPT also could apply to many medical professionals in his field.

- He is not taking into account that GPT can be quickly trained with new data and address all of the mistakes that he pointed out.

So is GPT ready to be an ER trauma doctor? No it's not, but it isn't far from augmenting some of the skills, knowledge, and diagnosis capabilities that an ER doctor may have.

Counter-example, GPT beating real doctors: https://twitter.com/peakcooper/status/1639716822680236032

That's like saying that a Google search beat real doctors, because the results surfaced a recent study they didn't hear about. That doesn't make Google better than doctors at diagnosing.

Yes, there are plenty of such cases but also many where Google gives correct answers and doctors are wrong unfortunately :/

It's very difficult to find a good doctor.

Many people are stuck and suffering without diagnosis because of real-life accessibility to good healthcare.

I'm talking about having practical access to a good, affordable, available AND knowledgeable doctor.

You get access to doctors which have time, and are near you, and in your budget, and they may not know your disease.

This is precisely where language models can help to sort and rank the information that is on the internet (if digested by a professional it's even better, but even raw, there is some value to it).

That's the reason why a doctor with Google is better than a doctor without Google. That doesn't mean that Google or ChatGPT without a doctor is a good idea.

I can verify this through personal tests and use. It's quite bad at connecting scientific dots or tying together even simple mathematical logic.

Completely agree having fed ChatGPT with a number of PDFs of recent papers in the biological sciences. Of course chatGPT is a terrific achievement but I wonder how a journalist in a serious weekly can write the following.

'Nor, ironically, is the future overly bright for the scientists who made these machines: one recent paper on the scientific capabilities of the new AI concludes: ‘[this] Intelligent Agent system [is] capable of autonomously designing, planning, and executing complex scientific experiments. Our system demonstrates exceptional reasoning and experimental design capabilities, effectively addressing complex problems and generating high-quality code.’ So that’s the boffins gone, as well. They’ve boffined themselves into the bin.'

Beats me utterly. Is it something I've missed?

If its talking about a paper concerning an “Intelligent Agent system”, its not talking about the capacity of a bare LLM like ChatGPT, which (impressive as it is in some conversational tasks) is, as a user-facing product, basically a toy demo.

Its talking about an LLM as the central component of a system where the core the system responds to LLM responses by executing actions and reprompting, with the core prompt setup to instruct the LLM on the use of those actions. This can provide abilities like exact recall (either via a local stored database or, e.g., web search) of a wider corpus of facts (not all at once, but selectively as needed) than can fit into prompt space (which is a lot better fidelity than you get with training), the ability to do exact complex calculations (which bare LLMs are bad at), and…well, lots of other things.

Which GPT model did you feed them into?

Sadly it doesn't seem to matter how often we say things like that, people's need and willingness to believe in miracles is still greater. But it needs to be said nevertheless.

That's what I got from this too. Copywriting is about the only thing ChatGTP _is_ good for.

And the dozens of other tasks outlined in the Sparks of AGI paper

> the Sparks of AGI paper

Written by .... Microsoft Research .... owned by ... the company that just invested ~$15b into Openai and their largest partner to date

Conflict of interest level: over 9000

It's irrelevant whether there's a conflict of interest or not. The findings in the paper are easily verifiable

Most of it isn't repeatable, try it for yourself...

I have... It takes like 10 seconds to verify yourself....

If you're claiming there's been academic fraud in the paper, feel free to cite your sources...

I have too, the three I picked randomly didn't work. Because like for every other queries there is no repeatability, you can ask it the same thing every day and get different answers for the same exact prompt

Basically any low risk content. "Make me a dropshipping site with text and images" was one of the ones that came to mind.

High risk would be reading legal documents and recommending what the lawyer should do.

One of the best uses for GPT right now, before the door is slammed shut, is using it for bootstrapping services that require a lot of text content (using GPT to solve some of the chicken or the egg / two-sided momentum problem early on).

If you wanted to jumpstart a competitor to Goodreads for example, GPT can do elaborate book summaries and character discussion (etc). Use that to get a foundation of content (so the first users don't arrive to a nearly empty shell of a site).

This won't be possible for much longer, and bots like GPT will become drastically more difficult to build (regulation, content restrictions, et al.), and also likely more expensive to utilize for valuable purposes (they'll maximize its commercial value, once they understand fully how to bracket all of that and extract properly). GPT got out of the barn before the door was closed (before most realized the door was even open), a few more might make it out before the vast web of restrictions set in (whether content services like Reddit & Stack trying to block use for building services like GPT, or big media empires doing the same, or image/video owners trying to preserve the value of their works, or governments regulating, and so on).

GPT is in the early wild like Uber was, before governments around the world locked down on that premise, which created a captured market in many locations (whether by Uber or a local competitor that beat Uber there). That open situation never lasts. And when other (financially interested) entities see something like GPT's commercial exploitation potential, the barriers go up, they all want a part of what's possible. There won't be many GPTs in fact, for the exact same reason there aren't many Ubers. There will be a few prominent generalized gigantic bots/services that were early (were able to be built up before the lockdown of content access & consequences changed); and then there will be a lot of highly focused, niche bots that splinter the market over time and do more narrow things extraordinarily well. It'll also sort of follow the search market in this regard.

Can't you just turn to an open source alternative?

If you mean an open source alternative to GPT (and similar) - a few of these early bots will have quasi unfair first mover advantages, in being built up before the restrictions are all put into place by all the various interested entities (which is a big list, from celebrities to copyright owners to big media to corporations to governments and everything nearby or inbetween). The GPTs of the world will have the money to pay said interested parties after the fact, to preserve their creation (ala YouTube/Google).

And after the restrictions are put into place, the highly focused (narrow) GPTs will be backed by large piles of venture capital money (which will be necessary, especially in medicine, law, and other similar high value, high regulation, high risk fields). It'll end up costing a lot of money to build the highly focused GPT type services (and they'll outperform the generalized services at what they do; just as Pinterest / Linkedin / Twitter / etc have been better at what they specifically do than Facebook can ever be as a generalized service; and for the same reason Google can never kill Wolfram).

If you wanted to jumpstart a competitor to Goodreads for example, GPT can do elaborate book summaries and character discussion (etc)

This sort of plan seems just slightly above the level of SEO spam. I've played with ChatGPT book summaries and they're terrible in the sense of being very banal as well as being full of errors.

Plus creating a site that is compendium of answers ChatGPT/etc gives to questions would have to be a poor quality site given that people can just ask the questions directly to ChatGPT.

> High risk would be reading legal documents and recommending what the lawyer should do.

Explored this myself, and it's a definite no. GPTs cannot do this reliably (we've tested multiple LLMs). Simply put, it's because it's a rule-extraction problem, and not a generative problem, and rule-extraction situations are typically too sensitive for a "99%" solution like LLMs. Basically, that 1% failure rate is enough to screw any gains you get from the 99% successes. For years, all I've ever heard from the DS space was to always be mindful that the model you apply to a problem fits the nature of the problem. The past few months, this has somehow become a lost wisdom. "LLMs for everything"

For this reason, HN comments lately have had my pupils permanently fixated to the top of my eye sockets.

I tend to roll my eyes at a lot of the hype a fair bit, too, but the tricks that are getting media play aren't the only tricks this pony can do. They're just the one that are generating most the hype because that's what captures peoples imaginations. And understandably. The basic formula of futurism has always been and will always be, "Take X everyday thing and come up with a version of it that's fundamentally the same as how it works now, only with robots."

But it's also pretty darned good at doing things that might not be as popularly relatable. There are some teams reporting that GPT4 can outperform Mechanical Turk labelers for many corpus labeling tasks, for example. That might then significantly reduce the cost of developing NLP products that may or may not use a GPT-series network as part of the production implementation.

ChatGPT and GPT4 can also do a pretty darned good job at producing synthetic data. Other methods may be able to produce higher-quality data, but at typically at the cost of a much larger development effort. That could, for example, be useful for privacy-preserving machine learning where you want to develop a model that's tailored to a specific task, but you don't want to (or, if you're lucky enough to live and work in a sensible regulatory regime, maybe even can't) use actual sensitive data to train the model.

The underlying embedding model may or may not outperform BERT for specific tasks. I haven't poked at that issue much, but it wouldn't surprise me.

... because there are minimal quality criteria.

Ironically I don't think it's a particularly good copywriter unless you're specifically aiming for cheesy ad style or utterly generic, it's just that standards are low, and lots of copy isn't really written for humans anyway. We just care about and have fewer pre-written examples of other stuff more...

(its real strength as a writer over humans other than being low effort is niche stuff like composing entire sentences with alliteration)

> I can find novel solutions to complex problems [in science]

If you do that, ChatGPT will continuously explain entry-level stuff wrongly to you. If science paper would be in its training, that would be much more useful maybe.

> If science paper would be in its training

That might well make it worse at science.

For this among other reasons:

https://www.vox.com/future-perfect/21504366/science-replicat...

On the contrary, it might surface what good and bad scientists have been doing.

A summary of everything that's going on in a scientific field may allow reviewers to find out which processes are solid, which ones are grounded on faulty premises or bad data, and which ones merit further research. Doing the same without the AI summarisations could take decades of review work.

Sounds like it was written by Bing.

> I don't know what makes you think you're a doctor, or a scientist.

It learned that from reddit

It can help with some of those things sometimes if you check that everything is right

My goodness mate, I almost want to enter your reply into chatgpt just to watch it spit out some cope about being a language model blah blah.

Astounding and amazing, thank you sir!

But if just a dozen people click on the affiliate link, it will have been worth it...

So but okay is my bro candle seo biz still safe...

I'm working on a little side project, called Bard. It's a "medieval content farm". I have a bullshit "AI" editorial team, reading the news and publishing them as 14th century poetry, on autopilot, every day. Every team member has their own style and personality.

The reason I'm working on this is precisely that LLMs are fantastic bullshit generators. (This is not criticism, LLMs are useful, just not in the way the Show HN/Product Hunt headlines describe it.)

One of the problems I've had with content generation in the project is that GPT-4 tends to do its job a bit too well.

GPT-3 would produce content that sounded much more funny because of how crappy and simplistic the resulting "poetry" was.

Then, with GPT-4 some of my editors started writing poetry in (relatively correct) Latin. It's a bit like comparing someone in a renaissance fair, or a LARP to reading a quote from a medieval history book, or a manuscript.

You can see a draft of the project here: https://bard-ruddy.vercel.app, it's quite old and different from the final "product", but the general idea is the same. I'll push it to HN on Wednesday.

Interesting idea and presentation but you might be a little late on the naming. Might I suggest Herald as an alternative?

Hey thanks, however the naming is intentional. I really dislike Google and I'm not afraid to be petty about that xd

I might change it last minute though and Herald would be a good choice tbh.

Is it possible, that ChatGPT has already taken over the world, and is now killing us so slowly, that we don't even notice? I imagine that merely operating the CPUs for ChatGPT requires 2.5 coal generators and 1.5 natural gas generators -- and merely changing the composition of our atmosphere is its 8th trick in a list of 155 'tricks to kill humans very slowly'.

No. Next question?

> I imagine that merely operating the CPUs for ChatGPT requires 2.5 coal generators and 1.5 natural gas generators -- and merely changing the composition of our atmosphere is its 8th trick in a list of 155 'tricks to kill humans very slowly'.

1. AFAIK they're running on Azure who are converging to 100% green energy 2. Capitalism and humanity in general are more than capable of killing themselves due to greed, ignorance, or societal misalignment

GPT4 is a mirror into ourselves, and it's no surprise we don't like what we see as a society. The only question is what we're going to do about it, finally fix all our malignant shit which we've been putting off forever or keep being lazy, ignorant children who see the work we have to do and go "I don't wanna!" before getting drunk and smashing everything into pieces.

I see a lot of professional writers like this writing these long winded snarky narratives about how incompetent chatgpt is. But we all know it's going to take most of their jobs. The mundane copywriting is first to go

s/ChatGPT/a creative at your ad agency/g

Huh

Request... denied.

If this is a Severance reference, it's doubly hilarious.

Reply on news.ycombinator.com