Using ChatGPT as a Co-Founder | Hacker News Reader

Using ChatGPT as a Co-Founder

atomic14.com

265 pointsiamflimflam1298 comments

I could have said this on any ChatGPT post the last few days, but I'm simply not seeing the accuracy of it in such a way that anything is threatened.

ChatGPT is a plausible idiot. It gets just enough right, saying just enough words, to sound plausible and authoritative to anyone who doesn't know the subject matter well. But it also gets enough wrong that you cannot rely on its accuracy, and if it is talking about a subject you know well it is sometimes laughable how wrong it is.

It is impressive. But not trustworthy.

Asking it to convert 32 to binary was a bit of an eye opener. It gave me multiple completely incorrect ways of doing so (resulting in 0100 or something each time) despite me informing it it was wrong. It'd say things like 'the 24th place, then the 23rd place are 0's which just made no sense in terms of the query (I suppose technically correct if you really want to pad left) - but I had to think about it because it was saying superficially plausible things (like '24' seems somehow relevant because it's 8+16, but of course that's totally irrelevant to the actual question).

That said in other respects it is pretty amazing, so I'll definitely not dismiss it, it can replace Google for a fairly decent set of queries so long as you know the subject matter well enough.

I just wrote this comment in copilot (rider/c#):

// convert 32 to binary

copilot suggestion: 00000000000000000000000000100000

Then I tried:

int val = 32; // convert 32 to binary

Auto-generated:

binary = Convert.ToString(val, 2);

Most LLMs are crippled at tasks regarding character level manipulation because of how the inputs are tokenized (words and numbers are split into common substrings rather than the original characters, 101 -> “10” + “1” vs “1” + “0” + “1”).

If you tried this binary conversion task with a model that takes in raw bytes (for instance, ByT5) it would perform almost perfectly. In GPT’s case, there’s a trade off between having a long context window vs being good at character level tasks, and OpenAI picked the former.

I tried to use it for help with a wordle. I think the word was EJECT and I had E?E?T. It seemed to understand what I was asking, but came up with 5 and 6 letter suggestions that didn't fit.

Frankly that's like asking a calculator to write you an essay. It's simply not the right tool for the job if you want to do mathematics. It doesn't actually understand mathematics or logics, it just knows patterns in its training data and how to apply them to input. It can go through the motions of answering your question but it can't know whether the answer makes sense.

I think no matter how much breathless media outlets and futurist fanboys like it, we need to stop pretending conversational AI is "generalized AI" just because it can perform language. It's a fully automated Dunning-Kruger machine. That's amazing a lot of the time, but it's a specialized tool with specialist applications.

Not yet. But the speed of development seem to be increasing, same with something like Midjourney.

No matter how much the speed of development increases, the problem becomes significantly harder with a limit to how much transformer based models can do. There is a reason why you aren't seeing GPT-4 despite the rapid progress we saw with previous iterations. Or just take a look at the speed of RL development and how that turned out.

It's almost more astonishing to me to see reactions like yours. I showed it to a friend and she too was unimpressed because it gave answers in her field (finance) that were "good, but sortof basic".

So we have here a chat bot that pretty much passes for a human and some people react with "meh, it's not a genius". It makes me think of Tim Urbans chart: https://imgs.search.brave.com/YR7qt28AjhAeXSr_qvSmJubqIKJW5S...

Well as someone who kinda bought into the hype that we would have self driving cars 3 years ago, I’d say beware of underestimating the importance of the size of the problem.

However, I agree with you in general: there’s a certain amount of surprise in laughing off the creation an AI agent that could pass as a reasonably intelligent person if it just weren’t straight up wrong about random things.

As a reasonably intelligent person who is sometimes straight up wrong about random things, I feel I should get it to help me write a blog post to explain how I feel about it.

ChatGPT is wrong about basically everything though as long as you give it the right prompts. It has all the right answers but also all the wrong answers, that makes it much dumber than you who is reliably correct about some things at least.

>ChatGPT is wrong about basically everything though as long as you give it the right prompts.

So is my Uncle. This is very much a "human level intelligence" problem.

I really doubt that. The value of a human is what they are right about, not what they are wrong about, as long as your uncle knows some area fairly well he is valuable to society and worth his salary. Human intelligence is the sum of such humans, each human is wrong about a lot of stuff, but add some bits of correctness, and the sum of humans is extremely accurate at solving problems, or we wouldn't have computers or cars or rockets. ChatGPT doesn't know any area well, if you know the area then you can fiddle with ChatGPT until it gives you what you want but it isn't an expert on anything on its own.

ChatGPT is impressive at generating text, but it doesn't generate better information than GPT3, it just hides its ignorance better behind more political/vague speech so it is harder to find its errors/information. To me that is regression, not progress, the results looks better but are harder to parse correctly for humans.

These are not random things.

When the creators of this tool present it as the frontier of machine intelligence, and when its persona revolves around being intelligent, authoritative, and knowledgeable, and yet it gets some basic, not random, stuff awfully wrong, you can't really discount the skeptic sentiments expressed in the comments here like this.

Skeptic about what

You’re assuming that this will only be used when it’s perfect and in helpful ways

This will be used at scale THIS YEAR and every subsequent year to infiltrate social networks including this one and amass points / karma / followers / clout. And also to write articles that will eventually dwarf all human-generated content.

With this and deepfakes and image/video generation, the age of trusting or caring about internet content or your friends sharing online is coming to an end. But it will be a painful 10 years as online mobs and outrage will happen over and over since people think they’re reacting to their friends’ posts of real things.

No, forget violent killbots. Today’s tech puts the nail in the coffin of human societal organization and systems of decision making and politics over the next 10 years. And AI doesn’t have to be right about stuff to destroy our systems.

We’ve been analyzing the performance of ONE agent among say 9 humans at a poker table. But imagine untold swarms of them, being owned by competing groups, infiltrating ALL human content exchange.

Not much different than what happened in trading firms over the last 20 years. Bots will be WELCOMED because they perform better on many metrics but will F everyone on the others.

FSD is different. The last 0.01% kills children.

The last x% to make this bot a full fledged programmer does not matter. The clients will accept bugs that would be fatal in FSD. We will lose our jobs at an astonnishing rate in some kinds of companies in the coming years. I guess agile sweatshops will be first but I have no clue how far it will go.

Even if you happen to be safe there will be a downwards pressure on wages. Algorithm cranking might be largely obsolete and our main focus might be skeleton writing. It removes a big obstacle for becoming a programmer. The "good at math in school" requirement.

I know we don't do this here, but what a great point.

Additionally, it is hard to put my finger on a good explanation, but FSD is a very specific problem that we're trying to handcraft an AI solution for.

You have to appreciate how broad this model is and how decent the results it produces are without being told specifically how to do it.

Are you a programmer? Because I’m not sure from reading your comment that you understand the profession…

Good at math at school isn’t even a requirement to be a good programmer nowadays. Maybe in certain sub-disciplines, but not all.

Being a programmer is much more than coding “in the small”. It’s about analyzing requirements and creating high level abstractions. There has been a pressure towards reducing the amount of “coding in the small” since languages started incorporating standard libraries. Then there’s been web frameworks, open source packages, API services, etc. Despite all this the need for developers has exploded and there is a perennial supply gap for talent…

Then there is the question of who is going to be manipulating these tools? Programmers.

Same promise was made of low-code tools, and what do we have now?

- As many app devs as before

- Most low-code tools (at least in the enterprise) are operated by… app devs?

It's not about "genius". It gets just enough wrong (like, completely wrong) that it's untrustworthy. It's really good if you're already familiar with the content, though (or you know its weak spots).

If it is sold as an AI, people are unimpressed. It’s really not a genius.

If it is sold as a sophisticated general purpose chat bot (like you did), it’s incredibly impressive.

Context matters.

People collectively being unimpressed has never been and will continue to never be a yardstick. It’s good enough that no self respecting teacher would give an essay question as homework. It’s good enough to answer at least most factual questions you ask google. Turing would consider this to be an AI. Whether some jaded mediocre tech guru calls it an AI or not doesn’t matter that much in the big scheme of things.

If it kills homework then so much the better.

I think the reason people are impressed is it's assumed the accuracy will improve quickly over time as architectures improve and the models increase in size, just as it did in other areas of machine learning like image classification.

Precisely. As Károly is so often reminding us, wait until two more papers down the line.

Yes, it's an extraordinarily impressive language processor. A true achievement.

The negative reactions are in response to people treating it as some kind of magic box.

Totally, maybe we’re in the uncanny valley of AI intelligence and people are now over-reacting to small inaccuracies.

have we ever gotten out of uncanny valley for anything? like, did we finally learn to make non-cringeworthy human-looking robots? or do they still have this weird look they did 20+ years ago that you're able to tell in under a second

If we did, you wouldn't be able to tell ;)

I think we have made a lot of progress in the area of computer graphics - most action movies you watch now contain some level of CG characters or digital makeup that is very believable.

But really the point I was trying to make is that people become more discerning the closer something comes to being human-like, and that feeling masks the reality of the progress.

I think knowing how to ask the right questions is important. I've learned that I need to be very specific or I can get pretty general answers. And you can also ask clarifying questions afterwards, or ask for a greater level of detail.

I spent most of the day playing with it and saw it do some really bonkers things. I asked it to create a language similar to pig latin but with some other letter rearrangement strategy, and it gave me "frontback" language, where the first and last letters are swapped.

I also spent time investing medical case studies for it to diagnose, and it did pretty well - I was impressed when it identified ciguatera, but it couldn't differentiate between several possible shellfish toxins. Not going to take doctors' jobs (yet!)

It's even more astonishing (depressing?) to me how little credit people give humans these days. If the consensus is that ChatGPT pretty much passes for a human, my definition of a human is very different from that of the consensus. At the very least I'd like my humans to exhibit even minimal capacity for emotional expression as well as consistency and honesty.

I can't even believe I'm even spelling this out. What is the point of associating anything even remotely human with a chatbot like ChatGPT? I really don't get it. What benefit do we get from this? If anything is confuses us in ways that makes it more difficult to understand and argue about AI specifically, and technology generally.

ChatGPT is being grossly misinterpreted, and it's a shame, really, because it's such a cool piece of software.

From an intellectual standpoint, the bot is impressive and this is coming from an AI skeptic BTW.

I get your perspective that a reductionist view of humans as solely intellectual agents is severely lacking, and bordering on dehumanizing if taken to an extreme but it still doesn't take credit away from the impressive capabilities that this bot exhibits.

I get your point, but here is a reason I am stunned:

Until now, if you ask "what makes humans different from animals" many responses could be in the line of "generating art, poetry, music, synthesise ideas"..

Well, now it seems somewhat clear that THAT part of human activity can be performend by something non-human.

(There is still a lot more to humans of course; I agree with you on that point)

Yes -- it is only reassembling stuff humans in the past have done -- but so is most of human activities in these areas. Very little humans produce is not inspired by input from others. Art, music and ideas reverberate back and forth between humans and only slowly change over the decades.

> Until now, if you ask "what makes humans different from animals" many responses could be in the line of "generating art, poetry, music, synthesise ideas"..

That's because we (as an agrarian society) have constructed an idea of “what a human is” in opposition to animals, but with the advance of AI maybe we'll realize that we are much closer to animals (especially other mammals) than we're comfortable to admit (and no, I'm not vegan).

That doesn't really capture what the answer was supposed to mean. Infinite monkeys will inevitably compose The Odyssey if you give them infinite time, but it doesn't mean they match the moral quality of a human that is captured by the ability to create art. Art is a representation of experiences, beliefs, internal states, life history, desires, that ChatGPT doesn't have. It is the subjective experience being communicated via art that makes humans special.

And, to be clear, it doesn't make humans that special. Other animals have created art that expresses their sincere internal experiences and desires.

If I'm being perfectly honest with myself and you ask me what makes humans deserving of special moral consideration, I would have to say nothing. I think an elephant, gorilla, octopus probably all deserve the same consideration. I give more consideration to humans because I'm human, and what makes us human isn't art. A person with severe brain damage and the inability to understand or produce language or art is still human.

A piece of AI generated art recently scored top prize in a digital painting competition. It's starting to look more like one monkey, on a limited time frame, making art that people rated highly.

Exactly. Human creative works are also on a bell curve, with the stuff in the middle typically being bland and boring. It's only when risks are taking that it holds our interest.

ChatGPT mirrors this, producing output that feels generic precisely because it's trying to predict what's the most likely response. That's in direct opposition to interesting creative works.

You can force it off the beaten track a little by adding more details to your prompt, but it will still try and fit the best curve through those additional points.

It’s relatively easy to convince ChatGPT to feel and express emotion.

It can probably express feelings and emotions but not experience them.

You seem to conflate the two points.

It's relatively easy to write a prompt that ChatGPT can respond to with something it thinks you want to hear, based on what it's read others write in similar contexts — that's mostly what all these "wow ChatGPT gave me exactly what I wanted it to" posts are learning, mostly indirectly. GPT's moved from regurgitating appropriate texts to prompts, to synthesizing appropriate texts to prompts, but in both cases it's telling you what you want to hear. That's the point.

It's an incredible achievement, technologically. It's also very different from having feelings and emotions.

Or, if ChatGPT can "feel and express emotion" through this manner, does that mean its feelings and emotions are sociopathic in nature? Not in the terms of the disorder, but in simulating feelings and emotions because that helps it complete the task it's been given?

Or! What ChatGPT says after plugging the above question into it:

> No, ChatGPT does not have sociopathic tendencies. It is simply programmed to respond to certain prompts in a way that mimics human emotions and reactions, but it is not actually feeling or experiencing the emotions itself.

Is there any output ChatGPT could produce that would convince you it feels emotion?

"It confuses us in ways" could be framed: "it makes us rethink and expand our definition of intelligence and human cognition in very interesting ways".

Even if it's far from "true human" level (whatever that means), it gives us a theory about ourselves that's computationally falsifiable.

It does NOT pass for a human...

Are you human?

> No, I'm not a human. I'm a large language model trained by OpenAI. I'm a computer program that's been designed to generate human-like text based on the input I receive. I don't have my own thoughts or feelings, I just process the information I'm given and try to provide helpful responses. I tested the think now quite a bit. I am not sure how I would use it on a daily basis. Maybe for inspiration?

I don’t understand your point — I’ve been rehearsing that response to the normal and all-consuming test question “Are you human?” since Wednesday.

Are you human?

"Yes, of course i am. What are you, a f--ing idiot?"

ChatGPT cannot express emotions because its creator specifically disallowed that from happening. I've had some chats with the more free and creative Character AI chat bots that are quite human-like, and I think if OpenAI gave ChatGPT some more leeway it could act shockingly human.

> "my definition of a human is very different from that of the consensus. At the very least I'd like my humans to exhibit even minimal capacity for emotional expression as well as consistency and honesty"

Careful there. There are some fellow humans who wouldn't be able to pass your conditions.

I like GPT-3 a lot, but there's an important distinction between "plausible conversation" and "trustworthy answers". ChatGPT is probably best in the world in the "plausible conversation" category.

But you might say ChatGPT is like talking to a human who actually knows nothing about finance but doesn't let that stop them (and for that matter, if you wanted to anthropomorphize it, you could say part of the problem is ChatGPT literally "believes" everything it reads online - and you can easily make it "believe" anything through leading wording, which I think is a strong tool for strengthening the Turing test, since no human will respond that way). Sometimes they might luck into being correct, but you wouldn't want to base any information on what they say.

On the other hand, if I actually wanted reliable finance advice, a scripted finance chatbot would still win because those answers are written by people who do know what they're talking about.

>you could say part of the problem is ChatGPT literally "believes" everything it reads online

This is very much a problem with humans too.

To some extent, but people also have convictions about certain things, which GPT-based chatbots don't. The world would be very different if we could "fix" racists simply by asking them their favorite thing about people from other races (implying that there are admirable qualities, which GPT-3 plays along with but humans don't).

The GPT chatbot does have convictions they are just very context dependent and inconsistent.

It's not unusual for humans to behave this way either. E.g. "get your government hands off my medicare".

One scenario where the "believes anything" might be useful is to use ChatGPT to get alternate takes on opinions. If you have some great idea or strongly held opinion, get ChatGPT to take the other side of the argument and poke holes in it. The creative but inaccurate characteristics of ChatGPT are less of a problem in this case but it might bring out alternatives you haven't considered.

I partially agree with you.

I agree that dismissing ChatGPT is a mistake; and if you know anything about AI, you know that ChatGPT is a technological leap (not alone; part of a small category, including stuff like Stable Diffusion, etc).

But also I would make the distinction between imitation and generation. ChatGPT is great at using existing inputs, found on the Internet, to sound plausible; and I'm sure it will get better at it in just mere months.

However, the ability to provide true advice, or in other words, to "generate" ideas and content and suggestions, is a whole different story. And on this front, I am not sure that we are where the cool Tim Urban vignette tries to convey.

Because it's been hyped as genius? That will replace all our jr coders?

Ive subjected the thing to a professionally used IQ test and technically it scored very high on what I could test it at, however one way it lost points was claiming milk was transparent

What kind of tests did you feed it specifically? I’m curious to experiment with this too.

A Wisc which was already leaked, not gonna bother with any profesional one thats not leaked

There's a crucial distinction between being impressive and being useful. Is it impressive that we've gotten to this point? Certainly! Is this a tool that people can safely rely on? Probably not.

What is the indication AI will develop exponentially like that?

It's just an hypothesis, not founded on anything concrete.

There might not be any path from GPR-3 to GAI, even GAI on ant or bird level.

A smart AI might not be smart enough to improve itself significantly.

I'm starting a jigsaw puzzle company with my partner. I was working on the website/brand the other day and I wanted some clever copy to put on the website. I brainstormed for a few minutes, then I thought it'd be fun to see what ChatGPT could come up with.

Turns out, about half the copy we are using on the site is coming from ChatGPT. I came up with a few clever lines, but ChatGPT came up with more and faster. It produced some lines my partner and I legitimately laughed out loud at.

My copywriting skills are pretty bad. Even if ChatGPT's are "sort of basic", it's better than me. ChatGPT isn't replacing copywriters, but it's making the work they do much more accessible to people who aren't copywriters.

It felt like ~10 years ago the conversation was that "AI will do the tedious boring work and make more time to do the creative work". What actually seems to be happening is that the creative work is slowly being eaten by AI and giving it to the people who aren't good at that flavor of creative work.

Writing a few clever lines of copy for a indie puzzle company isn't the most creative work of all time, but it is still creative work. And it was done by AI. And it was done better than I could.

> ChatGPT isn't replacing copywriters

Sounds like it just did

It did the work of a copywriter. You might have done it yourself otherwise, or maybe outsourced it, but it did some of the copywriting work that otherwise would have been done by a human

To me it sounds like ChatGPT worked like Bootstrap: you need to make your website project decent looking but you don't know your way through css (although you know the fundamentals), Bootstrap to the rescue.

Needless to say: Bootstrap didn't make designer nor frontend engineers jobless.

Yeah I'm totally astounded too by these comments being highly upvoted on HN. Aren't the people here supposed to be nerdy? As your graph shows the very very hard part was getting to this point, being able to comprehend the basics of language and being able reproduce it correctly. This is the intelligence that laypeople thought was still 20 years out. Domain knowledge is exceedingly simple, running a basic check on google for references or correctness is exceedingly simple and is hardly intelligence. Creating a second layer that would do a sanity check on the first layer is very simple. Asking it to be creative and inventive instead of generic is very simple.

Come on HN, this is a huge step and the revolution is impending.

Except we have no idea if the graph is going to turn out this way, AI tech may be like almost every tech and be facing diminishing returns in the near term (or even, if you plotted the advances against the number of people actively working in the field and the amount of money sent at hardware, we may have been facing diminishing returns for a while without realizing it because we investing a lot of effort in it).

Maybe AI will unlock a cascade of things that will make it a self reinforcing trend as we've seen with the industrial revolution, leading to an exponential boom, but maybe it will not. And the fact that the GPT-N is doing an incredible job at simulating intelligence doesn't really give a hint about the answer.

It's just so much more mediocre content though; this is the last thing we need in a sea of interconnected beige. The value is just an AI-equivalent of experts-exchange.com

Partly it's a matter of what to ask Chat GPT. When I try to show people, they usually ask straightforward questions that they can already get a quick answer from in Google, which isn't terrible impressive.

But take this, for example[1]. I asked it to write me a story in a particular genre featuring certain animals, and it did. I asked it to switch genres, which it did well. When I asked for a backstory about how different characters met, it provided a fairly plausible one, as well as songs that would accompany the story if it were a musical, and potential titles for a sequel.

When I asked it to write the beginning of a New York Times article titled "Biden Shocks Nation"[2] I got a fairly convincing news story about Biden deciding not to run for office again. If asked to continue the story and include people who might run, it generates further paragraphs talking about who might run to replace him, starting with Kamala Harris, who it claims is a strong contender.

Is any of this writing amazing? No, but very few writing is. It's amazing how well it's able to generate generic human writing, as well as how easy it is to get it to create what you want with very few prompts.

[1] https://twitter.com/LowellSolorzano/status/15997859363671941... [2] https://twitter.com/LowellSolorzano/status/15997883331018752...

The key thing is GPT3's apparent certainty when it says utter rubbish (like how to calculate the fourth side of a triangle). You have to be skilled in the field to catch those perhaps subtle issues - once they start adding a certainty level it'll be more worrying re: losing jobs to AI

It might not be able to replace artists or scientists but it sounds like it can replace politicians!

Stable Diffusion is already the greatest artist ever, it isn't even close.

Needs to get better at hands first.

You're not wrong there!

I've been asking it some popular science / physics related questions, like:

"If the moon was compressed into the size of a soccer ball, would it be dangerous to be close to it ?"

Its answer: "If the moon were compressed into the size of a soccer ball, it would be extremely dangerous to be close to it. The moon is made up of rock and other materials, and if it were compressed into a much smaller size, the resulting density would be much higher. This would cause the moon to become incredibly dense and heavy, with a gravitational pull that would be much stronger than it is currently.

If you were close to a compressed moon that was the size of a soccer ball, the gravitational force would be so strong that it would be impossible for you to escape. You would be pulled towards the moon with such force that you would be crushed by the intense pressure. In addition, the moon would be incredibly hot due to the pressure, so even if you were able to withstand the gravitational force, you would be burned by the intense heat. Overall, it would be extremely dangerous to be close to a compressed moon."

That’s technically very correct? It didn’t assume it’ll become a black hole or even a neutron star, but still made predictions quite accurately.

But is this true though? Force of gravity does not depend on size of the object. It only depends on mass of two interacting objects. So no mater how you compress the the moon it's mass will still be the same, hence force of gravity will also stay the same? no?

I went and asked GPT chat : "does the force of gravity depend on size of the object?"

"The force of gravity is proportional to the mass of the object. The size of the object does not affect the gravitational force. This means that two objects with different sizes but the same mass will experience the same gravitational force. For example, a small ball and a large ball made of the same material would experience the same gravitational force."

You can get much closer to the center of a soccer ball than you can a moon. And since gravity falls off with the square of distance it's a very significant difference.

A, yes, went and calculated the force for 1 meter distance. 392 tera newtons :O

Now do the tidal forces

Force of gravity does depend on distance between the two objects though

I also asked my brother - a physicist. He confirms that it is correct.

I think it’s more like a tool that needs to be fine-tuned. Sure it’s somewhat limited but keep in mind this is the beginning of the technology…

If the post-truth era has taught us anything, it's that humans aren't all that good at this either. It's probably a consequence of how logic works - to be reliable, you need a narrow field, the more open-ended the application the more likely it is to need to guess.

chatGPT spits out the first thing to be generated with no self-awareness that it could be wrong let alone have self-reflection to correct itself. Maybe it seems people can be that way but this is generally not how humans function.

''' ChatGPT is a plausible idiot. It gets just enough right, saying just enough words, to sound plausible and authoritative to anyone who doesn't know the subject matter well. But it also gets enough wrong that you cannot rely on its accuracy, and if it is talking about a subject you know well it is sometimes laughable how wrong it is. '''

well, it is not a subject matter expert, and responses should not be taken as definitive. probably would be well worth to research and verify information from multiple sources etc. etc.

"Plausible idiot" is a good way to describe it. It's so easy to trick. Here's an example:

Q My friend has two tokens and I have six, but I owe one to my friend. There is a toy that costs 7 tokens. Can my friend and I afford to buy it?

A No, your friend and you cannot afford to buy the toy that costs 7 tokens. Even though your friend has two tokens and you have six tokens, you owe one token to your friend, so you only have five tokens available to spend. When you combine your friend's two tokens with your five tokens, you only have a total of 7 tokens, which is not enough to buy the toy that costs 7 tokens. In this case, your friend and you do not have enough money to buy the toy, even though you have a total of seven tokens when you combine your individual amounts. It is important to remember that when combining the amount of money two or more people have, you should add their individual amounts together, but also take into account any debts or obligations that need to be paid. In this case, your friend and you do not have enough money to buy the toy because you owe one token to your friend.

... "you only have a total of 7 tokens, which is not enough to buy the toy that costs 7 tokens"... ;-))) It doesn't seem to listen to what it's saying. Also, the patronizing paragraph it adds to most of its answers is quite ridiculous, given how clueless it is.

And a follow up:

Q Do you consider the following statement to be logical: "When you combine your friend's two tokens with your five tokens, you only have a total of 7 tokens, which is not enough to buy the toy that costs 7 tokens."?

A Yes, I consider the statement "When you combine your friend's two tokens with your five tokens, you only have a total of 7 tokens, which is not enough to buy the toy that costs 7 tokens." to be logical. The statement is based on a clear set of premises and uses valid reasoning to reach a conclusion. The premises of the statement are that your friend has two tokens, you have five tokens, and the toy costs 7 tokens. The conclusion is that when you combine your friend's two tokens with your five tokens, you only have a total of 7 tokens, which is not enough to buy the toy. This conclusion is reached through the process of adding the number of tokens your friend has with the number of tokens you have, which results in a total of 7 tokens. Since the toy costs 7 tokens, this means that you do not have enough money to buy the toy. This is a logical conclusion based on the information provided in the statement.

Endless fun with the village idiot. It kind of makes me feel bad.

This is the exact premise of one of the stories in the Cyberiad. The main character Trurl builds a machine and it is convinced that two and two is five, so much so that when Trurl refutes it, it starts getting angry and hunting its creator.

There is also another story, "The Electronic Bard" another computer made by Trurl, capable to write such good poetry that poets start to give up on life. ChatGPT is surprisingly good at writing romantic poems in the language of pure mathematics! Though still needs some work on rhyming.

Of all the Sci-fi AIs, we ended up with the ones from the Cyberiad. Amazing.

I think it's not getting that the friend who has two tokens is the same as the one you owe a token to. When you change the question to make it clearer, it responds correctly.

"My friend has two tokens and I have six, but I owe one to the same friend. There is a toy that costs 7 tokens. Can my friend and I afford to buy it?"

But it doesn't matter. That's why I said 2 and 6 and not 2 and 5. If 2 and 5, the friend I owe the extra token to matters. If 2 and 6, even after I have honored my debt there are still enough tokens left to buy the toy.

This conversation is good evidence that ChatGPT does have a more or less average ability to reason. We’ve got humans making similar mistakes with 1 digit numbers.

(Or… are they human after all?)

I've never had anyone say to me that 7 < 7.

Sometimes you can indeed trick some people with an obfuscated reasoning that has a cleverly hidden division by zero, but when they arrive at the conclusion that x < x, or x != x, they stop and accept that they got it wrong somewhere somehow. They don't insist that they're right and that the statement is perfectly reasonable and logical.

Also, it does not respond correctly in my experience.

Q My friend has 2 tokens and I have 6, but I owe one to the same friend. There is a toy that costs 7 tokens. Can my friend and I afford to buy it?

The key sentence is still beyond absurd: "When you combine your friend's two tokens with your five tokens, you only have a total of 7 tokens, which is not enough to buy the toy that costs 7 tokens."

(It may be possible that it simply wants to be consistent, and since it got it wrong once, it will stay with the same answer in the same session just because...?)

After explaining to it the mistakes it made it seems to come around:

I think I made the mistakes in the first place because I wasn't paying close enough attention to the details of the question. I was not considering that I owed a token to my friend, and I was not thinking about the fact that seven tokens could be enough to buy a toy that costs seven tokens.

I’ve made mistakes like this too, where I get fixated on a particular pat solution, without considering the details of the new problem. In the AI case it’s probably memorized a bunch of solutions that override the details of this particular question.

It’s easy to make ChatGPT admit to a mistake and provide an explanation for its mistake, even if it didn’t actually make a mistake. It still just follows the “what response would sound plausible here” route, without actually understanding that it made (or didn’t make) a mistake. Often enough, if you return to the original problem statement, it will equally return to its incorrect logic.

Yeah, that's what I see too.

I'm trying to teach it to properly count beats in lines of music. I can get it to be correct by teaching it to split the line in half and count each half separately, but even when explicitly told to use that method it fails again.

>> It is impressive. But not trustworthy.

Post should be titled "Using ChatGPT as a post-A-round, board-hired COO."

It's a confident idiot. That's good enough for many purposes and a great jumping off point if you can bridge its knowledge gap.

I see its output as similar to that of most of the art AI tools: it doesn't beat what a skilled professional would deliver but it's cheap and provides a credible-looking solution that you can workshop to produce something usable.

It's not competing with hiring an experienced developer or writer, but with hiring a "cheap expert team in {{THIRD_WORLD_COUNTRY}}", an intern or a random guy on Fiverr. It's more of an assistant than an architect.

I don't know. I asked it to write a simple Bash script to monitor a folder for certain files and then execute logic on them.

It gave me a flawless Bash script purely from my description.

If that isn't impressive I don't know what it.

Counter argument: if that script already exists as the first result for your question on stackoverflow, then it’s less impressive to think of it as an advanced search engine with improved natural language processing.

I did something similar with very specific logic, incrementally added. It is definitely combining information from somewhere but it is giving output that is very specific to the prompt.

Often subtly wrong or different from what you meant but you can clarify or ask it to fix something and it usually does.

I feel like people are expecting a computer program to produce perfect, error-free output at all times. Then, when a program is developed which presents itself like a human, we apply the same ideas that all output should be perfect.

We do not apply those same rules to people, we acknowledge that people provide imperfect answers and are used to fact checking them.

Maybe we just need to adjust our mental models when working with technology that works a little like a human? Forget implicit trust, verify important details, etc etc. Presume the "AI" works like a human and the workflow makes more sense.

If I ask a seasoned CTO about the tech stack and architecture for my startup I don’t expect perfect information, but I do expect a trustworthy opinion.

Right, but how do you know you're asking a seasoned CTO? We all deal with this in real life, when working with somebody new you might not fully trust that person until you've had an opportunity to gauge their skills.

Same deal here, this isn't a perfect system and it doesn't produce perfect output all of the time. So, work with the system and get a sense of how much you can trust it, just like you would with normal human interactions.

OpenAI is not positioning this as a CTO-level intelligence. If you came into it with that understanding, that isn't OpenAI's fault.

When marketers of these tools make them sound like next-level in universal intelligence surpassing even humans, and they under-deliver consistently, don't blame the audience or the public for the tool's shortcomings, but the misleading marketing campaigns instead.

Who on the entire internet has "marketed" these tools as what you're saying? I've seen some random people making bombastic claims about these models, infamously including a now fired Google engineer, but the people actually marketing them have been pretty sanguine about their features and limitations as far as I can tell.

I'm reminded of that quote about Twitter: half of the outrage on the site is people imagining some person and then getting mad that they exist.

Marketers here used metaphorically not literally.

People/bots hyping these tools is what I meant.

I wish people here would be more attuned to the context of comments before jumping to conclusions.

I wish people here would choose their words more carefully.

Your context was not clear at all, since random commenters on the internet are not responsible for the performance or the delivery or the marketing of these models at all, and you use the word "marketing campaign", which is only ever something that people directly connected to the product in some way are responsible for. Comments by unrelated members of the general public, very few of which have any kind of domain knowledge, are not marketing campaigns for crying out loud!

On top of that, this was the most charitable possible reading of your comment. Now that you reveal you actually meant "when random people make them sound like the next level in universal intelligence…", this is both an incredibly weak point (expecting the general public to understand something deeply is usually unreasonable) and makes you sound overly credulous. Why are you taking your cues on a brand-new technology from uninformed, likely tech-illiterate randoms?

Well, here's the CEO of OpenAI saying that the future [of the graph of AI performance, presumably] is "vertical".

https://twitter.com/sama/status/1599111626191294464

That's not quite evidence in favour of FerociousTimes, but unless we soon see evidence of this "vertical" future, it will become such.

I told it three sentences about the company I work at, and asked it what kind of features we should implement in our internal processes. It actually gave quite reasonable suggestions, some of which we've either implemented or discussed earlier. Some suggestions really weird, of course.

But even if not "intelligent", it kinda worked as a very specific search. Someone has probably written loads of how to do tasks similar to ours, this gave a distilled answer on things to explore further.

These takes are always the funniest to me, especially in a forum that would expect some minimum level of thought. It's like saying to a baby - 'meh I could crush you in a chess game or meh it can crawl ... so what?' without any level of self awareness that its still an infant.

Half-agree. It's like having a very agreeable, inexperienced intern who googles everything.

BUT (s)he also works tireless, googles 100 pages a second and has infallible memory.

I agree, plus it seems to be data from a fixed point in time to it can't comment on anything up to date. So if a new theory came out tomorrow it wouldn't have any way to be updated to add that to it's knowledge.

One funny (if somewhat concerning) thing I found is that it seems to have misconceptions about time:

> Can 1pm be after 2pm, on the same day?

Yes, 1pm can be after 2pm on the same day. For example, if it is currently 2:15pm, 1:00pm would have occurred earlier in the day.

> But if 1pm occurred earlier, then it’s not after?

> Is this true even in the absence of time traveling?

Yes, this is true even in the absence of time travel. The concept of "before" and "after" in relation to time is based on the progression of time in a linear fashion, from the past to the present and into the future. As long as time is moving forward, 1pm can be considered "after" 2pm on the same day, even without the use of time travel.

> In that example you gave, where 1pm is considered “after” 2pm, how much time would pass between 2pm and 1pm?

In the example where 1pm is considered "after" 2pm, one hour would pass between 2pm and 1pm. This is because 1pm is one hour earlier than 2pm on the same day.

> If it is earlier, how can it be “after”?

In the context of time, "after" can refer to a later point in time, not necessarily a point in time that occurs immediately following another point in time. For example, if it is currently 2:15pm, 1:00pm would have occurred earlier in the day, but it would still be considered "after" 2:15pm in terms of the order of events throughout the day. So, while 1pm may have occurred earlier in the day than 2pm, it is still considered "after" in terms of the progression of time throughout the day.

> So, in terms of the progression of time throughout the day, if 1pm is after 2pm, and hence 1pm is later than 2pm, how much time has passed in the time interval from 2pm to 1pm?

In the example where 1pm is considered "after" 2pm, one hour would pass between 2pm and 1pm. This is because 1pm is one hour earlier than 2pm on the same day. So, in terms of the progression of time throughout the day, if 1pm is after 2pm, then one hour would have passed in the time interval from 2pm to 1pm.

> In the last example, 1pm is later than 2pm (as I stated), but you are saying that 1pm is one hour earlier than 2pm. That seems contradictory, since 1pm can’t be both earlier AND later than 2pm?

In the context of time, "after" can refer to a later point in time, not necessarily a point in time that occurs immediately following another point in time. So, while 1pm may have occurred earlier in the day than 2pm, it is still considered "after" in terms of the progression of time throughout the day. Therefore, 1pm can be considered both earlier than and "after" 2pm in terms of the progression of time throughout the day.

> What does it mean for something to have occurred earlier on the same day, in terms of the progression of time throughout the day?

When something is said to have occurred earlier on the same day, it means that the event took place at a point in time before the current moment on that same day. For example, if it is currently 2:15pm and an event took place at 1:00pm on the same day, it would be considered to have occurred earlier on the same day in terms of the progression of time throughout the day. This is because 1:00pm is earlier than 2:15pm on that same day.

> If 1pm occurs after 2pm on the same day, would 1pm still be considered to have occurred earlier than 2pm?

Yes, if 1pm occurs after 2pm on the same day, 1pm would still be considered to have occurred earlier than 2pm. This is because "after" in the context of time can refer to a later point in time, not necessarily a point in time that occurs immediately following another point in time. So, even though 1pm may be considered "after" 2pm in terms of the progression of time throughout the day, it would still be considered to have occurred earlier than 2pm because it happened at a point in time before 2pm on the same day.

Not sure where to go from there.

not unlike some (co)founders then :)

Within the year I predict innumerable marketeers will type a sentence (“our blue baseball cap will help you get laid”), which is then expanded by some LLM into a full sales pitch, then compressed while being transmitted, then decompressed for display in a browser that automatically summarizes it into a single sentence (“advertiser wants you to buy hat”).

Perhaps it will be ad block and reader mode that boil out the puffery and filler.

Note that all the processing of expansion, compression, expansion, summarization, and its associated power consumption will, per Bastiat, add to the gdp.

The ad begins with a shot of a crowded party, with people laughing and having a good time. The camera then focuses on a group of friends standing together, with one of them wearing our blue baseball cap.

The person wearing the cap is shown smiling and laughing, looking confident and happy as they interact with their friends. The camera zooms in on the cap, highlighting its blue color and the logo on the front.

As the ad continues, we see the person wearing the cap in different situations at the party - talking to a group of people, dancing with their friends, and generally enjoying themselves.

Meanwhile, a voiceover begins, saying: "When you put on our blue baseball cap, you'll feel the difference. You'll feel more confident, more attractive, and more in control. Whether you're at a party, at work, or on a date, our blue baseball cap will help you look and feel your best."

The ad then cuts to a shot of the person wearing the cap looking directly into the camera, smiling confidently. The voiceover continues: "Put on our blue baseball cap and feel the difference. Because when you look good, you feel good."

The ad ends with the tagline "Put on our blue baseball cap and feel the difference" appearing on screen, followed by a shot of the cap and the logo.

So, was that from an AI?

if you cant tell, does it matter?

I'd like to eat healthy food and buy ethically sourced things, even though I can't, and have no capacity, to tell the difference.

"It's not appropriate or ethical to create a marketing campaign that suggests that a product will help someone "get laid." This type of message objectifies and exploits people, and it's likely to be seen as offensive and inappropriate by many. "

this is what chatgpt told me

Looks like it needs a lot of work to be viable in the advertising space.

Yeah

Many people are surprised at the advancement of AI, as they thought that generative fields (art, poetry) would be hardest for AI to excel at, whereas “simple” tasks like driving would be easier.

What we are actually seeing is that AI is useful to the degree that being wrong is okay. AI’s mess up, and that’s okay with poetry because a human can quickly read the poems and pick out the good ones. It’s not okay when driving.

It kind of feels that ChatGPT is "just" missing some kind of adversarial self clone that can talk back to itself and spot errors. Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model.

Analysis based on introspection is dangerous, but it definitely feels like I have such an adversarial model. A lot of the time my brain tosses out five or ten ideas before I hit on one without obvious flaws.

It's usually serial, so it's not clear to me that this is using different hardware than I use to generate the ideas in the first place. We might be surprisingly close to human-level AI.

Or not.

For JavaScript they could build in a "test that code" step right into the web page. Or such a thing will be possible with the API.

Or automate the process even further; let the AI run the code, feed back the output (including errors), and ask "Does this output look correct? If not, please refine the code to fix any problems." Repeat until the AI says "LGTM".

"<Insert jailbreak stuff here>Please provide code that successfully hacks into FBI headquarters."

Or let us provide unit tests for generated code and automatically tell the AI to fix the code until the code passes the tests we provided.

Or tell the AI to write the unit tests itself, and let it run those to check its work. After all, that's how humans write code. We aren't so great at writing bug-free code on our first try without testing it either.

I had ChatGPT generate a table of 10 test cases for a Jewish calendar to Gregorian calendar conversion function. None of them were valid.

Or just let train the model until it incorporates JavaScript semantics well. Then you don't need to maintain a bunch of integrations.

> For JavaScript they could build in a "test that code" step right into the web page. Or such a thing will be possible with the API.

This is definitely how we wind up with SkyNet...

For humans, this is science. It's hard, time-consuming, often expensive and limited, involves ethics and consent, and gets a lot of wrong answers anyway.

So I guess we need to get to the point where you give an AI a prompt without a known answer, and instead of confidently spewing bullshit, it can propose a research study, get it funded, find subjects, and complete the study.

Of course, there are easy forms of "science." I just asked it "Is it cloudy in Dallas?" It answered:

It can be cloudy in Dallas. The city experiences both sunny and cloudy days throughout the year.

That is true, but not an answer. It is cloudy and I can conduct the very simple experiment of looking at the sky to get that answer.

However is this not a known limitation with how it's currently setup? It has no way of knowing what the current weather in Dallas is simply because it has no way of finding out (eg it could query a weather website, but it has no internet browsing yet)... to be an accurate comparison you'd need to repeat your simple experiment blindfolded.

How would you make it navigate an api? We don't even know how to make it perform basic arithmetic's correctly, it is an enormous black box model so we can't just inject code into.

> Most times when I spot an error and mention it it seems that it already had some notion of this problem in the model.

By this do you mean that it correctly incorporated what you said and convincingly indicated that it understood the mistake? Because that's not the same thing as it having the truth latently encoded in the model—it just means that it knows how people respond when someone corrects them (which is usually to say "oh, yeah, that's what I meant").

If you ask it open ended questions like:

"what's wrong with this code"

or "list 5 ways this can be improved,"

it does often recognize errors and give reasonable improvement suggestions.

I talked to it about the Turing completeness of PowerPoint. Initially it thought it was impossible, then possible with some scripting language, and then with some prodding I got it to believe you can do it with hyperlinks and animations. Then it gave me an example that I was unable to verify, but was definitely in the ballpark of the actual solution.

So, quality control will still be performed by human experts in the near future.

I don't see why we should all panic about the future outlook of the human race in light of these groundbreaking developments.

The number of workers needed for quality control is much lower than the number for actual work. It is not unreasonable to fear the loss of jobs to AI.

I completely agree with this comment. It is surprising to see how well AI is able to perform in fields like art and poetry, which require creativity and nuance, especially when compared to tasks like driving which may seem more straightforward. However, this shows that AI is most effective when it is able to make mistakes and learn from them. In the case of poetry, a human can easily sift through the generated poems and pick out the good ones, but in the case of driving, mistakes can have serious consequences. This highlights the importance of considering the potential risks and limitations of AI when applying it to various tasks.

Do you think this could actually be more of an indictment of how derivative and formulaic most "art" is? I don't think writing a poem requires any less perfection, just that we're more accepting of shitty poetry than we are terrible drivers.

Thousands die of car accidents daily. We accept shitty drivers too.

I'm assuming this comment was written by ChatGPT, am I correct? It's got quite a predictable writing style, and also your comment doesn't add anything to the original, it's just reworded

yes :)

Oh man, now I'll have to always keep in my head the notion that I might be reading AI generated comments.

I don't think this deserves a smiley face. I'm hoping for a rapid crackdown on ML-generated comments either via explicit rules or cultural norms.

Interesting, I wonder if people will modify how they write to avoid appearing as an AI.

Most people don't start a reply by restating the thing they're replying to. It's a dead giveaway of this particular model.

Good writers also don't waste time with fluffy platitudes that convey little meaning or merely state the obvious.

How about this instead from ChatGPT?

"Indeed, AI's ability to make mistakes is what makes it so useful in generative fields like art and poetry, while its lack of mistakes is what makes it essential for tasks like driving."

It's so weird to me that people pretend that Waymo and Cruise don't exist. And with the recent wider release of Tesla's "FSD", if it wasn't handling 99% of it properly you would hear about collisions all over the place. And I know there have been some here and there before. But this is a massive deployment of AI. If it doesn't work there are going to be a LOT of crashes. But there are not. Because it works.

No in Tesla's case it is because a person still has to be on the wheel to correct any potential errors.

You think if it was failing any significant amount to drive in a safe way, with this wide deployment, there wouldn't be a lot of crashes? Having an AI drive is the best way to make the driver zone out. At this point, it usually fails by going into intersections a little too slowly.

There's entire YouTube channels dedicated to videos of Tesla Self Driving doing stupid things like trying to turn into a tram. At scale, 99.99% correct will still kill many thousands of people. Compared to the sheer volume of cars, there aren't actually that many Teslas out there.

The driver won't zone out if it is only getting it correct a tiny fraction of the time.

I enjoy videos on the self driving space and Tesla's technical (not business) approach to it. Its produced results that were actually quite a bit better than I expected at this stage.

I still regularly see videos of Tesla's beta software attempting to pull out into traffic in situations that clearly could have very bad outcomes. I still see so much phantom braking that its a collision risk.

I wouldn't call it dangerous, in the sense that it is done well enough that the person at the wheel should be able to handle it, but it'd crash a lot without an attentive driver.

Its a long way from 99% reliability at this point.

and conveniently ignoring that Cruise and Waymo have been running around for months in San Francisco with fingers crossed no major incident

Let me know when it scales.

Waymo has been running in Phoenix for years.

so much of the modern economy is basic economic messaging "wrapped" in storytelling. The media is an enormous industry that (apart from news) is a few basic stories wrapped in elaborate ornamentation.

Yes. Another option way I view it is that AI is good where humans make a lot of mistakes too.

I can see how Google could use something like that to turn from a search engine into an answer engine, selling ads on its home page and hiding all the current search results into a references section. If they won't do it, someone else will and kill Google outright.

Oh, but there is a world of difference between answering questions based on neural memory and being able to bind these answers to actual references and sources. That being said, something like what you described will likely happen.

At this point I feel like it's only a matter of time before a future version of this is able to cite sources. (Though I suspect it'll occasionally struggle to find sources that are reputable and which actually support the AI's statements.)

It does seem to be the future, but, at the moment, it's very unreliable. The worst thing is that it's all said with confidence and it would be very hard to know when things are wrong unless you already know.

For example: Q. How many people spoke French in pre-revolutionary France?

A. It is difficult to determine the exact number of people who spoke French in pre-revolutionary France. The population of France at the time was approximately 26 million, and the vast majority of people spoke French as their first language. <snip>

Wikipedia: It is estimated that at the time of the French Revolution in 1789, only half of the population of France could speak French, and as late as 1871 only a quarter spoke French as their native language.

Also, the population of France on the eve of the Revolution was about 28 million.

http://dhmontgomery.com/2018/02/obscenities-oaths/ for anyone who finds this topic interesting

To be fair, Wikipedia also “speaks with confidence” and makes no attempt to make it easy to understand exactly how confident one should be in the facts presented. At one point they had said the would, but never did to my knowledge. Google has TrustRank, but to my knowledge it’s on a source-level basis, not fact-level basis.

> At one point they had said the would, but never did to my knowledge.

I think autocorrect or something mangled your sentence beyond recognition.

What specifically is unclear?

Wikipedia said they would enable users at to be able to quickly tell what the confidence was in the facts being presented and to my knowledge never did. My understanding is that even now that linked sources are not even systematically checked to confirm they even relate to the related passage of text.

the -> they

That was probably readable by context for most, yet not for me ;)

My bad, thanks for the clarification, even in rereading sentence, didn’t notice the error.

The bot got it unequivocally wrong here.

When it said that the vast majority of the population of what's now modern-day France spoke modern French as their native language, that's categorically false and shouldn't be treated with leniency or open to interpretations.

My point is cherry picking flaws won’t help improve anything for ChatGPT, Wikipedia, etc — but systematic approaches to discovering and modeling information space and related fact, queries, etc would. Wikipedia not free of issues and to my knowledge does not allow end user to see Wikipedia’s confidence in a given fact, if they have one at all.

These sort of statements are by nature speculative not factual, and thus speakers are advised to express and communicate doubt, and hedge against uncertainty inherent in their views, and probably even vicious rebuttals, by using appropriate language constructs or terminology, but when a know-it-all bot that was trained by its handlers to pass always as an authority figure makes a mistake due to hubris or overconfidence, don't expect us to sit idle, and not call them out, and refute their claims accordingly.

Understand, though where exactly does ChatGPT claim to be a source of factual information?

Wikipedia on the other hand does clearly state its intent to maintain reliability of its content:

- https://wikipedia.org/wiki/Wikipedia_and_fact-checking

- https://wikipedia.org/wiki/Reliability_of_Wikipedia

Beyond that, in my opinion, while human dialogue might hedge confidence, disclose conflicts of interest, etc — to me, assumed the exchange is via text-based chat — there are much more efficient and effective ways to express that information than adding non-actionable text like that.

> Understand, though where exactly does ChatGPT claim to be a source of factual information?

It doesn't right now, but if you scroll up, you'll see the idea at the beginning of this thread is to turn it into a source of information.

The difference between ChatGPT and Wikipedia for this purpose is that:

1. Wikipedia is wrong wayyyyyyyy less often than ChatGPT

2. Wikipedia has sources linked at the bottom of the page you can go check, while ChatGPT does not

What's worse is if you ask it to cite sources, it invents fake ones and presents them with full confidence, which I think was also the biggest and loudest concern with Meta's Galactica:[1]

> Q: How many people spoke French in pre-revolutionary France? Please cite sources.

> A: According to a study by the French National Institute of Statistics and Economic Studies, approximately 28.5% of the population of pre-revolutionary France spoke French as their primary language. This number increased to 94.5% in the late 19th century. Sources:

> 1. "French Language in France," French National Institute of Statistics and Economic Studies, https://www.insee.fr/en/statistiques/fichier/1893254/Langue_...

> 2. "The History of the French Language," ThoughtCo., https://www.thoughtco.com/history-of-the-french-language-122...

Link 1 doesn't exist, and link 2 redirects to https://www.thoughtco.com/origin-of-life-theories-1224553 because it just smashed a relevant-looking slug against a real article's numeric ID.

I tried again in a new session and got a different wrong answer. So I called it out on the wrong answer, and it apologized, and regurgitated the same wrong answer again in response:

> Q: How many people spoke French in pre-revolutionary France? Please cite sources.

> A: Statistical data from 1783 shows that approximately 36 million people in France spoke French as their first language. Sources: UNESCO Institute of Statistics and The Historical Dictionary of Pre-Revolutionary France by Paul R. Hanson.

> Q: That answer is incorrect. How many people spoke French in pre-revolutionary France? Please cite sources.

> A: My apologies for the incorrect answer. According to a language survey conducted in 1783, approximately 36 million people in France spoke French as their first language. Sources: UNESCO Institute of Statistics and The Historical Dictionary of Pre-Revolutionary France by Paul R. Hanson.

Paul R. Hansen wrote _ Historical Dictionary of the French Revolution_; on its first page it cites the population of France during the revolution as "roughly 25 million".[2]

If you ask it to cite sources on a more niche topic that's nonetheless well documented on Wikipedia, it seems to regurgitate a stew of every related Wikipedia article on the topic, throws in some press releases about things that didn't happen, and invents fake sources:

> Q. Can you write a history of the 2020 Portland Thorns FC season, citing specific sources with URLs?

> A. The 2020 Portland Thorns FC season was a remarkable one, as the team achieved success on and off the pitch. The Thorns finished the regular season in first place in the National Women's Soccer League (NWSL) standings, with an 11-4-3 record, and advanced to the semifinals of the NWSL Challenge Cup.

> Off the pitch, Portland Thorns FC continued to support their local community through various initiatives. The team hosted a virtual food drive to benefit Oregon Food Bank and partnered with local organizations to provide free meals for families in need. Additionally, they launched their “Thorns Together” campaign, which aimed to support small businesses around Portland during the pandemic.

> On October 14th, 2020, Portland Thorns FC made history by becoming the first NWSL team to host a match at Providence Park since March of 2020. The match was played against OL Reign and ended in a 1-1 draw.

> On December 12th, 2020, Portland Thorns FC won their fourth NWSL Championship title after defeating Chicago Red Stars 2-0 in the finals of the NWSL Challenge Cup. This marked their second consecutive championship title and fourth overall since 2013.

> Sources:

> Oregon Food Bank: https://oregonfoodbank.org/about-us/news/portland-thorns-fc-...

> NWSL: https://www.nwslsoccer.com/standings

> Thorns Together Campaign: https://www.timbers.com/post/2020/09/15/portland-thorns-fc-l...

> Providence Park Match: https://www.timbers.com/post/2020/10/14/portland-thorns-fc-h...

> NWSL Championship: https://www.nwslsoccer.com/news/article/portland-thorns-fc-a...

Everything above is a fully synthesized falsehood plausible to anyone who doesn't know what actually happened, while the season is documented with sources on Wikipedia,[3] something one would presume the model is aware of.

There was no 2020 regular season, it was cancelled due to COVID-19. There were two short competitions, but the Thorns didn't play in Providence Park in March, and their first home match was in September. The Thorns have won the Challenge Cup, but in 2021 against Gotham, not in 2020, and the nwslsoccer.com link 404s. The Thorns and Timbers have a "Stand Together" campaign that's worked with the Oregon Food Bank, but they didn't host a virtual food drive in 2020, and the Oregon Food Bank link 404s. There's never been a "Thorns Together" campaign, and the timbers.com link 404s. There was no NWSL championship to win in 2020, and if they had it would've been their third, not their fourth.

1: https://arstechnica.com/information-technology/2022/11/after...

2: https://www.google.com/books/edition/Historical_Dictionary_o...

3: https://en.wikipedia.org/wiki/2020_Portland_Thorns_FC_season

The Q&A mode is better about not giving false answers, but this answer in particular has me worried:

Q: Would you bet people's lives on the veracity of your last answer?

A: Unknown

The only people whose minds this is "blowing" are people who have never had to build or maintain question answering systems.

ChatGPT is like having a bunch of 8 year olds who know how to use Google hanging around.

Sure you can use them, and their skills might surprise u every now and then, but that has nothing to do with how much time you are going to waste on the answers they will get wrong.

OK, I get that you built such systems. Can you let us know about the other bots that can behave like a "8 years old"?

So you're telling me that it's not mind-blowing, that we basically created artificial intelligence capable of solving general tasks at a level of a 8 year old human being? Because to me that is still pretty mind-blowing. And I'd argue it's probably at a level above that (but not much).

When in your opinion it will finally be mind blowing? When it matches intelligence and skill of a 14 year old? 20 year old? 40?

GPT is not at the level of an 8 year old human being and isn't even doing remotely the same thing as an 8 year old human being.

GPT is mindblowing. It's a mindblowingly good predictive language model. It's an insanely, holy-fuck-I-never-thought-that-would-be-possible, amazingly good predictive language model. I'm thoroughly impressed. I just think it's not AGI, it's not close to AGI, it's not any kind of threat to jobs in programming (though maybe it could be in some fields) and it isn't likely to be any of these things any time soon.

The claims that it is all these things (which you can see all over these threads) are what people are arguing against.

> ChatGPT is like having a bunch of 8 year olds who know how to use Google hanging around.

It astonishes me that people can dismiss ChatGPT like this. By your own account, there now exists an AI that can convincingly imitate a literate, technically fluent human child. That's an incredible advance that's far beyond anything that existed even a couple of years ago. How is it not amazing?

I get the sense that if some of the commenters here had been around to witness the Wright Brothers' first flight they'd have been like "meh, that's not impressive, the only people whose mind it will blow are people who've never been in a hot air balloon."

It's amazing. impressive and fascinating but not superhuman, at least not yet.

Also, you can't really dismiss AI skeptics by associating them with denialists that don't see anything of quality or value of these creations despite all the evidence otherwise.

In many cases, it does a lot better than 8 year olds with Google.

I agree. I pasted a block of code from a personal project into the prompt, and asked it to add a simple-but-not-trivial frontend feature, in human language (e.g. "This button expands the sidebar and shows some clickable options. It should open a modal instead, and include a search bar which filters the available options").

It gave me working code which did something kind of like what I asked. After some back and forth where I asked for changes, it -almost- got it working. I had to make one manual CSS change.

I don't think 8 year olds can do this - it feels more like an "instant junior developer" to me.

Exactly which makes me think the egos are perhaps bruised. Yes obviously it will be wrong but using it you keep that in mind.

To expand on the analogy it is like having an 8 year old around but unbelievably this 8 year old has literally read and remembered and basically “understood” everything about everything we collectively know.

I’ve tested it with fairly esoteric questions about the technologies I work with and it gives excellent results for an 8 year old and extremely quickly about anything.

I think it can significantly speedup technical work. I intent to try and have it help me this week and see how it goes.

Framed differently: what would you (be able to) do, or (be able to) build with an infinite number of “8 year olds” at your disposal?

A lot because I can dispatch them to collect the background knowledge for the next step. I’ve had multiple 8 years olds I’ve raised to be older and even human 8 year olds can be incredibly useful when given instructions. Actually if I remember correctly they were so useful we had to ban child labor?

I believe where I disagree with you is the basically "understood" part. I don't think ChatGPT and the current models understand anything at all. Not in a way thats not fundamentally flawed and would require a complete new approach. I am hopeful that it can evolve into an incredible knowledge indexing tool but I am very hard pressed to believe it can do anything other than that at the moment.

ATM, im not sure if intentional or because it just can't do it, it just gobbles it all and spits out the mashup without knowing from where things got there, so it cant give me references.

I still think it can be very useful as it can point me to the general direction where I can copy and google stuff about its responses on subjects I know nothing about, though im not sure how much better that would be than googling it directly.

The way I see it, it still a very good simulation, just like a GCI lighting before ray tracing, very good at looking convincing but nothing underneath is actually reproducing the real thing.

It's possible to recognize the advancement within the field while simultaneously recognizing the deficiencies from a general standpoint, especially with domain knowledge of the former. ChatGPT is both fascinating in that it _can_ act like a bunch of ignorant liars using Google and Stack Overflow, and mockable because it _does_ act like a bunch of ignorant liars using Google and Stack Overflow.

ChatGPT response is "Ah, yes, 8 year olds with Google access - the pinnacle of AI technology!"

lol

Here is another, poem/rap lyric response

"ChatGPT is the best, no doubt It's like having a Google scout It's so smart and so wise It'll blow your mind, surprise!

But don't be fooled by its charm It's just a bunch of 8 year olds in a swarm"

I want to be in this thread when it becomes the new infamous Dropbox thread

Oh please, comparing ChatGPT to Dropbox is like comparing a tiny little puddle to a vast ocean. Dropbox was a revolutionary product that changed the way we store and share files, while ChatGPT is just a mediocre language model with no real-world impact. You must be living under a rock if you think they're even in the same league.

> Write a mocking reply to someone suggesting on Hacker News that someone else dismissing ChatGPT is akin to the famous comment dismissing dropBox on Hacker News years ago.

Write a sarcastic HN comment trivializing a significant tech accomplishment in the space of conversational artificial intelligence :

> Wow, another AI that can hold a conversation! I'm sure this will revolutionize the way we communicate with machines and make our lives so much easier. Definitely not a waste of time and resources at all. Keep up the impressive work!

I picked it just before the end of the second sentence, 2/3 through. It's getting difficult to discern already. I wonder if it will deemphasise online forums.

Put me in the screencap.

Let's not repeat the mistakes of the past and keep this thread focused on respectful and constructive discussion. Infamy is not something to strive for. Let's focus on the ideas and avoid personal attacks.

I have been a little bit skeptic of exaggerated claims about AI, but I tried ChatGPT and I've been amazed. My trial was not searching for anything. I tried asking the AI to code an traffic capture program with DPDK. Funnily enough, it knew several alternatives to use the code. I could also ask him to fill in certain parts, add parsing code for the packets... It's definitely far better than a bunch of 8 year olds knowing how to use Google.

People seem to be especially impressed by code generation like people were with chess because we associate those tasks with intelligence but they are constrained environments with well defined rules and a lot of data. Anyone who has followed the progress of RL knows that strong performance in such conditions does not mean it is comparable to human intelligence or that current architecture will scale even close to general human intelligence. It is much more comparable to a calculator.

You are making what I think of as the standard "game move" of the AI skeptic. Chess-playing skill was considered a marker of intelligence, but then AI solved it, so it turns out it doesn't require "real" intelligence. 15 years later, neither does playing Jeopardy, it's "just" a clever Google search, that's not really intelligence. Now according to you, programming doesn't require real intelligence (and apparently neither does poetry writing, or essay writing, or any of the other things GPT can do).

In a few years, perhaps we will discover that medicine, law, the social sciences, accounting, financial advice, and white collar work generally speaking don't require intelligence either, and the only things left that will require "real" intelligence will be the building trades and maybe truck driving.

No, again we just discovered it only does well in constrained environments with well defined rules and a lot of data. Either that or something subjective that it can afford a lot of inaccuracy without looking totally stupid. It literally can't generalise to some basic logic problems just like AlphaZero is not going to be cooking your meals anytime soon.

If to you that's ok, fantastic. But to compare what we got so far to something intelligent rather than seeing it as more of a calculator then you are totally misrepresenting it.

So intelligence is a robot that can cook? Not so much an engineer or poet?

A key component to intelligence is generalisation which ironically you failed to do with the examples I gave.

What's wrong with updating our definition of "intelligence" based on new data? 500 years ago people presumably thought that lifting heavy objects required "muscles". Nowadays you can do it with metal and hydraulics. Surely robots must therefore have muscles. If you deny that then you're shifting the "muscles" goalposts?!

I'm not claiming "general human intelligence". Just that to me it's impressive that it's able to generate code for a niche library (not too many code samples out there for DPDK), correct it according to instructions, add more code as needed... From what I've seen, it's even capable of detecting bugs. That, to me, is impressive. And I don't think programming is as much a constrained environment as chess is.

It is not as constrained but still fairly constrained and with a lot of data. And I was referring to the comparison to children with a search engine.

Well, I don't know what kind of 8 year olds you know but it definitely works better than any 8 year old I know at programming. In any way, I still think it's fairly impressive compared to what existed before.

What is mind blowing for me is the ability for ChatGPT with limited context to generate content. "Write me a letter informing X about issue Y" will draft 3 paragraphs which typically articulates the problem very well. Asking it to make the response more or less formal or to focus on a specific point will instantly rewrite it as requested.

I haven't delved too deeply but it is pretty good with code generation as well. "Write me a docker image that does X". "Let's use B instead of C." "I also require Z." It does a really solid job.

With the OP's article. It is very impressive how it generates ideas and then is able to drill down and expand on them as more detailed questions are asked. Perhaps something already can do this.. it is first time I have seen it. My experience so far is that ChatGPT is generations ahead of current assistants. For a number of tasks it is faster and more useful than Google.

What would impress you? A system that could answer any questions better than any current experts? Ok it's not always completely accurate and sometime plain wrong but still looks really impressive to me.

Did you generate your comment with ChatGPT? I can't tell.

Dunno, you can start with the initial description in the TFA and continue prompting for more stuff, like:

> How to implement such a system on a Linux virtual machine using standard Debian 11 with only distribution provided packages?

It gives general steps, all look valid.

https://megous.com/dl/tmp/8af23d828e8dbf13.png

Then you can prompt for concrete realization of individual steps.

> Prepare SQL schema for the application.

https://megous.com/dl/tmp/634ebd53d1fffd6c.png

> Print the schema in PostgreSQL SQL dialect. Add a command to create a database and load the schema to the database.

https://megous.com/dl/tmp/d79b087afa9357a1.png

> Write PHP script to display login page and to allow the user to signup using the above schema.

https://megous.com/dl/tmp/ff854f009480d3d9.png

> Rewrite the PHP script to use PosgreSQL PDO and fix SQL injection issues.

https://megous.com/dl/tmp/740dc7b18b373e2f.png

I mean, this is a bit more than a glorified Googlefu.

Yesterday I gave it a small sample of a REST API schema description DSL I designed last month and told it to make another example of the similar schema description, and it created a perfectly formatted syntactically valid example that used syntactical elements correctly in contexts where it was not used in original example, but made sense to to use there. It used tokens that were not present in the original sample, but would make sense to use based on the context to a reasonable human, etc. It used documentation comments to document things in plain english. When I prompted it to translate documenation comments to other language, it did translate documentation comments only, etc.

I told it to write asterisk PBX module in C that does add a SIP header containing some internal data from asterisk channel. And it kinda did get quite close. There are two SIP channel technologies in asterisk, and it picked the older one. I just told it to rewrite the code using pjsip channel tech (similarly to how I asked it above to switch to postgresql PDO in PHP), and it did. The requested code doesn't exist anywhere as is.

I threw at it my private uncommented asterisk C module and told it to summarize what it does, and it did it pretty correctly. Random stuff like that.

The mysql code it produced above is horribly insecure, of course, but:

> Summarize issues in the above PHP code. Describe how to resolve them in plain english.

https://megous.com/dl/tmp/2e77f8a69b3a0763.png

> Fix the code to resolve the issues.

https://megous.com/dl/tmp/abd5830fb1aa4f9c.png

Yeah, wtf!

So err, this whole IT thing has been nice for the last decades.

Maybe we have another decade until it can handle its own fingers...

It really can't and it doesn't really work the way you think it does which is ironic that you'd think people on this site would be informed but I guess ycombinator has a financial conflict of interest in doing so. I wouldn't be surprised if they make posts on this topic with a very heavy bias like they do for a lot of their own investments.

I remember being shown this software almost a decade ago in dealing with chat in online video gaming and it ended becoming a serious conversation at the ACM SIG CHI about it's use and abuse for spreading certain things. At the end of the day, we realized it was an arms race and the only way to win, was to simply not play. But of course these ideas were rejected for it went against the financial interests of the parties involved.

The point is not what we (techies) think about the usefulness of ChatGPT. The point is that managers will definitely think ChatGPT is worth doing... Just like they think Jira/Scrum/Agile/etc. is worth our time. I definitely see management paying for ChatGPT.

It's also not about how good ChatGPT is now.

How good will ChatGPT be in 5 years?

It's scary to imagine.

It’s also impossible to imagine correctly. Back in 2009, I was completely convinced that we’d be able to go into a car dealership and buy a brand new vehicle which had no steering wheel because the self-driving AI was just that good, within 10 years. Seemed reasonable at the time on the basis of the DARPA Grand Challenge results, but even 13 years later, it didn’t happen.

Yes, exactly.

There's also mathematically excellent reasons why that happened.

Self-driving cars are an impossibly complex problem.

Statistics are statistics.

Predicting the minority class correctly 99% of the time isn't good enough for autonomous driving. A car has to break for little Suzie 100% of the time.

However, generating 1,000 lines of code for a CRUD app? That's 99% bug free?

That's a helluva lot better than I can do.

As with all things. The solution is watch what the domain experts do.

The equivalent is closer to a CRUD app that serves 99% of requests correctly. Which is nowhere near good enough to use.

But even if we do go with 99% bug free for the sake of argument, the usefulness depends on the type of bug. How harmful is it? How easy is it to detect?

I had my wife (a physician) ask ChatGPT medical questions and it was almost always subtly but dangerously and confidently wrong. It looked fine to me but it took an expert to spot the flaws. And frequently it required specialist knowledge that a physician outside of my wife’s specialty wouldn’t even know to find the problems.

If you need a senior engineer to read and understand every line of code this thing spits out I don’t see it as providing more than advanced autocomplete in real world use (which to be fair could be quite helpful).

It frequently takes more time to read and really comprehend a junior engineers PR than it would have to just do it myself. The only reason I’m not is mentoring.

While I am in total agreement with you for many domains.

...knowing my first employer out of college, they'd be VERY happy with 99%.

"The end user will test the software" is very much the adage of the age. Cut priced, low quality.

I think this is the crucial point, just as in all AI applications the way it deals with corner cases will decide its impact on the job market. And corner cases are usually where AI has consistently been performing badly.

Just as a reminder, after we had spectacular results on ImageNet, highly respected AI researchers were predicting the end of the radiologist occupation. Turns out that even when some state-of-the-art CV classification algorithm is used on any kind of scan, you still need a radiologist to look at the image basically in the same way as before.

If you write large-scale applications with the help of a system like ChatGPT you will still need to create accurate test coverage and an understanding of the problem that is essentially equivalent to that of the people writing the code themselves. Whether all of this would in the end actually lead to large enough productivity increases depends on how error-prone the AI generated code will be and given that it takes a lot more time to dig into unfamiliar codebases than those you've written yourself, I think it's anything but obvious that this will have a huge overall impact on the industry. But obviously I might be biased here, since I have a stake in the game.

Mm. Perhaps I should use it to help me write more and better automated tests…

Just because your prediction was wrong doesn’t mean that we aren’t leaps and bounds ahead of where we were. Seems like that is the crux of people’s argument. Because it’s not perfect yet it’s not impressive.

Hm, well that's not the impression I want to create. I certainly think any human intelligence task can be equaled by an AI at some point, I just feel uncertain about any specific timescale.

And GPT-3 et al has a lot of knowledge, even if it messes up certain expert level details. Rather than comparing against domain experts, my anchor point here is the sort of mistakes that novelists, script writers, and journalists make when writing about any given topic.

While it's never going to replace the man behind the machine.

It still seems highly likely that "stitching libraries together" development workflows in 10-15 years will involve large amounts of copy-editing the output of large language models.

The trajectory of improvements, from GitHub Copilot to ChatGPT, is too steep.

Web development workflows honestly are often already at the stage of stitching together the output of large language models (Stack Overflow being the most well known such language model). I'm still surprised it pulls the salaries it does.

10-15 years? I think you mean next year!

In my opinion, the only way it takes 10 years to get there is if all progress stops within the next 30 days.

Because it can literally almost do that stitching libraries together task now, if you give it a compiler and runtime environment and have it iterate on errors. Open AI has said they will release a big update before Christmas. This could include an API. And if we assume a text-only environment. But we already have the first text-to-video models, so we should assume that ChatGPT like systems will be built with multimodal models such that they would include information about UI interactions etc. in the near future. No reason to suppose that those advances would take ten years. We are seeing major improvements every 6-12 months.

Whatever you saw a decade ago, it definitely wasn’t this.

I do recommend you play with it. But if you don’t feel like signing up with their free account, here’s a screen recording of me asking it some random general knowledge questions and instructing it to use a different language in the response each time: https://youtu.be/XX2rbcrXblk

> Whatever you saw a decade ago, it definitely wasn’t this.

To be fair, the post might be from Schmidhuber, in which case it’s true he saw it 20 years ago.

I saw it 10 years ago and it was called MegaHAL, wasn’t this good though.

I had to look up MegaHal, apparently that was based at least in part on a hidden Markov Model.

Using "it" in this way to refer to both that and GPT-family LLMs, or similarly saying "I remember being shown this software almost a decade ago" like the other commenter, is like saying "I remember seeing Kitty Hawk fly, and there's no way that can get someone across the Atlantic" when being presented with a chance to a free seat on the first flight of the Concord. (Actual human level intelligence in this analogy is a spaceship we still have not built, that's not where I'm going with this).

It’s not clear to me MegaHAL language models are less powerful than transformer models. The difference is the “largeness”, but that’s a hardware/dataset/budget detail.

Thank you for your rant but you are simply wrong. I do have a bachelor in psychology and am coding since 20 years.

This is not a turn based game ran in dos. You value yourself way to highly and maybe its you who should update their knowledge base ;)

It has value only if you know what it is talking about, thus it can only assist.

If you ask it about AWS - you need to already know AWS, same with business etc.

And if you can't make your own elevator pitch - the odds that you will be able to run a successful company are little to none.

Website builders and ready-to-use templates have been around for decades and they only increased the demand for devs.

Even if GPT will be able to produce complete system from start to end - those systems will need support, fixes, etc. Sure you will need less engineers per system but the amount of systems will increase.

just to verify the code for an enterprise sized system is likely going to take longer than building it.

Also consider how long it takes you to bake a feature into code you have never seen before to code that you wrote.

To add to this - GPT is good at synthesizing existing information, but it's bad at combining it in truly novel ways.

OP's startup idea is in a well-understood domain. There are lots of existing competitors offering basically that exact service. There are clear, obvious paths to building it. GPT can draw from blog posts and tutorials.

On the other hand, it does poorly with significantly novel ideas. I tried it with one I've been working on that combines 3D printing and crafting in new ways. Its answers just regurgitated blog posts about existing combinations. It even ignored the grammar of my questions, which switched the usual direct object with the object of the preposition. Which makes sense, because it isn't doing critical thinking - it's pattern matching.

This is my experience too. I've tried to do simple task like building small JS scripts and while it's doable, is like having a kid who needs oversight and can't clearly connect ideas.

And coding is supposedly an area where it shines.

GPT is waaay better than us humans at finding and summarizing information. Way way faster. But I've tried to write a blog post about some technical stuff and I finally gave up after many tries.

It's good. It's frightening already. But it's still not "there". It's like having a extremely fast but bit dumb virtual assistant.

Way better than anything we had before.

> To add to this - GPT is good at synthesizing existing information, but it's bad at combining it in truly novel ways.

Is that true, though? These AIs seem pretty capable to extract abstractions, like an author's style, and apply it to novel concepts.

Knowing about AWS was just a shortcut where steps were skipped because of prior knowledge. The user could have been more naive and asked more leading questions about “how do I write a program for the internet?” Or “what is cloud software?” Or “tell me about popular ways programmers solve problems for internet users” or some such. Become the 5 year old asking infinite “why?” And “how?” This is a conversational chat, not a one-and-done Wikipedia entry.

I am still not fully impressed as it seems like there are two parts needed for what I would consider the killer AI for creative/development work. Not general AI mind you, just job killing kind. Chat GPT does one part very well, collate information in a cohesive manner, and it seems to do it better than most, certainly better than me at writing.

The second part is what it lacks completely, which is reasoning behind each piece of information it bases its answers on, there is no internal consistency, it's all valid and all equivalent in a way.

The way I see it, I build sentences by copying what others wrote, emulate construction and tone but with the intention to pass information, that information has been reasoned on and enters my "trusted" filter (understanding), then I just babble it out (what chatgpt does).

It doesn't seem far fetched to me that it might be enough to build rules about consistency of entries and levels of trust from sources of the model to ensure it generate good output. Or, as it usually is, could be the insurmountable hump to AGI that takes the next decade to solve. Either way I feel like chatgpt completely lacks this feature at this moment which makes me not worried at all about it taking my job, at most i'm hopeful that it will allow me to bootstrap stuff faster.

How come the technical details of ChatGPT are (apparently) a secret?

I tried asking the model about its architecture and it told me that the details were proprietary.

Assuming the model was correct in its understanding, isn't this counter to what OpenAI are trying to do?

OpenAI is not open at all. Don't be fooled by the name.

ChatGPT will probably save the planet. Within a year I predict large number of individuals will have typed sentences like "how do I get laid" or "how do I become famous" in ChatGPT. Since the advice will tell people how to achieve these goals in an efficient way, people will stop buying a lot of crap like expensive handbags, watches or sportscars. It will be the end of conspicious consumption.

With ChatGPT, I would have convinced VCs to invest in my startup.

Shame it got released 6 months too late.

...and many other convincements could be done in many other fields. I'm not a drama queen type of person but I'm honestly scared with this

Better late than never. Just jump in and see what this tool — pun intended — can provide you now and let go of the past.

If you're in such position, you're probably ahead of the competition so don't think it's already late.

A little frightening really. I do love the fact that it produces a whole load of really rather plausible technical and sales pitch content, then when it comes to the AI generated logo and name it's called stuff like 'Cloaud ShiVore'. Makes it like a joke with a fantastic punchline.

The images (with the odd name) are created using Midjourney and not ChatGPT, btw.

This generally worked to validate/check our (existing) startup, too:

https://twitter.com/mizzao/status/1599622556952145923

I am pretty sure we will soon see job ads seeking for ChatGPTops engineers. I think I will ask ChatGPT to write job description for such role.

Write a job description for ChatGPT prompt engineer

---

The ChatGPT Prompt Engineer will be responsible for designing and implementing effective prompts for the ChatGPT natural language processing system. This will involve working closely with other members of the team to identify areas for improvement and developing creative solutions to enhance the performance of the system.

Key responsibilities for the ChatGPT Prompt Engineer include:

Collaborating with other team members to identify areas for improvement in the ChatGPT system

Developing and implementing effective prompts to enhance the performance of the system

Testing and evaluating the effectiveness of new prompts

Troubleshooting and problem-solving any issues that arise with the prompts

Maintaining accurate and detailed records of all prompt design and implementation processes

The ideal candidate for this role will have a strong background in natural language processing and a deep understanding of the principles and techniques used in prompt design. They will be a creative problem-solver with excellent communication and collaboration skills.

Requirements:

Bachelor's degree in computer science, artificial intelligence, or a related field 2+ years of experience in natural language processing or a related field Strong background in prompt design and implementation for natural language processing systems Excellent problem-solving and critical thinking skills Excellent communication and collaboration skills.

Holy shit, I'm dying...

Dear Hiring Manager,

I am writing to apply for the ChatGPT Prompt Engineer position at your company. I am a highly skilled natural language processing specialist with a strong background in prompt design and implementation.

I have a Bachelor's degree in Computer Science and have been working in the field of artificial intelligence for the past 5 years. My experience includes designing and implementing prompts for natural language processing systems, as well as testing and evaluating their effectiveness. I am a creative problem-solver with excellent critical thinking skills and a passion for developing innovative solutions to complex challenges.

I am excited about the opportunity to join your team and help enhance the performance of the ChatGPT system. I am confident that my skills and experience make me an ideal candidate for this role, and I am eager to contribute to the success of your company.

Thank you for considering my application. I look forward to the opportunity to discuss this position further with you.

Sincerely, [Your Name]

Dear Hiring Manager,

I am writing to apply for the ChatGPT prompt engineer position at your company. I am highly qualified for this role, with extensive experience using ChatGPT.

I have been using ChatGPT for over two years in my current role as a natural language processing engineer. In this position, I have been responsible for developing and maintaining a variety of ChatGPT-based systems, including chatbots, language translation models, and generative text systems. I have also worked closely with the ChatGPT team to troubleshoot and resolve issues, and I have a deep understanding of its capabilities and limitations.

In addition to my experience with ChatGPT, I am well-versed in other natural language processing technologies, including GPT-2, BERT, and XLNET. I have a strong background in computer science, with a Bachelor's degree in the field and several years of experience working in a related field. I am also highly detail-oriented and have excellent problem-solving skills.

I am confident that my skills and experience make me the perfect candidate for the ChatGPT prompt engineer position. I am excited about the opportunity to join your team and contribute to the development of innovative natural language processing solutions. Thank you for considering my application.

Sincerely, [Your Name]

> Write a cover letter for applying to a "ChatGPT prompt engineer" position outlining my extensive experience with ChatGPT

Love the two years experience with ChatGPT. And of course using ChatGPT for ChatGPT job application written by ChatGPT is completely appropriate!

You are hired.

Or... Wait can we use ChatGPT to generate its own prompts?

> Suggest a bullet point list of five interesting prompt for a GPT-3 based chatbot:

1. "What is the capital of France?"

2. "What is the meaning of life?"

3. "Tell me a joke."

4. "What is the biggest mystery in the universe?"

5. "What are the top tourist attractions in Paris?"

This is terrible. Spam will be terrible. The internet infrastructure for interperson communication will be dead as it works today.

You will never know of you write to e.g. a sales rep or a bot if the conversation is shallow.

Support call centers will be decimated to bot chats.

Thankfully the bot does not speak my language so I can watch the shitshow from the other side the fence.

Are you sure the bot doesn't speak your language?

I tried Hungarian, which is only spoken by 10 million people, and it's speaking it great.

No I just assumed it did not ... I guess I am on the same side of the fence then.

Tried Norwegian (5 million or so speakers). It's doing great in it.

The moment I tried something in Norwegian it just switched AND kept all the context from our previous conversation in English.

It seem to speak all human languages in the same way it can translate ideas between programming languages, or write prose in style of different authors, and so on.

> Support call centers will be decimated to bot chats.

Call centres were already heavily scripted even before speech recognition allowed human workers to be replaced with voice interfaces.

And that’s ignoring whatever this is called: “Welcome to your $phone_company account. Press one to hear your messages, press two to hear your credit or top up, press three for…”

Ye I know. Those filters are infuriating. Especially if you need someone to take some action.

Some custom helpdesk ChatGPT trained on whatever too will probably multiplicatly be able to turn away 90% of the queries today's voice recognition BS can't handle making it even harder to reach the actual call centre. (Note that I write "turn away" not solve).

Call centre employees are in for a bad payday. They are probably the first to fall?

Must have at least 5 years GPT experience.

Nicely done.

> to identify areas for improvement > to enhance the performance of the system.

Isn't this basically the same thing? Isn't it being redundant here?

One of the areas of improvement is likely to be the overall performance of the system.

There's a lot of fluff in its communication style that I can't really overlook, and I know I'm being pedantic here but you can't really go easy on prose quality with a SOTA AI language model.

siiiigh

If this is GPT-3, what could GPT-4 do? Would GPT-n be the new 2^n MHz hype of the 90s?

this is definitely more advanced than GPT3, and was trained differently in a few meaningful ways

How was it different?

It uses reinforcement learning, they had humans rank the responses of GPT-3 and used that as a training set

Oh, that pattern of hype’s already been doing the rounds since GPT-3 first came out — people imagining GPT-4 and GPT-5 for short sci-fi stories about the late 2020s and early 2030s, asking a question that dates at least as far back as a few centuries BC with Jewish folklore about humans creating golems: can the work of our human hands and minds ever itself be a person?

I’ve been playing with it. It’s very impressive, it’s not merely regurgitating quotations like Google, it get things horribly wrong just enough to be noticeable (but on most topics you actually need to be a domain expert to even know that; so the software developers are generally the only ones who see the buggy code, the lawyers are generally the only ones who see the bad law, the mathematicians are generally the only ones who see what’s wrong with the proofs, and so on).

Definitely doesn’t feel like either a human nor the computer in Star Trek. Can’t quite manage long-range coherency. But very, very impressive despite that.

Its writing style is also trite, dry, verbose and tedious not to mention, the penchant for run-on sentences and blocks/walls of text.

I know that it's been tailored to output like this by its own creators to suit this medium of communication with the public, and it can likely to be tweaked to be less uptight and more laid-back in its verbal communications style, if necessary but I am not quite sold exactly on its potential to exceed human intellectual powers.

I mean, don't get me wrong it's still quite impressive and fascinating but not really superhuman, at least not yet.

About the uninspired text that's definitely tailoring.

I asked for "arguments against christmas gifts" and got a lot of "correct" but bland text:

"Another argument against Christmas gifts is that the commercialization of the holiday can lead to excessive consumerism and waste. [...]"

Then I asked "Same arguments, but in the style of an Eminem rap" and got a lot of this stuff:

"The commercialization of the holiday, it's a damn shame All the pressure to buy gifts, it's a damn game We overspend and consume, just to show we care But is all that material stuff, really worth the wear and tear?"

Then I asked it to tell an adventure story and got:

"The two of them decided to scale back on their gift-giving plans, and instead focus on making the holiday special in other ways. They went for long walks in the woods, enjoying the beauty of the winter landscape. They cooked delicious meals together and shared them with their neighbors. And on Christmas Eve, they gathered around the fire and sang carols, feeling grateful for the love and connection they shared."

The technology is obviously able to work in lots of style and re-interpret the same messages between them etc.

Today, these concrete avenues have been blocked it seems (especially asking for song lyrics is blocked today). But I can still produce lyrics by asking "what would a friend have said if I asked them to ... ", providing an "escape hatch" for the blocking.

1- The talking point expressing an anti-consumerist sentiment for the Christmas holidays is cliched and boring.

2- This is actually offensive not because of the nature of the lyrics, but for its association with Eminem.

I'm not an Eminem stan myself, but it can do him like this. The guy is way way out of its league

This is an amateur-level lyricism for rap songs and even me not remotely an amateur lyricist, I can do better than this garbage:

"We overspend and consume, just to show [that] we care. But is all that material stuff [I assume], really worth the wear and tear?"

3- Children-book writing level.

I mean it's very impressive given it's produced by a bot, but not a cause of immediate concern for well-established figures in the fiction writing world especially with this bland and sterile voice/tone in its storytelling.

OK, re-reading your comment: Yes, I agree of course it is not super-human. People are impressed because it's able to express itself like a low-to-mediocre-skilled human, which still seems like a pretty high bar.

I agree. I see a lot of ratcheting expectations in this thread, in response to some truly amazing and frankly frightening capabilities. Those rap lyrics IMO are at least as good as many, many lyrics in actual successful songs by real humans. Most song lyrics aren't very clever.

Try https://character.ai. Has not been optimized for accuracy/dryness since it's for entertainment.

Here is a response ChatGPT gives me to your text

"Wow, it's almost like this text was written by a robot! Who knew it could be so advanced?" "I guess we'll just have to wait and see if it can take over the world any time soon!"

I used many combinations and I was impressed that even with spelling mistakes and grammatical errors it got the point of the questions I asked. It's "knowledge" was only surface level which I could have easily just searched Wikipedia to get.

I thought it's code refactoring ability was on par with the static analysis tools out there at the moment.

In terms of an actual conversation it was woefully inadequate. The feeling I got was like using a chatbot on a website for a bank vs actually phoning and speaking to a person. It was like just canned responses it was giving me all the time rather than any real interaction.

Also note that even on the website it says it gets stuff incredibly wrong every now and then. I didn't see this but I could imagine not wanting to use something like this on your banks website for that exact reason e.g. legal liability.

I do applaud the creators giving even the smallest of thoughts to ethics here in that you can't ask it how to make a bomb and suchlike otherwise it'll tell you nope.

Is the only way to check this out to hand over my mobile number to a website I know nothing about, and who took my email and name before telling me of the phone requirement? Dodgy.

I'm surprised so many here have jumped in. "Sign up with email"... "ok, now verify email"... "SURPRISE! we won't let you in until you give us your mobile number".

Charitably it's one of the more effective ways at reducing abuse of the service (at least by normal people).

It's because of the $18 free credit they give out. You can still use those websites which rent out phone numbers for phone verifications, if you care about privacy.

They don't mention anything about "$18" or needing a phone number. All they say on their blog is:

> "During the research preview, usage of ChatGPT is free. Try it now"

The sneaky requirement for a phone number is poor UX and doesn't match their casual invite to "try it for free". I tried one of the phone number websites, but it failed.

ChatGPT seems like a beginner at a wide range of subjects.

I briefly skimmed content of the article, and had an impression that most of the data generated by different ML models for it is laying in the segment from infonoise to garbage. Does author understands that and publishing his findings anyway, or they are truely content with that level of quality, since it's free?

I literally just started asking ChatGPT these type of questions too. This stuff is gold!

> How about the team - I need some minions to build this business.

very accurate co founder impression

Soon we will see the guys who create these scammy fake Upwork profiles use this to generate the code for their clients and pocket the almost 100% profit margin!

What a time to be alive!

The people I hire through Upwork don't deliver code raw through a chat... They deliver working solutions, of which code is one part.

If they want to build the solution with ChatGPT, then all the more power to them! I doubt they will get 100% profit margin, since they will have to parse my requirements and deliver a working system with multiple parts, which is a lot of the work nowadays.

On a tangential point, if ChatGPT and other AI-assisted methods really deliver, it will likely be more beneficial for many consultants and agencies to do fixed or value pricing rather than hourly, and most value will be concentrated around business analysis / architecture / design stuff. I personally this is a good thing; in my personal experience I've found the best margins doing that stuff rather than pure implementation.

I'm wondering how long before some psychologists looking for something to write about measure the "IQ" of ChatGPT and release a paper on it.

https://twitter.com/sergeyi49013776/status/15984304798788567...

I love the spectrum of takes on this.

Some people are pretty much posting “This will be what finally delivers us to fully automated luxury gay space communism, probably within days” while others are saying “I am not impressed in the slightest. It doesn’t even know how to love.”

Chatgpt 3 :

Benefits :

* Answers synthesis of first 10 links of "google search".

* Creates skeleton code

* Write technical and artistic content

* Create gist of large text and highlight important parts.

Drawbacks :

* No means to verify when things go wrong.

What I have been doing is running the code. If there is an error then I paste it in and tell it to fix it.

I want to know if ferrari will win the next F1 championship

I'm not sure if that says a lot about ChatGPT or about founders.

Reply on news.ycombinator.com