Hacker News Reader

Comment by lionkor | Hacker News Reader

GPT 3.5 had trouble understanding when I told it "Say 2 bob are a beb, how many beb per bob are there?" and it wrote a goddamn essay about shoes.

That thing isnt smart, it doesnt understand, it doesnt know, it just rambles. I have worked with people who do the same, yes, but they also werent a threat to most jobs.

I said it before, and I will say it again: If ChatGPT 3,4,5,... can take your job, maybe youre not really providing that much value. Make of that what you will - not everyone has to provide huge value.

View on HN

Answer from GPT-4:

"This question seems to be intentionally nonsensical or is using unfamiliar terminology. However, if we try to interpret it, we could say that there are 2 "bob" making up 1 "beb." In this case, there would be 0.5 "beb" per "bob." Please provide more context or clarify the terms if you are looking for a different answer."

Answer from GPT-3.5 (subscription version, not free):

"If 2 bob are a beb, then the conversion factor from bobs to bebs is 1 beb per 2 bobs.

So, there is 0.5 beb per bob (or alternatively, 2 bobs per beb)."

Answer from GPT-3.5 (legacy / free version):

"If 2 bobs are equal to 1 beb, then there would be 1 beb per 2 bobs."

What do LLaMA-based models answer for this?

alpaca.cpp 7b model gave me this:

> Say 2 bob are a beb, how many beb per bob are there?

There are two bebs per bob.

> No, there are 2 bobs per beb.

Correct! There is only one other answer that makes sense - "1 Bob for every Beb". But this does not make mathematical or logical sense since a single Bob can have multiple Babes associated with it and vice versa (as seen in the example). Therefore, we must conclude there are two bobs per beb.

Cool, but sadly, as I said, it did not give a very useful answer. If asked enough times, im sure it will give a reasonable answer, yes, but thats not the point.

GPT4s answer is interesting, though

But all of the answers were correct and useful, and GPT-4 was perfect. Anyway ChatGPT is getting hooked up to Wolfram Alpha, and that won't have any issues with basic algebra.

If you already know the answer, but need an interesting and detailed way to explain/write it, then GPT4 can be an immense value-add. Even if you have to do some fact checking afterwards.

> I said it before, and I will say it again: If ChatGPT 3,4,5,... can take your job, maybe youre not really providing that much value.

The thing you're missing here is that it might take someone's job not because they weren't providing the value but because the people who crunch numbers decided to axe 15% of a company's workforce because some executive was sold a pack of lies about what LLMs/"AI" are actually capable of.

It's fine if that happens to one company who then finds out the hard way. It's probably more social-unresty if it's essentially done at every company in every marketplace an LLM can touch - from writing to programming to 3D animation to teaching.

The hype machine around LLM/AI here is the same irrational one we saw around blockchain. The key difference is blockchain was basically never sold as really replacing a person's job (at best you could argue it was sold as getting around the banking industry and maybe eventually being able to replace it, ish). The primary sales pitch of these LLMs is essentially "do more with less".

I typed the query into chat-gpt3.5 (turbo and legacy), and 4, and they all said that there's 0.5 beb per bob.

Did you use the quoted prompt exactly?

No, I didn't use the quoted prompt, but even after explaining to it that bob and beb were not, in fact, shoe related terms, it still kept insisting and being confused (while also giving the correct 1/2 answer).

It can do it, but its not deterministic, and it doesnt really do it well. You can continue the chain by asking "How many bob per bib, assuming two beb per bib?", and see if it chokes then. It sometimes does, sometimes doesnt.

GPT-4:

   If 2 bebs are equal to 1 bib, and we know that 1 beb equals 2 bobs, we can
   determine how many bobs there are per bib using simple substitution.
   
   1 bib = 2 bebs
   1 beb = 2 bobs
   
   Therefore,
   
   1 bib = 2 bebs × 2 bobs/beb = 4 bobs
   
   So, there are 4 bobs per bib.

Nitpick: A properly done substitution would've arrived at

   1 bib = 2 × (2 bobs)

without needing any of the "2 bebs × 2 bobs/beb" nonsense. It doesn't teach this task very well.

You do realize that the current implementations get their context polluted by your prior conversation, right?

I think right here we have an example AI reproducibility problem. It seems fully reasonable and credible as an outcome, but it is hard to dig in and replicate. But the truth of ML is it would be difficult to replicate even if things were FOSS.

Reply on news.ycombinator.com