Hacker News Reader

Comment by cubefox | Hacker News Reader

It absolutely was accurate. Back in 2018 (GPT-1) nobody expected we would soon have something as powerful as GPT-3, or even ChatGPT. The massive progress in LLMs (and text-to-image models) took everyone by surprise. They achieved things that, until recently, were regarded as pure science fiction about the distant future.

View on HN

I feel that "soon" is doing a lot of the work here, as far as I recall back in 2018 people in the research community absolutely were expecting something similar to GPT-3, only the expectation was that these capabilities would land perhaps 2-3 years later than they did.

And at least part of that was the (implied, natural, understandable) "linear extrapolation" of the total cost of compute applied - if your intuitive expectation of progress if effectively "oh, what would happen next year if we'd triple the budget" but actually the applied compute power increases hundredfold, then obviously the outcome arrives faster than you expected.

What’s so “powerful” about ChatGPT 3 and 4 ?

They respond so well that a lot of people believe their results without question, even while we know that they hallucinate all the time. They’re good enough for many. Even here in HN, where everybody should know that they are terrible with numbers, majority of us cheered that somebody generated diagrams with them, I couldn’t even find a single comment which were about “wait a minute, they’re fucking unreliable numbers”.

Also regarding coding, they can definitely produce junior level code, especially if you follow a similar trial and error way of thinking which happens all the time during your work, if you try to use an unknown interface.

Their ability to superficially mimic cognition, in a way that not only raises significant philosophical questions but also seems (rightly or wrongly) to bring the prospect of "true" AI tantalizingly closer.

What are the new philosophical questions that it raises for you ? Existential crisis ?

> bring the prospect of "true" AI tantalizingly closer

I do think that it’s quite a leap to assume that. It is very good at mimicking, but the same way some parrots can talk, they are endlessly far from human cognition. That jump is very very non-trivial.

A stochastic parrot, a blurry jpeg of the web is able to provide sound, actionable, verifiable advice on topics such as modeling in Blender or managing a Linux server -- despite not being generally intelligent.

Reply on news.ycombinator.com