Hacker News Reader

Comment by fnordpiglet | Hacker News Reader

I find it amazing from the point of view that this is not a linguist or a grade schooler, but a general purpose computer program. Have you seen a computer program that can do these things, especially one that wasn’t written to do these things but can also turn itself into a plausible linux box, translate python to rust, and all sorts of other wacky party tricks? That’s the amazing thing - not that the feats themselves are novel or unique, but they’re novel and unique for a a general purpose computer program to be so adaptive and flexible in both what it takes as input but also how it interprets that input into remarkably coherent and salient outputs.

Taking into account it’s a beta and that the underlying tech is really new, you can extrapolate that some of these gaps can be tackled - then will you be amazed?

View on HN

ChatGPT's superpower is hijacking the human desire for meaning into causing people to believe it is much better than it really is.

https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach

has dialogues between Achilles and the Tortoise where they work out the limitations of various fantastic A.I. systems based on mathematical logic. Roughly, neural networks don't repeal the results of Gödel, Turing and Tarski.

The short of it is that ChatGPT is good at charming people, better than some people are at charming people, but when it comes to getting accurate answers some of the things which it tries to do are logically impossible, so it is not like it gets it right 70% of the time now and they'll get that up to 100% but rather people will be puzzled about how it hit a plateau and we'll hear a few years later about what a disappointment it was just like Amazon's Alexa.

I’ve been using it to teach me programming languages. If it plateaus here I’m delighted by its uniqueness and utility.

You seem to think people are hinting this is general ai. I don’t think that’s what’s amazing people.

I’ll bet you Siri and Alexa would be 100x more useful with a language model like this behind it. Part of their uselessness is the inability to generalize or reason, but relying instead in coded prompt and replies. Is that lost on the Alexa team (or what’s left)? I’m sure not. So, hey, I guess Alexa won’t plateau yet either eh?

There will come a time that it’s harder and harder to distinguish what’s AI and what isn’t. At a certain point, will it matter? Is the utility of these tools their ability to manifest human sentience? I don’t know why that’s useful to me - I already did that creating my daughter. What I want is a tool that enhances and extends and supplements my own mind and abilities to make me more effective in what I do.

And ChatGPT actually already does that. I can ask it questions about programming problems I’m having and it’ll largely give me a right answer - at least as often as a person would, and more reliably than I get on stack overflow. I’ll still use stack overflow, but I’ll filter my questions through assistants like this.

"100x more useful" is pure hyperbole. 2x maybe, but 100x is a symptom of exactly what's wrong about people's thinking about ChatGPT.

I would point to "There is no royal road to geometry"

https://en.wikiquote.org/wiki/Euclid

high-social status people have often been put off by quantitative reasoning because you spend years learning to do it, spend time gathering facts, thinking hard about things and often end up with tentative statements. High-social status people can skip all of that and get deference anyway the same way ChatGPT does.

Have you ever used Alexa? 2x more useful than alexa is as useful as my doorstop. It can’t understand most prompts, and defaults to reading a snippet from Wikipedia on some random subject then starts playing a pop song from Malaysia at volume 10 while ordering 60,000 incandescent black light bulbs with 2 hour shipping.

Even if this ends up in a plateau, that doesn't discount the fact that it's a huge step forward from what we were capable of building 5 years ago. If the next generation of language models make us believe they're superintelligent but are actually at the level of an average college student, that would still be an amazing achievement.

That wasn't Hofstadter's point. Hofstadter didn't try to "disprove" the possibility of having AI -- he only set out to show that certain approaches couldn't work. GPT's approach isn't one of those he argued against. You seem to be referring instead to Penrose and Lucas, but their arguments have convinced virtually no one.

My point is that you can relax constraints such as Gödel's incompleteness theorems if you are going to give it credit when it gets wrong answers.

It might be able to write a TV Show like Sliders but it won't be able to do real physics, for instance.

I have some level of concern that we have the same superpower.

I had a weird effect with it: I played around with it for about an hour and was pretty impressed - I can't really wrap my head around how it can do some of the things it does. Then I started to see the patterns, and got repeatedly disappointed by having to adjust and refine my prompts. Then I got downright nonsense back from it and gave up. Either I made it to some kind of uncanny valley, and it would get better if I invest more time, or most people who get amazed don't make it that far.

At this point I'm mainly concerned about the unimaginable heap of garbage it will release on the world - good enough to impress decision makers at first glance, annoying to deal with if you actually have to work with it.

I feel like it’s state can be corrupted and it ends up in some weird rut. Often by telling it to try again I can unwedge it but sometimes I have to reset the entire state and start the convo over again. Each time is slightly different. These seem like bugs that optimization heuristics can overcome (like maybe replaying the same convo repeatedly and choosing the highest information next prompt to present? Who knows. It’s basically magic and I’m waving coconuts hoping the planes will land)

Pretty much! It probably sounds lame, but the main reason I enjoy programming is that I can, if I take enough time, reason about everything that's going on and use that knowledge to build, fix and improve systems people get some value out of. For me, the magical part is the possibility to understand the magic.

I find it even hard to reason about models I have full access to, downright impossible if it's some blackbox on someone else's severs.

Maybe I'm outdated, I suppose time will tell.

ChatGPT's superpower is hijacking the human desire for meaning into causing people to believe it is much better than it really is.

It was a low bar to clear, given human celebrity culture.

We live in interesting times, and they're about to get a lot more interesting.

> some of these gaps can be tackled

Seems like the major gap is in facts. It'll often make up completely plausible specific details that are flat wrong, or refuse to proceed if it's "not allowed" to do so.

Coupling it (handwave) with a massive structured data repository like Wolfram Alpha and Wikidata would be really something.

I find the guard rails they put in maddening and weirdly implemented. I wanted it to make up jokes for me and it’s jokes are the same few over and over. (I can never remember jokes so this is a super useful tool for me!) so I wanted it to tell me jokes in the style of Jerry Seinfeld. The guard rails prevented it from doing that because he’s a real person and it tries to not bend its personality pretty hard. However by just resetting the convo a few times I randomly got it to agree to tell me a joke. I can’t remember it but it was actually funny and in the style of Seinfeld. shrug

It almost certainly has indexed Wikipedia, fwiw.

Indexing Wikipedia (or any) text is a statistical thing though. When you ask a specific question like the size of a car, say, it will often come back with a compelling number in context, but if you actually look it up, it's usually not quite right.

Another example is that it can just "invent" new functions when coming up with code snippets. Syntactically usually correct and completely plausible in the context, but simply doesn't exist.

I also am pretty baffled by the limitations. I just assume they're trying to avoid "scammers use AI to construct false financial documents" type of furores in the early days (once the market is saturated with competition, blame will be too diffuse to stick) and convincing customers of their proprietary systems that they won't end up with a customer support bot that starts reciting racial slurs.

It insisted to me for a while that the sun was red. The "guardrails" do nothing to stop it from lying confidently about almost everything.

A lot of people say that the model is "lying" when it returns incorrect facts.

Generally when friends or family confidently assert facts that they are sure but also incorrect about, we call them "misinformed" rather than connoting malice with the word "lying".

Have they heard the right facts but associated them incorrectly? Heard the wrong facts from whatever source they obtained them from? Either or both could also be true of ChatGPT.

Yes, and either word is anthropomorphising. I stick with "lying" nonetheless, because if you start a conversation with "What color is the sun?" it returns a correct answer. It already has the association between "sun" and "white" in the model, but for whatever reason the context of my conversation made it say the sun is red. "Lying" seems to describe this behavior better than "misinformed."

It's more like you can't actually read but you've seen a lot of media about cars that contain text saying they have a thing called a "width" around the 1.7-2m mark and usually aren't round numbers. So when someone asks you how wide a 2011[1] Ford Focus is[2], you say "1,769mm" and look confident. Syntactically right and by any probabilistic model, pretty good and plausible.

[1] and if you ask for 2012, it tells you "I'm sorry, but I am not able to browse the internet to provide you with the specific dimensions of the 2012 Ford Focus."

[2] according to Google results, it's 1823 mm to 1842 mm.

I’ve only seen it do stuff like that if I assert a false premise and ask it questions off the false premise.

It seems also to do it if you ask about something that's not documented in detail. It'll get details of the 1976 Tangshan earthquake dead on, but ask for the 1982 Bonin Island earthquake and apparently it was at a depth of 40km (it was actually 22.2), but it gets the magnitude right.

The Nicobar Islands earthquake on 26 December the same year apparently killed 4000 with a magnitude of 7.1. It didn't, it injured 52, and there were two: magnitude 6.2 and 6.3. And it was in January. 26th December was indeed an earthquake that affected those islands. In 2004.

Which is not that surprising, as how would it know, considering it probably didn't have much input about it. What is more amazing is the seamless blending of correctly regurgitated details with very plausible wholesale fabrication and, even more convincingly, blending of fact with fiction.

> this is not a linguist or a grade schooler, but a general purpose computer program

This is sleight of hand. The training data almost certainly includes thousands of grade schoolers and linguists. Your argument seems to be that this is all generated ad nihilum from first principles (it's "just" a computer program bro), but that's not how it actually came to be. Rather, it was trained on literally terrabytes and terrabytes of human-generated data.

So, in a sense, it is actually a linguist. Just a really bad one.

It’s still a computer program with some state attached. That’s literally all it is. There are no linguists in the program. That it is able to distill the knowledge and ability of a linguist by parsing ascii text into byte buffers and doing some fancy math on the bytes should be breathtaking.

The sleight of hand would be if it had a mechanical Turk and there was a linguist. The fact that it collected knowledge to have knowledge and can apply that knowledge is no different than a linguist for sure, but that it’s a computer program distilling binary data into abstract concepts at the level of a grade schooler or a linguist (not sure who is more sophisticated) is mind bending. If you can’t see it, you’ve lost sight of things my friend. I hope you find it again. The world is a beautiful place.

> There are no linguists in the program.

I suppose intelligence actually resides in the culture, and we're like GPUs running it. Then both humans and AI have the same right to claim intelligence.

https://youtu.be/YfXc4OBDmnM?t=883

Why is that relevant? Grade schoolers and linguists also learn from other people. A single computer system with the capabilities of a grade schooler (or beyond) across a huge variety of domains (poetry writing, computer programming, linguistics, joke writing, scientific knowledge, etc, etc.), is an amazing thing that is far beyond the capacities of the best AI systems even a few years ago.

I'm perplexed to see all the people scoffing at it because it isn't as good as a Ph.D. in domain X at task Y (yet). Are we ready to declare that grade schoolers and median IQ people don't count as intelligent?

But aren't most linguists trained similarly? Broad spectrum knowledge throughout their elementary education and specifics in higher education? Mostly regurgitating things they have either memorized or been shown repeatedly.

I'm still super into this as it is, without considering real growth on the tech side. It's dumb but I think it's still useful.

It'll be crazy when it replaces us but for the moment I've been using it to learn and explore (C, some linux tools, game dev in lisp) and the results are surprising - and we're looking at the newest tech demo, as you say what will it look like in a few years? Even just give some app devs some years with it and I'm excited to see the results

If "useful" means we don't need the likes of Donald Trump and Elon Musk anymore I am all for it.

(Secret: we never did)

Reply on news.ycombinator.com