Comment by monkmartinez | Hacker News Reader

Comment by monkmartinez | Hacker News Reader

Parent Full thread

I disagree and think this is a very controversial opinion.

Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit.

I asked it to create a UI with input boxes for material, weight of material, price and loss due to wastage. The program takes all of those inputs and converts the material into grams from KG, pounds, ounces. It then calculates the price per gram and takes a loss percentage (estimate given by user). It then writes a text file and saves it to a directory.

I literally pasted the code into VS code and had to change Tkinter to tkinter. Hit run and it worked flawlessly. I have NEVER used tkinter and it took about 30 minutes from start to finish.

This morning, I asked my 9th grade son what he is learning in 9th grade biology. He told me he is learning cellular endocytosis. I asked chapGPT to explain endocytosis like I was a 5 year old and read it to him... he says; "Ask it to explain it like a scientist now." After that he said it was a really good and we started asking it all kinds of biology questions.

I happen to agree that search will be the first thing disrupted. However, I think simply saying "search" doesn't come close to capturing how deep this will change the way we think, use and progress in terms of the way we define "search" right now.

I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exchange or in documentation somewhere it doesn't work that well. Like I said, its a search engine with a lot of downsides and some upsides.

> If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

Fooling a 9th grader is amazing. That's a pretty well formed human being right there except with less life experience. Fundamentally no different from you in general reasoning terms except on a smaller set of information. So fooling you is merely a question of model size.

"Fool" is the operative word here. ChatGPT is quite capable of producing very plausible sounding text about biology that is totally incorrect. See, for example, the example in my comment at https://www.lesswrong.com/posts/28XBkxauWQAMZeXiF/?commentId...

You're basically complaining that a single model doesn't have full knowledge of every single area of all of human knowledge. It's got decent knowledge of most areas including programming with probably better overall knowledge than a high school student. That's downright amazing and probably more knowledge than any single human actually has. The rest is likely a matter of improvement along the same lines versus some radical redesign.

Well, I agree that it's amazing - it almost always produces grammatical output, for instance. But it's not a reliable way of obtaining knowledge. One should not, in particular, try to learn about biology by asking ChatGPT questions. It often produces made-up stuff that is just wrong. And it's very confidently wrong, with the output often coming across like someone barely concealing their contempt that you might doubt them.

It may or may not be fixable without radical redesign. The underlying training objective of mimicking what humans might say may be too at variance with an objective of producing true statements.

My wife (a physician) asked it multiple medical questions and the majority of the time they were dangerously wrong, but looked perfectly fine to me.

I asked it a series of questions about my area of expertise and they were wrong but looked perfectly fine to my wife.

It even confidently “solved” the 2 generals problem with a solution that looks completely plausible if you don’t already know that it won’t work.

This was my experience when I asked it questions about database or distributed systems. I wouldn't consider my an expert in these fields, but I do have more than a passing interest in them.

All the answers looked good, used several of the correct terms, and one even referenced the project I worked on, but they just contained flat out wrong information.

You reach a point, where when you ask a ML model to generate text given the internet as a corpus, where there just isn't enough text to make something that is both true and convincing. In niche fields, this is just where we are at.

Reply on news.ycombinator.com