Comment by ahelwer | Hacker News Reader

Comment by ahelwer | Hacker News Reader

I finally went ahead and tried ChatGPT this morning (along with everyone else seemingly - it is now heavily rate-limited!) and I am truly blown away. I ask questions about various things (in my case, embedded systems development on the PineNote) and it gives the same quality of answers I would expect from a median coworker. Sure they're maybe not 100% correct, but most coworker answers aren't 100% correct either. They're at least in the right ballpark, and very rarely do we achieve detailed of knowledge of something from a single source - it's all about taking in info from various places and integrating them (conflicts and all) to derive a more-and-more-detailed grasp of the things we're learning about. The ability to ask questions about a specific thing (example: What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files?) very quickly without having to sift through mountains of blogs and random Q/A websites and documentation and textbooks for the answer is incredibly useful.

If you ask it about things which require deduction like Math, even simple Math questions like multiply binomials or solve a quadratic it gets it totally wrong, confidently, and even if you correct it, it often still gets it wrong.

It’s not even close to something like Wolfram Alpha.

I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.

Sure. But if you combine the understanding that this chatbot has with a Wolfram Alpha backend, you could build an even more amazing system. I'm sure someone is working on hooking up language models to math backends (anywhere from a simple calculator to Wolfram Alpha).

DeepMind published a system that does sort this with a backend theorem prover a year ago. My point is, I don’t think transformer based text prediction systems are the right model here. I could be wrong, but it think about how formal systems work, they seem a far cry from what decoder architectures are doing.

https://www.nature.com/articles/s41586-021-04086-x

Youre commenting on an article where the answer was not even just a little wrong - it was completely wrong. Sometimes it’s “in the ballpark” - which is apparently t good enough these days - but often times it is just confidently entirely correct. How are you able to use such a tool as you propose practically?

I could easily imagine an ordinary person giving this exact same wrong answer (confusing Hobbes and Locke) - we're talking about value-over-replacement here!

In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.

If this were a test question, the response given would be marked wrong, likely with no partial credit awarded. It's that egregiously wrong, even if the attribution is perhaps understandable.

Well thankfully I haven't been in school for over a decade at this point, so rarely (never?) encounter these hard-cutoff no-second-chances trivia situations. I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.

> not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.

Not-quite-correct is a wordy misrepresentation when something is plainly incorrect.

In many fields being confidently wrong is how you get fucked over hard.

That ChatGPT will gleefully fabricate references like a schizophrenic is just more icing on this shit cake.

> I operate in the real world, where continually making something that's not-quite-correct then refining it and fixing the errors is how basically everything is accomplished.

This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.

You can correct the bot.

If you tell it that a fact it has wrong is wrong, you can get it to change its answer.

Also, you can just make tweaks manually.

Seems super useful to me as long as you understand the limitations.

Trust it the right it amount - don't trust it too much, and you're golden.

Not clear that context goes further than your individual conversation, in fact it almost certainly doesn't.

> If this were a test question, the response given would be marked wrong, likely with no partial credit awarded.

Probably. So, don't use ChatGPT unsupervised to take tests.

How can you use it anywhere? If you need to know enough about a subject to judge whether an answer it gives is correct why would you need it in the first place?

Endeavours in which someone getd benefits in throughput by working with less skilled collaborators whose work they supervise, review, and send back with comments for rework when it is flawed are...not uncommon.

I think the problem is assuming GPTChat is a reliable source at all. You can probably assume your median coworker knows something correct about embedded systems but it's not clear why you would assume or if you should ever assume ChatGPT is correct about anything.

>What are waveform files for e-ink screens? Followup: Is it possible to damage e-ink screens by modifying wave form files?

What is the value of these answers if there is a 1% chance of them being wrong?

From my experience it is a better way to google things, which certainly is an advancement and quite novel.

Reply on news.ycombinator.com