Hacker News Reader

Comment by JackFr | Hacker News Reader

This is terrible. The idea of ChatGPT being able to fill in gaps of not knowing what you don’t know is appealing, but you need to actually know something about the topic to evaluate ChatGPTs ability.

This is just spewing pure nonsense, and the author thinks he has learned something from it.

View on HN

The worst part is the "p-values" it completely made up.

Verification that the numbers were legitimate -- "What's a p-value."

At least it got the values right .9912 (twice), .8813 (twice). It estimated 88-99% chance of bullshit... To 4 significant digits... Repeating the exact values.

Turns out, this is an excellent example of how poor these models are with accurate knowledge vs confident bullshitting.

This. If you've been following GPT at all you'd be crazy to try to use it to learn new things. It's very clear the output is not accurate, and prone to hallucination, and requires an already knowledgable user to judge its veracity and validity.

I was worried something like this would happen. AI will usher in a dark age thanks to overfitting and convenience and naivety. It's going to be like the current info literacy crisis raised to the nth degree.

It's also highly suggestible. If the input was interpreted wrongly, it can deviate from a set goal. Garbage in => garbage out.

Example:

>> If I'm feeling a little horse, where am I?

>> If you're feeling a little horse, you might be experiencing a sore throat or hoarseness in your voice. This is a common symptom of many respiratory illnesses, such as the common cold, flu, or bronchitis, as well as other conditions that affect the throat and vocal cords, such as acid reflux, allergies, or overuse of the voice. It's important to stay hydrated and rest your voice if you're experiencing hoarseness, and to seek medical attention if your symptoms persist or worsen over time.

Treated as an opportunity for criticism, it can be an effective learning tool, especially for developing your critical literacy. Think of it as a game where the point is to prove it wrong.

> but you need to actually know something about the topic to evaluate ChatGPTs ability.

I don't necessarily think you have to, but you have to be ready to look up further, more trustworthy sources after your discussion with ChatGPT.

People already don't do that with our current tools, what makes anyone believe it will happen with ChatGPT? Especially when they now have to go back to their favorite search engine to find those sources and validate them.

It's as if everyone has forgotten how humans behave, they don't even read or go to actual wikipedia sources that are properly laid out! Nevermind having to go find trustworthy sources.

I don't really care about how others use it. I myself find it enough useful and productive to use it, and what I wrote before is how I "protect" myself. As you say, people who don't check sources won't do so with or without GPT4.

It's useful to learn new tips and tricks. e.g. the GPT4 demo showed me how to get much better code output.

Maybe you'll eventually care about how others use it when your trustworthy sources are flooded with irrelevant, outdated, and downright false information due to the constant generation of "kind of true" content.

I mean like it already currently is?

spoiler: the vast vast vast vast vast majority absolutely will not ever do that

this morning I needed to populate an SQL database with dummy values, only a few dozen though

so I told ChatGPT the model with types and asked it for random values in csv format. I’m sure the API could spit out far more if I used that

I imported into the database and moved on. 30 seconds? mostly waiting for it to output

maybe we should make a site where people share what they used LLMs for

but... this has nothing to do with what parent is talking about...

Upon second reading, oh yeah youre right

Do you think that other sources are infallible? Almost every source is wrong sometimes. ChatGPT is as good as my professors were in college. They would regularly say things that were just completely incorrect but were knowledgeable and correct often enough to be helpful.

Other sources are fallible as well, but also less likely to make big mistakes and more likely to cover the major pitfalls of a hobby/area of expertise by virtue of actual experience and understanding of the subject matter. (versus statistical probabilities in word choice)

Also ChatGPT has no consistent internal sense of self and is highly sensitive to input. You can convince it that just about anything is true if you aren't careful about how you phrase your input.

Reminds me of people being up in arms when Wikipedia turned up because anyone could contribute. Seems like the same thing is still happening.

It is quite similar, isn't it? To say that a potentially flawed source of information is worthless is a silly extreme.

Wikipedia works since the world is full of pedantic sticklers. If you remove the pedantic sticklers then wikipedia would no longer work, it would just be full of bias or bullshit, so we should be thankful for all the pedantic sticklers out there.

Pilots make mistakes sometimes. I, the bike rider also make mistake sometimes. Therefore I can fly an airplane and I will be as good.

Jokes aside, making mistakes is not a good common factor in order to evaluate if two different systems can be practically interchanged. You need more. What type of mistakes? In what situations, and how to mitigate them. This is a surprisingly common misunderstanding running wild here. You cannot even switch the places of two humans, even two professors in real life using this line of reasoning.

Your professors generally don’t make stuff up if they don’t know something. Your professors are also more likely to catch on if something doesn’t make sense - unlike ChatGPT which can’t seem to reason effectively because most of the time it doesn’t know what’s it’s talking about.*

* Maybe it has the ability to reason, maybe not. Maybe it just needs more information - after all it’s trying to model reality with just the words used to describe it. Or maybe it really can’t reason at all, a limitation of its algorithm, and has to rely on having arguments/explanations already memorised - which seems to be mostly what it does now. Either way so far it hasn’t demonstrated that it’s all that good at reasoning.

Some of them most certainly did make stuff up. Ego prevents many people from admitting that they don't know something.

I also interacted with some who wouldn't even admit to being wrong when they were confronted with facts. So I don't know that I would say they catch on either.

I feel most won’t risk it. It’s less embarrassing to say you don’t know than to be wrong.

How do you know that ChatGPT is as good as your professors in college? How do you know that they aren't just as good as a cable news pundit? Or a relatively uninformed Reddit commenter?

How do you compare anybody like that?

All I'm saying is that ChatGPT is knowledgeable. It has absorbed a ton of information and can distribute that information fairly well. I don't think that can be argued with.

Yes, it's completely wrong sometimes but the exact same can be said for any human (my professors, the cable news pundit, average redditor).

To say I can't learn from it doesn't make any sense. I have learned everything from flawed sources. We all have.

> How do you compare anybody like that?

I was thinking maybe you had some specific metrics or examples in mind since it's your own comparison...

The rest of your post reads like the old "there are two possible outcomes therefore the odds are 50/50" probability fallacy.

Do I believe it's more useful than chatroullette? Yes. Does that mean it's as useful as a college professor on a given subject? That's a very different claim, and would need some serious evaluation and demonstration.

> I have learned everything from flawed sources. We all have.

Sure, some historical books in the US South are wildly inaccurate!

This is why we deal with multiple sources, as well as defining, validating, and exploring our sources. ChatGPT does not do this. It does not pull from definable sources, it is an LLM that outputs text based on statistical word sequences from an almost bottomless source.

You can learn from it, but the sources may be sketchy, and it will mix source biases. I tried to ask it for sources on the fall of ancient Rome:

>> As an AI language model, I don't have the ability to consult sources or remember my previous responses. However, the information I provided about the fall of Rome is widely accepted in historical scholarship and can be found in many reputable sources such as encyclopedias, textbooks, and academic articles.

Personally, I think we need to tune the LLMs according to use. Creative? Then it one way. More factual, the other way. This does mean we would need to know sources for LLMs.

Unfortunately this says more about your alma mater than anything else.

Reply on news.ycombinator.com