I spent 5 minutes searching Google for what an email error code meant, it kept showing me threads on Microsoft's forums that never gave me a clear answer. Took about 10 seconds to get the correct answer on ChatGPT.
If you treat it like it's a bumbling assistant that has remarkably encyclopedic knowledge, it's a very helpful tool. If you treat it like it's truly intelligent and let it do your thinking for you, then you're going to have some problems.
But, generally speaking, if you don’t have some way of verifying what the correct answer is, how can you be sure that the answers you get in 10 seconds from the LLM are actually correct?
In many cases, programming included, you don't actually care if it's 100% correct, it is enough to just give you a nudge, show you the way forward, suggest some ideas etc.
If whatever it is you are doing can be verified for correctness, then of course you will check the answer. But for creative pursuits there often is no correct answer, and generative AI (LLMs included) excel at that.
I don't understand how people feel they can use it as a search engine replacement. If you have to go separately check the results then it doesn't seem like it's saving any time. I suspect people are just trusting the output a lot more than they should.
Before ChatGPT I had to manually investigate and read through multiple pages of potentially irrelevant stuff to find an answer I could try to verify. If you find an answer on StackOverflow, you still need to integrate and verify it.
Now I get an answer straight away just by asking a question, and can skip manual knowledge extraction step to get right on answer verification step (i.e. compiling, testing, extending).
Even if ChatGPT is nothing else except a way to fuzzy search and summarize found results, even just that is already hugely helpful and has almost completely replaced Google for me.
But on a serious note I have only gotten good content out of ChatGPT when it's used as a smart Thesaurus, or a code checker (even that was 50:50). If I have a paragraph that I feel needs rewriting or condensing I can throw it in and resubmit until I get a good result - which I still then need to edit to make it less... ChatGPTish. If I open up old scripts (data extraction/analysis) that I forgot what they do I chuck it in and find out what will happen.
Like any large technological advance, the hype is much larger than the eventual reality (see VR goggles).
[1] technically it doesn't even make "mistakes", it's just an LLM doing its thing, but we humans like to anthropomorphise
just like a child, chatgpt can not tell the difference between reality and fiction.
chatgpt lies because it presents fiction as fact. there is no intent to deceive necessary.
I'm done anthopomorphising now.
To me, it is like a junior developer. It does the bulk of the work and whenever there is an issue, you can tell it like in a code review to fix a particular piece. Look past the confidence level and focus on the work product.
Remember, it doesn’t need to always be accurate. It doesn’t need to be perfect. It just needs to outperform the alternative (passable job here) for the price paid (amazing value here).
But indeed it's still quite useful. It's best used in situations where it doesn't matter much if the answer is right, or when the answer can be easily checked if it's correct, yet more annoying to come up by oneself in the first place.
90% of the time I know the code I want to write. ChatGPT just writes it for me, when I read it I can confirm it is what I would have written, or fix it if not (or more likely: tell ChatGPT to fix it)
This means I don’t have to google for syntax, canonical ways of doing things, etc.
If you are asking ChatGPT for facts, triple check everything it tells you. Or better still, paste the Wikipedia entry to it first, then ask for the style or type of writing you are interested in about those facts
> People are enthralled with this tech when they should still be very very skeptical.
Yes, that's the real issue. Everyone should be very skeptical about ChatGPT responses. But instead it's often trusted blindly, because "it's often enough correct".
But I get the idea behind this particular move and at least it can be a chance to teach the students to distrust it.
Isn’t denying these facts just as dogmatic as accepting them blindly?
Instead, think of it as a computer that can perform transformations on text, described in plain language and based on text's tone and meaning.
There's its ability to write small one-off scripts for me. I've given it an example of a structured log file and told it to "write a pyparser script to parse this string" and it will do that. What would take me a hour to learn pyparser and get somewhere now takes a few minutes. You can literally see it iterate on different solutions until it comes up with something acceptable. (ChatGPT 3.5 doesn't do that.)
Another great use case is as an assistant when I'm learning some new topic.
I was reading A Mathematical Theory of Communications, the breakthrough thesis by Claude Shannon. It's quite accessible, but there are always points where you don't quite understand a point that is being made and you want to get more examples. I can ask ChatGPT to create a concrete example, use actual numbers, plot a graph that illustrates the concept. And it will do that, along with the Python script that it used to create the graph so I can double check what it did. "Create an Eb/No graph for the following parameters." And then ask to modify the graph to illustrate a particular point. Google can deliver similar information, but it will push me a document that contains the answer, along with a truckload of info that I wasn't looking for. ChatGPT pulls just the info that I'm looking for.
Or lately I was looking into Laplace transformations, low pass filters, pole/zero plots etc. ChatGPT 4.0 uses the SymPy symbolic math package to derive the math, I can once again examine the scripts, ask for concrete examples with graphs and so forth. It's way more efficient than doing the same thing with Google, which will usually send you StackExchange with an answer that is not quite what you're looking for.
In these kind of cases, I know enough math and subject knowledge to know when it's bullshitting, but it doesn't have to be 100% accurate to a huge help in learning.
Here's an example of some thing I did off the cuff. I have a bunch of hand written notes that are mind maps and graphs written on old notebooks from 10 years ago. I snapped a picture of all of the different graphs, threw them into chatGPT and asked it to convert them to mermaid UML syntax.
Every single one of them converted flawlessly when I brought them into my markdown editor.
The amazing thing here is that that the ChatGPT large language model wasn't even explicitly trained for this at all, but with the massive quantity of data that it's trained on its able to just connect the dots.
If you're using chatGPT as nothing more than a glorified fact checker and not taking advantage of the multimodal capabilities such as vision, OCR, Python VM, generative imagery, you're really missing the point.