Is it technically though?
I still struggle to find any use to it in my daily life. It is a cool demo, but no one wants to read AI generated text from other people.
I still struggle to find any use to it in my daily life. It is a cool demo, but no one wants to read AI generated text from other people.
I still believe anyone using these tools on a day-to-day basis should have a sense of "trust but verify."
Example: Using a small part of a new, big, unfamiliar library. Rather than digging through the library docs, I can ask ChatGPT about it, which often points me to the relevant parts, which I then can still confirm in the official docs.
But the code it gave me was a great starting point. I found it much faster & easier to rewrite the bad code it wrote than pore through documentation and figure out how to solve my problem from scratch.
- if I'm working in a field that's new to me or that I don't understand, I ask for help understanding the basics and vocabulary of the field. it does very well at this.
- if I have a well defined problem, but am simply not familiar with the libraries for a given situation, it tends to do a good job translating my english queries into the right code. I do examine and test the code afterwards to make sure it's correct. You can really see this in action when you ask it to do data analysis; and the REPL loop in that mode is also great at catching bugs.
- if I have like a copy-paste from a documentation site, I can ask it to transform that into code or into a better-formatted version. this saves a lot of time, and I don't have to remember regexes or vim keybinds
I also do this, but I am careful about being confident that "it does very well at this". We can't actually evaluate what it's putting out, other than that it sounds plausible, which is something LLMs are truly great at.
Like random formal letter to AI? Maybe it is a societal problem more than a technical one if we all hate writing and reading these.
I don’t get the search documentation part. It has obvious blind spots on many things and hallucinate on others.
No, I'm talking about things like writing SQL queries (even complex ones), CMake files, Docker configs or plotting stuff in Python. Of course, if you're not already an expert in these things, you'll have a hard time distinguishing useful replies from hallucinations - that's why I said it's mostly for senior devs. Without expert knowledge, you will likely not be able to benefit from it in its current state. But if you have that and know how to write efficient queries, it can easily up your productivity by a factor of 10 (i.e. going back and forth for 6 minutes with GPT4 to make it get your requirements can save you an hour of work looking through documentations).
Sounds like what a junior developer would say, given they tend to depend on it even when it hallucinates the wrong answers badly.
Also explains the rampant title inflation that is going around in the tech industry these days.
Sometimes it comes up with really good techniques that are different from my usual approaches, other times is plain wrong and I'm correcting it.
I am more productive with the ChatGPT in my life. Whenever stuck on some weird error, instead of googling I paste the log and discuss with the bot what can be going wrong there. We can talk about pointer analysis, performance bottleneck comparisons. Could I do it all on my own? Sure. However it is boring and quite certainly would require 10x more time to perform all calculations on my own.
In the end of a day it is just another tool. Brings advantage when used properly.
But I use the web interface, not the app.
That said, sharing the output with others is not necessary to get value out of it.
For example: "Help me work through an [idea/plan/problem] by asking the next Socratic-method-style question."
But for quickly creating some template when i want to write a big report or email for something, then yes it's very useful.
ChatGPT is about as accurate as random websites on the internet, and you don’t get obliterated with ads.
Simple example, ChatGPT will give you a clear recipe for whatever you want, sans life story designed to make you scroll past a million ads.
In general, the context of search gives some insight into the credibility of the source.
Only way to know if a recipe is good is to look at it.
The only reason those blobs of text exist is to get you to look at more ads. Put more things in your head against your will, sell you more garbage, and manipulate your feelings.
If it wasn’t true, why are the recipes always at the bottom? Why not put the most valuable part right front and center? These websites have no respect for you and likely copy pasted the recipe anyways.
I could specify for it to use MDN exclusively but at that point I might as well use search.
In addition to that I could judge the quality of search results (a lot vs little mentions of a technology, shady vs reputable site etc.) to make educated guess of the output I'm getting from search. Can't do that with GPT.
These are key differences off the top of my head.
You don’t. But i’dtrust a top rated Stackoverflow answer over whatever LLM spits out.
There is no “confidence score” from an LLM output. You cannot tell whether it is making things up (and potentially make very bad decisions based on it’s output)
https://community.openai.com/t/new-assistants-api-a-potentia...
Try something more complicated! Ask for a gingerbread recipe without sugar, for example.
I think I'll ask it for a calzone recipe this weekend. The one I use now makes the dough a little too bready.
It's not very good at it, but it doesn't need to be, to be far better than I am.
People have some sense that someone giving them information may be an {expert, charlatan, idiot}, or that a website they’re looking at is run by a university vs a blogspam content farm, but many have not developed a sense for when or how much they can trust LLM output, which is delivered with the same tone and confidence regardless of whether it’s entirely fabricated.
There is probably a component of personality involved in how people approach this. Collectively we are all learning how to interact with this new source of information and people take varying paths.
I've had ChatGPT return very serviceable "true" results and I've had ChatGPT return utter fiction.
1. you don't know the answer, but you can check yourself and easily whether a given answer is roughly correct
2. you don't know the answer and wouldn't be able to check how valid a potential answer is
LLM-based tools are great for 1 to synthesize various sources into one coherent answer, since in this case, you won't become a victim of their hallucination. E.g. "write a one-off Python script to do this": you can quickly check if it does the job, even though you couldn't say whether that's idiomatic Python.
I would say it is not good at giving a sophisticated answer to anything that requires a lot of nuance. And I've also asked it questions with fairly objective factual answers that it gets hilariously wrong.