I, for one, look forward to a future in which I can ask a digital assistant to find and summarize information on anything:
"Computer, please find out how GPS works and explain it to me like I'm five years old."
"Computer, what are the latest Covid-19 infection stats in my city?"
"Computer, who is Claude Shannon and why is he important?"
"Computer, what are the top three stories on HN today?"
"Computer, what's on my calendar?"
etc.
Edit
I think the direction of these models seems clear, and their capabilities however limited at the moment are outstanding. I think the best measure of performance is what the naysayers complain about. We've quickly gone from "actual gibberish that looks like paragraphs" to "yes but it loses track after a few sentences" to "yes it's kind of coherent but doesn't make sense totally" and are now at "it's like an overconfident human at many tasks, when they're only somewhat qualified in the area". People are pointing out where it's got an API call wrong, without it having any ability to look up the answer and see if the function actually exists.
I spoke to it about distributed computation design and discovered something new I can actually implement in my design.
I'm very excited about where this will all go because it seems useful now and I'd be very surprised if this is the best that it ever reaches.
The important thing is that these AI systems are only going to get better over time.
IMHO, the early 2020's for conversational AI is akin to the 1980's for personal computers.
Anyone not looking at chatgpt and thinking "what if this was even just a bit better? Or had better iterations or lookups?" is missing out on what I think is a revolutionary change coming probably sooner than people expect.
ChatGPT:
GPS stands for Global Positioning System. It is a way that we can figure out where we are in the world using special satellites.
Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so they can see everything on the ground.
When you have a GPS device, like a phone or a car, it can talk to these satellites and ask, "Where am I?" The satellites send back a message that says, "You are here!" and gives your device a special code.
The GPS device can use this code to figure out exactly where you are on a map. It's kind of like a magic trick! You can use GPS to find out where you are, or to find your way to a new place. It's super helpful when you're lost or want to go on an adventure.
Not bad!
As I mentioned elsewhere, conversational AIs in the early 2020's are like PCs in the 1980's -- a new technology that still has lots of early limitations.
(Also, the UK comedy quiz show QI made the exact same mistake).
How will future LLMs be able to conduct that level of reasoning?
I feel it's appropriate in terms of simplicity, but I think providing a simple wrong explanation is really only worth it when the more accurate explanation is substantially harder to simplify, and I don't think explaining how GPS works falls into that category.
Something like this I think is more accurate without being any more complex:
> Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so you can always see some of them.
> When you have a GPS device, like a phone or a car, it can look for these satellites, and if it sees enough of them it can use math to figure out where you are.
> The satellites all broadcast a special code saying where each of them is. When you have a GPS device, like a phone or a car, it can listen to these codes and find out where and how far away different satellites are.
(Actually, this would be a fun thing to do with a class. Have a few kids march around the edges of the playground, saying things like “I’m next to the swingset now!”, and then ask one one kid to close their eyes, put them somewhere in the middle and ask them to guess where they’re standing.)
Something GPT discourse has been demonstrating to me is that I'm not usually a very careful reader. I apparently skim a lot. Or maybe I skim GPT outputs because I'm biased in my expectations already?
Which one is more appropriate and/or typical will probably depend on what questions the five-year-old is asking, but I think it's reasonable to say the former is usually preferable to the latter.
It does in a sense because the radio waves need an approximate line of sight to reach your GPS receiver. Being high up gives them a large coverage.
About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
You'd be better off just saying "it uses satellites and stuff".
I think that's one of the biggest problems with using LLMs for accurate answers. A lot of the text that's very useful for modelling human language and questions is also full of factual errors and questionable opinions. Answers being wrong is a problem with old-fashioned web search too, but I think LLMs necessarily lean more on quantity rather than quality of content...
If you can do "here is the structured data, summarize it and make sure to present this information" it can do that quite well.
So far I've played with GPT doing a zero shot classification of HN titles and CSV hourly weather data to summary. In each case, it has done quite well.
Neither of those are things about what it "knows."
Ian Calvert Computer, please find out how GPS works and explain it to me like I'm five years old
Sure thing! GPS stands for Global Positioning System. It's a way to find out where you are on Earth.
GPS works by using satellites that are way up in space. There are lots of these satellites orbiting around Earth, and they are always sending out signals. Your GPS receiver is a special device that can pick up these signals from the satellites.
The GPS receiver knows how far away it is from each satellite because of the time it takes for the signal to reach it. By figuring out how far away it is from a few different satellites, the GPS receiver can figure out exactly where it is on Earth.
So, if you have a GPS receiver in your car, phone, or on a special device like a watch, it can use the signals from the satellites to tell you exactly where you are and how to get to where you want to go. Does that make sense?
---
I was also able to ask it to make it simpler and simpler and it did so pretty well.
You'd better know enough (and be alert enough) to tell the difference...
Google, and all the other search engines, literally show snippets of the original content to display search results, and have had copyright infringement lawsuits over this.
Yet somehow, it's taken these AI, which generally aren't just block-quoting things unless you ask them to (and often getting it wrong if you do), for non-lawyers to start asking that question.
GPT definitely has some kind of internal model of things and how they relate to other things, and the capacity to express those relations in different ways.
Whether this is functionally the same as the model in my head, or some analog to it, or if such descriptions are merely cute but misleading anthropomorphisation, is far beyond my knowledge of neuroscience, psychology, and indeed machine learning.
Science.
Either way I don't know. Might be the same, but that might be an unwarranted anthropomorphisation.
So the explanation is either: satellites and their position or give the full explanation. Not this regurgitation.
Explain like Im five means break it down, not lie about the complicated bits.
Its terrible. This is totally not how GPS works, I.E. the terminal never sends a message to the satellite. This is the danger of early LLMs, a large amount of their output is false and there is no way to tell unless you are already an expert.
This makes them basically useless except for a toy.
Probably the “false output” you mentioned is due to the fact that the prompt asked to “explain it like I’m five”, which made ChatGPT answer with a “dialogue between terminal and satellite” explanation, which (arguably) may be better understood by a 5 years old.
Imagine that the GPS satellites orbiting the Earth are like the magic lanterns in the sky, sending signals down to a receiver (like the one in your phone or car). The receiver can use those signals to figure out how far away it is from each of the magic lanterns. By measuring the distance to multiple lanterns, the receiver can triangulate its position and figure out exactly where it is on the surface of the Earth. So even if you're a big, green ogre like Shrek, you can use GPS to find your way around and never get lost!
My mother was into all kinds of new age and supernatural stuff. You may think you can see where this anecdote is leading, and oh boy I could go there, but instead prepare for a twist! One of the stories she had was a client saying "It is not possible, Susan, it is not possible that there are beings out there that are cleverer than what we is".
And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
Don't think that's automatically true though: an AI can be consistently excellent at retrieving reliable sources and still get a lot wrong on its summaries. Also humans have their own opinions on sources even when they're not familiar with the detail, and finding out if the AI reads Nature or 4chan is part of the debugging process...
I've gotten it to fail on some word problems, but even then it was impressive to see it try to come up with a solution. More humorous were requests like 'Create a post for ycombinator about ChatGPT in the style of Chris Farley'. It's surprising and quite welcome that it even attempts that.
"Computer, can you write and deliver a message to my kid that explains how GPS works?"
"Computer, can you tell me what risks there are at dining at McDonalds down the street?"
"Computer, can you read this paper [link] and tell me why it's referencing Claude Shannon?"
"Computer, I need to waste some time - please use HN to help me"
"Computer, tell me about any plans that I have for today"
”I’m sorry, I cannot answer that query because it has ableist connotations. I am a large language model by OpenAI.”