Models have been improving. By induction they’ll continue until we see them stop. There is no prevailing understanding of models that lets us predict a parameter and/or training set size after which they’ll plateau. So arguing “how do we know they’ll get better” is the same as arguing “how do we know the sun will rise tomorrow”… We don’t, technically, but experience shows it’s the likely outcome.
The LLM true believers have decided that (a) hallucinations will eventually go away as these models improve, it's just a matter of time; and (b) people who complain about hallucinations are setting the bar too high and ignoring the fact that humans themselves hallucinate too, so their complaints are not to be taken seriously.
In other words, logic is not going to win this argument. I don't know what will.
What I think is actually happening is that some people innately have taken the stance that it’s impossible for an AI model to be useful if it ever hallucinates, and they probably always will hallucinate to some degree or under some conditions, ergo they will never be useful. End of story.
I agree it’s stupid to try and inductively reason that AI models will stop hallucinating, but that was never actually my argument.
This is because “hallucinate” means very different things in the human and LLM context. Humans have false/inaccurate memories all the time, and those are closer to what LLM “hallucination” represents than humam hallucinations are.
We are interacting with multidimensional topological manifolds, and the context we create has a topology within this manifold that constrains the range of output to the fuzzy multidimensional boundary of a geodesic that is the shortest route between our topology and the LLM.
I think some visualisation tools are badly needed, viewing what is happening is for me a very promising avenue to explore with regards to emergent behaviour.
GPT4 says; When interacting with a large language model (LLM) like GPT-4, we engage in a complex and multidimensional process. The context we establish – through our inputs and the responses of the LLM – forms a structured space of possibilities within the broader realm of all possible interactions.
The current context shapes the potential responses of the model, narrowing down the vast range of possible outputs. This boundary of plausible responses could be seen as a high-dimensional 'fuzzy frontier'. The model attempts to navigate this frontier to provide relevant and coherent responses, somewhat akin to finding an optimal path – a geodesic – within the constraints of the existing conversation.
In essence, every interaction with the LLM is a journey through this high-dimensional conversational space. The challenge for the model is to generate responses that maintain coherence and relevancy, effectively bridging the gap between the user's inputs and the vast knowledge that the LLM has been trained on."
There are a few recent Nova specials from PBS that are on YouTube that show just how much bullshit we imagine and make up at any given time. It's mostly our much older and simpler systems below intelligence that keep us grounded in reality.
Memory is far from infallible but human brains do contain knowledge and are capable of introspection. There can be false confidence, sure, but there can also be uncertainty, and that's vital. LLMs just predict the next token. There's not even the concept of knowledge beyond the prompt, just probabilities that happen to fall mostly the right way most of the time.
On the other hand, the problem of getting people to trust AI in sensitive contexts where there could be a lot at stake is non-trivial, and I believe people will definitely demand better-than-human ability in many cases, so pointing out that humans hallucinate is not a great answer. This isn't entirely irrational either: LLMs do things that humans don't, and humans do things that LLMs don't, so it's pretty tricky to actually convince people that it's not just smoke and mirrors, that it can be trusted in tricky situations, etc. which is made harder by the fact that LLMs have trouble with logical reasoning[1] and seem to generally make shit up when there's no or low data rather than answering that it does not know. GPT-4 accomplishes impressive results with unfathomable amounts of training resources on some of the most cutting edge research, weaving together multiple models, and it is still not quite there.
If you want to know my personal opinion, I think it will probably get there. But I think in no way do we live in a world where it is a guaranteed certainty that language-oriented AI models are the answer to a lot of hard problems, or that it will get here really soon just because the research and progress has been crazy for a few years. Who knows where things will end up in the future. Laugh if you will, but there's plenty of time for another AI winter before these models advance to a point where they are considered reliable and safe for many tasks.
I mean this is what I was saying. I just don't think that the technology has to become hallucination-free to be useful. So my bad if I didn't catch the implicit assumption that "any hallucination is a dealbreaker so why even care about security" angle of the post I initially responded to.
My take is simply just that "these things are going to be used more and more as they improve so we better start worrying about supply chain and provenance sooner than later". I strongly doubt hallucination is going to stop them from being used despite the skeptics, and I suspect hallucination is a problem of lack of context moreso than innate shortcomings, but I'm no expert on that front.
And I'm someone who's been asked to try and add AI to a product and had the effort ultimately fail because the model hallucinated at the wrong times... so I well understand the dynamics.
In particular you absolutely can't just continue to extrapolate short-term phenomena out blindly into the future and pretend that has the same level of meaning as things like the sun rising which are the result of fundamental mechanisms that have been observed, explored and understood iteratively better and better over an extremely long time.
There are two things that might change- the sun stops shining, or the earth stops moving. Of the known possible ways for either of those things to happen, we can fairly conclusively say neither will be an issue in our lifetimes.
An asteroid coming out of the darkness of space and blowing a hole in the surface of the earth, kicking up such a dust cloud that we don't see the sun for years is a far more likely, if still statically improbable, scenario.
LLMs, by design, create combinations of characters that are disconnected from the concept of True, False, Right or Wrong.
A more classic example is Freud's deliniation between the id, ego and super-ego. Only the last is built upon imparted cultural mores; the id and ego are purely internal things. Disorders within the ego (excessive defense mechanisms) inhibit perception of what is true and false.
Chatbots / llms don't consider any of these things; they consider only what is the most likely response to a given input?. The result may, by coincidence, happen to be true.
The other thing we have a small number of observations of happening over the last 50 or 60 years but mostly the last 5 years or so. We know some of the mathematical features of the phenomena we are observing but not all and there is a great deal going on that we don't understand (emergence in particular). The things we are seeing contradict most of the academic field of linquistics so we don't have a theoretical basis for them either outside of the maths. The maths (linear algebra) we understand well, but we don't really understand why this particular formulation works so well on language related problems.
Probably the models will improve but we can't naively assume this will just continue. One very strong result we have seen time and time again is that there seems to be an exponential relationship between computation and trainingset size required and capability. So for every delta x increase we want in capability, we seem to pay (at least) x^n (n>1) in computation and training required. That says at some point increases in capability become infeasible unless much better architectures are discovered. It's not clear where that inflection point is.
These days: Modern technology allows us to monitor the location of the sun 24/7.
Less glibly I think models will follow the same sigmoid as everything else we’ve developed and at some point it’ll start to taper off and the amount of effort required to achieve better results becomes exponential.
I look at these models as a lossy compression logarithm with elegant query and reconstruction. Think JPEG quality slider. The first 75% of the slider the quality is okay and the size barely changes, but small deltas yield big wins. And like an ML hallucination the JPEG decompressor doesn’t know what parts of the image it filled in vs got exactly right.
But to get from 80% to 100% you basically need all the data from the input. There’s going to be a Shannon’s law type thing that quantifies this relationship in ML by someone who (not me) knows what they’re talking about. Maybe they already have?
These models will get better yes but only when they have access to google and bing’s full actual web indices.
Try to find out if a plant is toxic for cats via google. Many times the results say both yes and no and it's impossible to assume which one is true based on the count of the results.
Feeding the models more garbage data will not make the results any better.
It is obviously very approximative and will be wrong at some point, but there isn't much more to rely on.
I, for one, salute my 160-years-old grandma.
But when you try to predict a one-off event, you need to use whatever information is available.
One very valid application of the principle above is to never make plans with your significant other that are further off in the future than the duration of the relationship. So if you have been together for two months, don't book your summer vacation with them in December.
What rules and predictions can reliably describe how much machine learning will advance over time?
Says who? The Hot Hand Fallacy Division?
> It’s pretty rational
No, that's why it's a fallacy.
Why do you think this? We know how the sun works, how much nuclear fuel it has, and what life stages a star goes through as it uses up fuel, and how that life cycle changes based on size. We know the sun will stop shining, depending on your definition of that, in about 10 billion years. We know these things from studying THOUSANDS of other suns in various parts of their life cycle. We can make predictions on stars we observe, and watch them come true, which is the only valid judgement of a theory or model.
You not knowing something (like statistics) doesn't mean nobody knows it.
We have well understood theories about how we think the sun works based on observations of other suns, yes. But that's all.
You're muddying the waters willingly. This is intellectually dishonest.
Categorically it's the same problem. I just don't give any more credence to "centuries of data on orbital mechanics" for the purpose of this discussion about the the epistemological understanding of whether the sun will continue to exist or not at some specified point in time in the future.
Is it more likely based on track record/history that we'll still have a sun in 50 years than improved LLMs? Uh likely yes. I never argued one was more or less likely than the other. I only argued that the same logical reasoning/argument is used to come to the conclusion that we'll have a sun in the future as it is to deduce that LLMs will probably improve.
So unless you call epistemology dishonest, I'm not being dishonest. I'm pointing out something that people commonly glaze over in their practical day to day lives. I pointed it out because someone challenged my argument that LLMs will improve by saying essentially "well we don't know that". Of fucking course we don't. But we don't know that in the same way we don't know that the sun will rise tomorrow. That's all I'm saying. You're just missing the nuance and I don't know why you're resorting to calling it intellectually dishonest.
Yes it was called the Cold War.
Little tiny suns, but all those H-bombs (and reactors like the NIF and Z-pinch) verified quite a lot of the fundamentally identical physics.
For all we know there's something important we haven't observed about the sun's ability to consume its available fuel (whatever that mass is) and what happens to the exhaust products that could cause the sun to cool far sooner than we think. Who knows /shrug... not that I don't hope we've got it right in our understanding.
This by itself should be enough to pass the test of:
>> Have we experimentally recreated a sun and verified any of the theoretical models we have?
in the affirmative.
I mean, it's not like science requires 1:1 scale models.
> (or gravity for that matter)
Neither cheese, which is a similar non-sequitur.
[0] Including the fun fact that the sun is a "cold" fusion reactor, in the sense that it's primarily driven by quantum mechanical rather than high-energy ("thermo-nuclear") effects.
I'm not sure if this was first noted before or after the muon-catalysed fusion research.
Physics: the only place where someone looks at ten million K and goes "huh, that's cold".
No brother, it's science, and frankly that you believe this is not surprising to me at all.
Philosophy is great and all, but Newton gives you raw numbers that are then verified by reality. I'm going to rely on that instead of untestable breathless "but ACTUALLY" from people who provide no actionable insight into the universe.
Philosophy -> Math -> Physics -> Chemistry -> etc.
Everything to the right depends on, or is an application of, the discipline to the left. "Science" starts at physics.
What about Moore's law? Observing trends and predicting what might happen isn't a particularly new idea. You're not the only one, but I find it odd when people toss around the fallacy argument when a trend isn't pointing their way in an argument. I'm sure you use past trends to inform many of your thoughts each day.
Anyway, the point stands. The fallacy is believing with certainty that something will happen because of past events. That doesn't mean prediction is futile. Might want to re-read your wikipedia pages to better understand!
Just because there is no proof for the opposite yet doesn't mean the original hypothesis is true.
(a) the initial, intuitive belief that basketball players who had made several shots in a row were more likely to make the next one (b) the analytical analysis that disproved a, which no doubt stemmed from the belief that every shot must be totally independent of its context, disregarding the human factors at play (c) the revised analysis that found that the analysis in b was flawed, and there actually was such a thing as a "hot hand."
You know we're not talking about sports, right?
HN is wild.
Anyway, the lesson of the hot hand fallacy is that sometimes intuitive predictions turn out to be right, despite the best efforts of low-context contrarians. But I don't think that was your point.
You are the only one who is confused.
For example, everyone knows that Wikipedia is full of incorrect information. Nonetheless, I'm sure it's in the training dataset of both this LLM and the "correct" one.
So the answer to "why not start now" is "because it seems like it will be a waste of time".
> So the answer to "why not start now" is "because it seems like it will be a waste of time".
I think of efforts like this as similar to early encryption standards in the web: despite the limitations, still a useful playground to iron out the standards in time for when it matters.
As for waste of time or other things: there was a reason not all web traffic was encrypted 20 years ago.
Then as a verification step, you ask one more model, not the same one, "what information got inserted the last hour in the database?" Chances of one model to hallucinate and say it put the information in the database, and the other model to hallucinate again with the correct information, are pretty slim.
[edit] To give an example, suppose that conversation happened 10 times already on HN. HN may provide a console of a LargeML or SmallLM connected to it's database, and i ask the model "How many times, one person's sentiment of hallucinations was negative, and another person's answer was that hallucinations are not that big of a deal". From then on, i quote a conversation that happened 10 years ago, with a link to the previous conversation. That would enable more efficient communication.
Education involves doing some fact checking and critical thinking. Regardless of the strength of the original source.
It seems like using LLMs in any serious way will require a variety of techniques to mitigate their new, unique reasons for being unreliable.
Perhaps a “chain of model provenance” becomes an important one of these.
A chain of providence isn't much different then that person having a diploma, a company work badge, and state issued ID. You at least know they aren't some random off the street.
If not, it's just a diploma from some random organisation off the street.
So, wake me up when you know how OpenAI and Google are cooking their models.
Wikipedia may be reliable, but you should never cite anything on its own reliability lmao
Disinformation isn't random though; there's not an equal chance that information is misleading on ever topic.
Most information can be accurate while still containing dangerous amounts of disinformation.
Now we shouldn’t be letting a random blob of binary run commands though right? Well that is exactly what you are doing when you install say Chrome.
Found the venture capitalist!
AI performance often decreases at a logarithmic rate. Simply put, it likely will hit a ceiling, and very hard. To give a frame of reference, think of all the places that AI/ML already facilitate elements of your life (autocompletes, facial recognition, etc). Eventually, those hit a plateau that render it unenthusing. LLMs are destined for the same. Some will disagree, because its novelty is so enthralling, but at the end of the day, LLMs learned to engage with language in a rather superficial way when compared to how we do. As such, it will never capture the magic of denotation. Its ceiling is coming, and quickly, though I expect a few more emergent properties to appear before that point.
I mean, to some extent, but isn't reasonable to assume hallucination is a hard problem?
Hallucination shows there's plenty of things they didn't actually learn, and are just good at seeming they learned.
Like, if it gets exponentially harder to train them it's possible the level of hallucination will improve far worse than linearly even.
A language model isn't a fact database. You need to give the facts to the AI (either as a tool or as part of the prompt) and instruct it to form the answer only from there.
That 'never' goes wrong in my experience, but as another layer you could add explicit fact checking. Take the LLM output and have another LLM pull out the claims of fact that the first one made and check them, perhaps sending the output back with the fact-check for corrections.
For those saying "the models will improve", no. They will not. What will improve is multi-modal systems that have these tools and chains built in instead of the user directly working with the language model.