computers already do math. language models just need to translate problems into code of some kind that can be run to get the answer.
executive function/planning is probably the biggest problem at this point for ai.
executive function/planning is probably the biggest problem at this point for ai.
I went through a burnout in 2019 that felt like having a stroke. My brain finally reached such a level of negative reinforcement after years of failure that it wouldn't let me work anymore. I'd go to do very simple tasks, everything from brushing my teath to writing a TODO list, and it was like the part of my brain that performed those tasks wasn't there anymore. Or at least, it no longer obeyed if it perceived a potential reward involved. It was like my motivation got reversed. I had to relearn how to do everything, despite knowing that no reward might come for a very long time, which took at least 6 months before I began recovering. The closest answer I have is that my brain healed through faith.
I only bring it up because executive function may be associated with a subjective experience of meaning. If there's truly no point to anything, then it's hard to summon the motivation to string together a sequence of AI tasks into something more like AGI.
I guess that's another way of saying that nihilism could be the final hurdle for AGI to overcome. It's like the human philosophical question of why there's something instead of nothing. Or why angels would choose to be incarnate on Earth to experience a life of suffering when it's so much easier to remain dissociated.
This is the glaring hole in LLMs, a paradoxical semantic incoherence despite impressive sentenial and gramatical coherence.
As glue it is so thin as to be potable.
"Here's a brief glimpse of our INCREDIBLE near future.
GPT-3 armed with a Python interpreter can · do exact math · make API requests · answer in unprecedented ways"
[0]https://twitter.com/sergeykarayev/status/1569377881440276481
There is a fairly simple program in
https://www.amazon.com/Paradigms-Artificial-Intelligence-Pro...
that solves word problems using the methods of the old AI. The point is that is is efficient and effective to use real math operators and not expect to fit numbers through the mysterious bottleneck of neural encoding.
A huge "just"! Isn't this the magic step? Translating ambiguous symbols to meaning and combining them in meaningful ways is a big deal which, apparently, these AI models cannot do. They can just parrot things.
GPT3 can't turn as many problems into code as I can, but it can do some, and GPT4 (or whatever) will be able to do more, etc.
Plenty of AI models do exactly this. Very clear examples include question answering models and code generation. In both cases novel, meaningful responses are generated.
> They can just parrot things.
That isn't true. While language models can parrot things it is generally special conditions that make them do it. Specifically, the conditional probability of the next character (or BPE or word depending on the model) has to be much higher than anything else which happens when the thing being parroted is unique text.
If you ask most Americans or a language model what word comes next in this: fourscore and seven year.. they'll give the same answer, for the same reason.
In my opinion, it's not solved. GPT-3 is not General AI, it's a more clever mechanism for parroting back text it cannot truly understand. Comparisons to ways humans confuse themselves are a red herring in my opinion: the old ELIZA program could reply like a very confused or trollish human would, but nobody would argue ELIZA was a general AI.
It's just that GPT is a fascinating and more convincing illusion than ELIZA. Unlike ELIZA, it can also be used for meaningful purposes.
I think that in general most human decision making is just pattern matching (plenty of evidence for this - read "thinking fast and slow" for an overview).
I think the extrapolation that ML models can do is a form of intelligence. I also think that the compression and encoding of inputs into a lower dimensional space is exactly the "turning symbols into meaning" that you call for in your definition.
We obviously disagree. I don't think we are near general AI (and yes, I know the objection that everyone who says this is simply moving the goalposts-- regardless, I'm unconvinced). I think GPT et al are very interesting tricks, but still not general AI; and that the path to it doesn't lie in this direction.
I subscribe to the view we think of the human mind as a pattern matching computer simply because this is the current major tech, much like people in the past thought of "humors" or "steam machines". I think some of the analogies are useful, to a point, but I don't think there's hard evidence the mind is like a neural net (irony notwithstanding) or a pattern marching GPT-like algorithm.
Re: Thinking Fast and Slow, I see there are serious doubts about the validity of the book's foundations and conclusions, and that it's been challenged.
Nor do I. But I don't agree with your definition of general AI at all.
I think by your definition we are well on the way towards it. I'd note that you didn't address the idea that compression of input concepts into a lower dimensional space is exactly the "reliably turning symbols into meaning" idea you suggest.
The fact the latent representations of concepts can be manipulated in ways that make logical sense is a good indication that the symbols have meaning. The classic Word2Vec experiments showing how the relationships Paris->France ~= London->England and King - Man + Woman = Queen show this well. Modern large language models are much more complicated of course, but the principles remain.
> I subscribe to the view we think of the human mind as a pattern matching computer simply because this is the current major tech, much like people in the past thought of "humors" or "steam machines". I think some of the analogies are useful, to a point, but I don't think there's hard evidence the mind is like a neural net (irony notwithstanding) or a pattern marching GPT-like algorithm.
I don't think the computational approach is an interesting question. No one thinks brains operate like software neural networks (not sure what "pattern marching GPT-like algorithm" means - it is just a neural network). That doesn't matter because all computational methods are ultimately equivalent.
I think outcomes on metrics like benchmarks is important, and while some benchmarks have issues that some approaches exploit I think things like Chollet's "On the Measure of Intelligence" (https://arxiv.org/abs/1911.01547) are reasonable frameworks for discussion of progress.
> Re: Thinking Fast and Slow, I see there are serious doubts about the validity of the book's foundations and conclusions, and that it's been challenged.
This in inaccurate. There are some reasonable criticisms of the studies on the priming affect (one chapter of the book). These and Kahneman's response (he accepts the criticisms) are outlined in https://replicationindex.com/category/thinking-fast-and-slow... and https://retractionwatch.com/2017/02/20/placed-much-faith-und...
These issues don't detract from the overall theme of the book about the two systems of decision making (rational and reflex) and how often we use the reflex decision system but convince ourselves we are using the rational system.
If you can point at any additional doubts about the books conclusions I'd appreciate a reference.
Great, so we agree then!
> No one thinks brains operate like software neural networks
A bold claim.
> (not sure what "pattern marching GPT-like algorithm" means - it is just a neural network).
Yet you understood me perfectly.
Anyway, you are more well read than me on this subject, yet we both agree we are not near general AI.
> Great, so we agree then!
Maybe? I think all forms of computation are equivalent, so if the brain is implemented in the same way as a neural network is uninteresting. They can do the same thing (which is interesting).
> yet we both agree we are not near general AI.
Sure. My disagreement is with the idea with your original statement:
"Translating ambiguous symbols to meaning and combining them in meaningful ways is a big deal which, apparently, these AI models cannot do. They can just parrot things."
Modern AI systems can do this, and do not just parrot things.
They are capable of novel outputs.
This is because the models have sufficient "understanding" to manipulate "things" (latent vector representations, which you call symbols) to output novel but meaningful outputs.
https://www.wolframalpha.com/calculators/integral-calculator...
https://en.wikipedia.org/wiki/Automated_theorem_proving
you just don't hear about it much because the technology is not so fashionable today. Also it is more clear what the limits are, I mean, Turing, Godel, Tarski and all of those apply to neural networks as well any other formal system but people mostly forget it.
Knuth wrote a really fun volume of The Art of Computer Programming about advances in SAT solvers which are the foundation for theorem provers
https://www.amazon.com/Art-Computer-Programming-Fascicle-Sat...
Everybody is aware that neural network techniques have improved drastically in performance, it's much more obscure that the toolbox of symbolic A.I. has improved greatly. Back in the 1980s production rules engines struggled to handle 10,000 rules, now Drools can handle 1,000,000+ rules with no problems.
It doesn't seem like there has been much progress for anything but FOL?
SMT solvers have gotten quite good over the past couple decades, there are tons of domain-specific tools (eg in software and hardware verification), tons of niche applied decidable or semi-decidable theories (eg various modal and description logics), a lot of progress on the proof assistant ("non-fully-automated theorem proving") paradigm, and so on.
It's not hard to add some extensions to logic for a particular application but a very hard problem to develop a general purpose extended logic.
I look at the logic-adjacent production rules systems which never really standardized some of the commonly necessary things such as agendas, priorities, defaults, etc.
Computers can solve math problems and prove theorems; this remains a significant subfield of Computer Science with lots of industrial use cases. However, pure machine learning based approaches toward these problems remain subpar.
> Even mathematical objects are approximated up to an approximation error in a computer (like a differentiable manifold or a real number).
Only because it caught on (and in the case of non-computationally-intensive applications, for purely historical reasons). For example, Mathematica has Reals and even functionality for Reals that is literally impossible to implement for integers [1,2]. There are also precise characterizations of objects in differential geometry [3]. You could imagine applying LLMs to these types of programs a la Copilot, but when you do this you will find yourself agreeing with Paul Houle's observation that math is harder to fake than eg art, language, or even glue code for web apps.
[1] https://reference.wolfram.com/language/ref/Reduce.html
[2] https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...
But the specification of the problem must be done by a human, translating to a formalized system that the software can understand. And if there's a problem in the formal specification, it's mostly up to the human to notice and fix; the computer will happily output garbage or crash or enter an infinite loop.
So it seems this translation, going from an exploration of the problem statement, usually in ambiguous terms, to a formal specification, and the awareness to possibly detect whether the answers make sense and the specs were right, is uniquely human.
I think sharemywin is probably on to something. It's going to be really hard for an AI to prove that e.g. x>0 && x+y <= 1 && y>1 is unsatisfiable, but it's trivial for an SMT solver. On the other hand it probably isn't that much of a leap to make an AI that can feed that problem into an SMT solver.