We have made progress in efficiency, not functionality. Instead of searching google or stack overflow or any particular documentation, we just go to Chatgpt.
Information compression is cool, but I want actual AI.
Information compression is cool, but I want actual AI.
Your whole brain might just be doing "information compression" by that analogy. An LLM is sort of learning concepts. Even Word2Vec "learned" than king - male + female = queen and that's a small model that's really just one part (not exact, but similar) of a transformer.
One level deep information compression is cool, but I want actual AI.
Its true that our brains compress information, but we compress it in a much more complex manner, in the sense that we can not only recall stuff, but also execute a decision tree that often involves physical actions to find the answer we are looking for.
The minute you take a token and turn it into an embedding, then start changing the numbers in that embedding based on other embeddings and learned weights, you are playing around with concepts.
As for executing a decision tree, ReAct or Tree of Thought or Graph of Thought is doing that. It might not be doing it as well as a human does, on certain tasks, but it's pretty darn amazing.
Sort of. You can get LLMs to produce some new things, but these are statistical averages of existing information. Its kinda like a static "knowledge tree", where it can do some interpolation, but even then, its interpolation based on statistically occurring text.
(if not obvious.. you'd shove it in right after the embedding layer...)
Is Ibn Sina (Avicenna, year ~1000) fine?
> [the higher faculty proper of humans is] the primary function of a natural body possessing organs in so far as it commits acts of rational choice and deduction through opinion; and in so far as it perceives universal matters
Or, "Intelligence is the ability to reason, determining concepts".
(And a proper artificial such thing is something that does it well.)
It is a tool that can be given a project in language X and produce an idomatic port in language Y.
It is a tool that given a 20 pages paper spec will ask the questions needed to clarify the specs.