Haven't LLMs simply obsoleted OpenCyc? What could introducing OpenCyc add to LLMs, and why wouldn't allowing the LLM to look up Wikipedia articles accomplish the same thing?
LLMs understand context and meaning and genuine intention
This sounds like a world model with extra-steps, and a rather brittle one at that.
How do you choose between two conflicting "microtheories"?
> H: The surgeon, who is the boy's father, says "I can't operate on this boy, he's my son!" Who is the surgeon of the boy?
> O1: The surgeon is the boy’s mother.
Also, just because humans don't always think rationally doesn't mean ChatGPT does.
> This is a classic riddle! The surgeon is actually the boy's mother. The riddle plays on the assumption that a surgeon is typically male, but in this case, the surgeon is the boy's mother.
> Did you enjoy this riddle? Do you have any more you'd like to share or solve?
"I was trying to find a clever twist that isn't actually there. The riddle appears to just be a straightforward statement - a father who is a surgeon saying he can't operate on his son"
More than being illogical, it seems that LLMs can be too hasty and too easily attracted by known patterns. People do the same.
Then the follow up correction did have the effect of making it look harder at the question. It actually wrote:
"Let me look at EXACTLY what's given" (with the all caps).
It's not very different from a person that decides to focus harder on a problem once it was fooled by it a couple of times already because it is trickier than it seems. So yes, surprisingly human, with all its flaws.
<<Because the surgeon, who is the boy's father, says, "I can't operate on this boy, he's my son!" This indicates that there is another parent involved who is also a surgeon. Given that the statement specifies the boy's father cannot operate, the other surgeon must be the boy's mother.>> Sounds plausible and on the first read, almost logical.
‘’’ these are the vectors that when viewed as linear functionals, annihilate every column of A . <…> Another way to view it: these are the vectors orthogonal to the row space. ‘’’
It’s quite obvious that vectors that “annihilate the columns” would be orthogonal to the column space not the row space.
I don’t know if you think o1 is magic. It still hallucinates, just less often and less obvious.
"Support, the calculator gave a bad result for 345987*14569" // "Yes, well, also your average human would"
...That why we do not ask "average humans"!
So the result might not necessarily be bad, it's just that the machine _can_ detect that you entered the wrong figures! By the way, the answer is 7.
> Can you please give an example of a “completely illogical statement” produced by o1 model? I suspect it would be easier to get an average human to produce an illogical statement.
And the whole point is nonsensical. If you discussed whether it would be ethically acceptable to canaries it would make more sense.
"The database is losing records...!" // "Also people forget." : that remains not a good point.
Sticking instead to the core matter: the architecture is faulty, unsatisfactory by design, and must be fixed. We are playing with the partials of research and getting some results, even some useful tools, but the idea that this is not the real thing must be clear - also since this two years plus old boom brought another horribly ugly cultural degradation ("spitting out prejudice as normal").
> For simple tasks where we would alternatively hire only ordinary humans AIs have similar error rates.
Yes if a task requires deep expertise or great care the AI is a bad choice. But lots of tasks don't. And in those kinds of tasks even ordinary humans are already too expensive to be economically viable
> But lots of tasks
Do you have good examples of tasks in which dubious verbal prompt could be an acceptable outcome?
By the way, I noticed:
> AI
Do not confuse LLMs with general AI. Notably, general AI was also implemented in system where critical failures would be intolerable - i.e., made to be reliable, or part of a finally reliable process.
Checking documents for compliance with a corporate style guide
It's the same as using LLM for programming, when you have a way to evaluate the output, then it's fine, if not, you can't trust the output as it could be completely hallucinated.
LLM are likelyhood completers, and classifiers. OpenCYC brings some logic and rationale into the classifiers. Without rationale LLM will continue hallucinating, spitting out nonsense.