Tunneling.
152 karma · joined February 20, 2021
Don't know about X? Trouble getting started with X?
Just ask ChatGPT! What could go wrong?
"vibe-suicide"
Guard rails = fig leaves
s/prove/produce/g
I'm inclined to regard an LLM as modelling a collection of fuzzy production rules which occur in a hierarchical collection of semi-formal systems; an LLM attempts to produce typographically correct theorems, the proving occurs at the level of semantics. Meaning requires a mind to erect an isomorphic mapping which the LLM is not capable of. In other words, for the LLM the math is just symbols on a page that are arranged according to the typographic rules which it has an imperfect model of. On this view, nothing about what is happening with Gen AI is particularly surprising or novel.
Is an answer that is correct by chance the same as one that is correct by reason?
If I ask a factual question of AI it will issue some output. In order for me to check that output, which I am apparently bound to do in all cases, I must check reliable sources, perhaps several. But that is precisely the work I wanted to avoid by using AI. Ergo, the AI has increased my work load because I had the extra useless step of asking the AI. Obviously, I could have simply checked several reliable sources in the first place. I see this as the razor at work.
It ought to be clear now that the use of AI for factual questions entails that it be trustworthy; when you ask an AI a factual question, the work you are hoping to avoid is equal to the work of checking the AI output. Hence, no time can ever be saved by asking factual questions of an untrustworthy AI.
QED
P.S. This argument, and its extensions, occurred to me and my advisors 25 years ago. It caused me to conclude that building anything other than a near perfect AI is pointless, except as a research project to discover the path to a nearly perfect AI. Nearly perfect should be interpreted to be something like "as reliable as the brakes on your car" in terms of MTBF.
No, I mean exactly what I said. Given a system's state, one evolves the state using wave equation du jour, nondeterminism does not occur.
The central analogy of the article is entirely bogus.
This article does not rise to the level of being wrong.
So maybe that is a test, or something to strive for. If you can get a new developer to make a moderate change to your code base on their first day successfully, then your code is fine. If not you have work to do.
To expand a little on this thought, any one developer's experience is probably not significant but if you hire say 10 new devs and 6-8 are making changes on their first day or in the first few days, your code base is likely fine.
I recomend that a new dev should first make the most trivial change possible but see it through all the way to release, to expose them to process. Following that, a moderate change to expose them to paradigm. All on the first day or in the first few days. If only say 3 of 10 new hires can accomplish the above, the problem is in your code base (or hiring practices).
A better way to get at intelligence metrics would be to test over a number of years in many ways, again for humans.
It seems fairly obvious to me that an LLM is the projection intelligence in the language domain. In other words, if you killed Intelligence and gave it a push in direction of language, the chalk outline you could draw around its dead body on the ground would be an LLM.
Full disclosure: I have taken 2 IQ tests, both online and timed. First was in late 90's after graduated electonics eng. was free, scored 149. After 4 years and obtaining theoretical physics degree, I did another scoring 169. The second test was not free, but I did not pay. I got the second test results because the test site owner personally emailed me my results for free with congrads, because they were the highest ever recorded on the site to date. I did both for fun just see the questions, I think both results are meaningless, the same variability occurs on farmers studied as mentioned above.
1) As far as I recall this program of formalizing mathematics fails unless you banish autoregression.
2) It is important to point out that a theorem in this context is not the same as a "Theorem" from mathematics. Production rules generate theorems that comply with rules and axioms of the formal system, ensuring that they could have meaning in that formal system. The meaning cannot justify the rules though, fortunately, most know to use the rules of logic so that we are not grunting beasts, incapable of conveying information.
I think the author wonders why theorems that don't seem to have meanings appear in the output of AI.
Dozens or perhaps hundreds of people have considered embarking on the brute force approach being pursued, nobody decided to try until a few years back because it is mathematically limited, costly, and might just make our problems much worse before it can make them better in any important way.
What is very salient is that 10 of the most powerful corporations and the leaders of the largest countries have decided on this route and they could force all software engineers to wear extra long, wide red ties if they wanted to so buckle up. It looks like we are riding this train all the way in the hope that cash + time will solve the truncation of the science.
It's like living in an Escher painting.
Most of what humans think of as reason is actually "will to power". The capability to use our faculties in a way that produces logical conclusions seems like an evolutionary accident, an off-lable use of the brain's machinery for complex social interaction. Most people never learn to catch themselves doing the former when they intended to engage in the latter, some don't know the difference. Fortunately, the latter provides a means of self-correction, the research here hopes to elucidate whether an LLM based reasoning system has the same property.
In other words, given consistent application of reason I would expect a human to eventually draw logically correct conclusions, decline to answer, rephrase the question, etc. But with an LLM, should I expect a non-determisitic infinite walk though plausible nonsense? I expect reaaoning to converge.
The only thing I said is: there are probably no _obvious_ mathematical certainties in reply to what you said, as the history of mathematics shows with plenty of examples."
There is nothing more obvious and certain mathematically than the fact that exponential growth of bubbles entails increasing valuations and increasing rates of growth in those valuations.
I use the word "obvious" quite properly as in "can be read from any graph", or "is contained within the structure or the model". I use categorical because the implied conclusion of the original comment is a categorical error.
In other words, the statement "I know I am not chasing a bubble because valuations are high and growing at an increasing rate" is categorically false because this property is a common indicator of the exact opposite; that you are in an exponentially growing bubble.