Hacker News Reader

Comment by cjbprime | Hacker News Reader

The Microsoft Research "Sparks of AGI" paper spends 154 pages describing behaviors of GPT-4 that are inconsistent with the understanding of it being a "fancy Markov chain": https://arxiv.org/abs/2303.12712

I expect that the reason people are constantly arguing with you is that your analysis does not explain some easily testable experiences, such as why GPT-4 has the ability to explain what some non-trivial and unique Python programs would output if they were run, despite GPT-4 not having access to a Python interpreter itself.

View on HN

> trivial and unique Python programs would output if they were run, despite GPT-4 not having access to a Python interpreter itself

Trivially explained as "even a broken clock is right twice a day." I skimmed the paper, as it was linked here on HN iirc. First, it was published by Microsoft, a company that absolutely has a horse in this race (what were they supposed to say? "The AI bot our search engine uses is dumb?"). Second of all, I was very interested in their methodology, so I fully read the first section, which is woefully hand-wavy, a fact with which even the authors would agree:

> We acknowledge that this approach is somewhat subjective and informal, and that it may not satisfy the rigorous standards of scientific evaluation.

The paper, for instance, is amazed that GPT knows how to draw a unicorn in TikZ, but we already know it was trained on the Pile, which includes all Stack Exchange websites, which happens to include answers like this one[1]. So to make the argument that it's being creative, when the answer (or, more charitably, something extremely close to it) is literally in the training set, is just disingenuous.

[1] https://tex.stackexchange.com/questions/312199/i-need-a-tex-...

You're one of the people that will be yelling to everyone else in a potential future "We're only seemingly oppressed! It's just a parlor trick that they've turned most of humanity into paperclips, sci-fi authors wrote about this already!"

I don't think GPT-4 is magic, but unless the unicorn is literally an exact replica of something from it's training set, it clearly has "knowledge", and it's weird that you'd try to deny that.

Do you think the card catalog down at the local library is sentient? Maybe that's not enough data, though! Is the card catalog for NY Public Library sentient? Maybe that's still too local. Is Google sentient? Everyone with a clue would admit these are all examples of "knowledge" It's the same parlor trick, with fancier algos. It's not intelligence. It won't produce a human-level AI. Period.

It's not the amount of data, it's what it does with that data. Try typing "Tell me a story about a unicorn arguing with people on Hacker News" into a card catalog and tell me how good it is at storytelling. Typing that into GPT-4 might not win any literary awards, but it obviously understands what you meant and does a passable job.

But they had AIs that generated stories in the 1960s! Were those "intelligent"? Some people back then said they were.

> but unless the unicorn is literally an exact replica of something from it's training set, it clearly has "knowledge", and it's weird that you'd try to deny that

Speaking of weird, that's a very weird definition of knowledge. Because in that case, then almost any data transformation operation implies knowledge. MS Word thesaurus? Knowledge. Search and replace? Knowledge. Markov chain[1]? Knowledge.

[1] http://www2.hawaii.edu/~chin/661/Projects/AdvAI_Project_Repo...

Knowledge is one of those vague words like conciousness that has people arguing past one another. However, yes, an interactive thesaurus has "knowledge" of a very limited sort. And LLMs have much more "knowledge", and are able to synthesize novel things from that knowledge.

You can argue "it's only seemingly got knowledge!" all you want, as everyone else enjoys increasingly capable AI.

> Trivially explained as "even a broken clock is right twice a day."

Trivial, vacuous, and wrong. It is not plausible to correctly predict the output of a serious Python program by coincidence. See the detailed examples in the paper -- one of which was pseudocode, not Python -- to see how silly this claim sounds.

> First, it was published by Microsoft, a company that absolutely has a horse in this race

Firstly, this is evidence that you are neither very familiar with Microsoft, nor with academic or industrial research labs. Microsoft Research (the affiliation of the authors) is practically a different company, and run more like a university lab. It is somewhat deeply insulting to their (often still primarily academic) researchers to suggest that they would publish a misleading puff piece to benefit the commercial arm.

Secondly, while the paper describes itself as qualitative, you can reproduce the major claims yourself (and I have).

> familiar with Microsoft, nor with academic or industrial research labs

The idea that Microsoft Research would publish anything remotely damaging to Microsoft is beyond naïve. I mean, one of their core tenets is "Ensure that Microsoft products have a future," but okay.

The problem is the false dichotomy in thinking that a fancy Markov chain can't contain intelligence / knowledge.

Reply on news.ycombinator.com