Comment by saurik | Hacker News Reader

Comment by saurik | Hacker News Reader

Parent Full thread

So, to verify, your claim is that GPT-3, when trained on a corpus of human text, isn't merely managing to string together a bunch of high-probability sequences of symbol constructs--which is how every article I have ever read on how it functions describes the technology--but is instead managing to build a model of the human world and the mechanism of narration required to describe it, with which it uses to write new prose... a claim you must make in order to then argue that GPT-3 works like a human engineer learning a model of computers, libraries, and engineering principals from which it can then write code, instead of merely using pattern recognition as I stated? As someone who spent years studying graduate linguistics and cognitive science (though admittedly 15-20 years ago, so I certainly haven't studied this model: I have only read about it occasionally in passing) I frankly think you are just trying to conflate levels of understanding, in order to make GPT-3 sound more magical than it is :/.

What? I don't think I made any claim of the sort. I'm claiming that it does more than mere regurgitation and has done some amount of abstraction, not that it has human-level understanding. As an example, GPT-3 learned some arithmetic and can solve basic math problems not in its training set. This is beyond pattern matching and replication, IMO.

I'm not really sure why we should consider Copilot legally different from a fancy pen – if you use it to write infringing code then that's infringement by the user, not the pen. This leaves the practical question of how often it will do so, and my impression is that it's not often.

It's not really comparable to a pen. Because a pen by itself doesn't copy someone else's code/written words. It's more like copying code from Github or if you wrote a script that did that automatically. You have to be actively cautious that the material that you are copying is not violating any copyrights. The problem is Copilot has enough sophistication to for example change variable names and make it very hard to do content matching. What I can guarantee it won't be able to do is to be able to generate novel code from scratch that does a particular function (source: I have a PhD in ML). This brute-force way of modeling computer programs (using a language model) is just not sophisticated enough to be able to reason and generate high level concepts at least today.

The argument I was responding to--made by the user crazygringo--was that GPT-3 trained on a model of the Windows source code is fine to use nigh unto indiscriminately, as supposedly Copilot is abstracting knowledge like a human engineer. I argued that it doesn't do that: that GPT-3 is a pattern recognize that not only theoretically just likes to memorize and regurgitate things, it has been shown to in practice. You then responded to my argument claiming that GPT-3 in fact... what? Are you actually defending crazygringo's argument or not? Note carefully that crazygringo explicitly even stated that copying little bits and pieces of a project is supposedly fair use, continuing the--as far as I understand, incorrect--assertion by lacker (the person who started this thread) that if you copied someone's binary tree implementation that would be fair use, as the two of them seem to believe that you have to copy essentially an entire combined work (whatever that means to them) for something to be infringing. Honestly, it now just seems like you decided to skip into the middle of a complex argument in an attempt to made some pedantic point: either you agree that GPT-3 is a human that is allowed to, as crazygringo insists, read and learn from anything and the use that knowledge in any way they see fit, or you agree with me that GPT-3 is a fancy pattern recognizer and it can and will just generate copyright infringements if used to solve certain problems. Given your new statements about Copilot being a "fancy pen" that can in fact be used incorrectly--something crazygringo seems to claim isn't possible--you frankly sound like you agree with my arguments!!

I think a crucial distinction to be made here, and with most 'AI' technologies (and I suspect this isn't news to many people here) is that – yes – they are building abstractions. They are not simply regurgitating. But – no – those abstractions are not identical (and very often not remotely similar) to human abstractions.

That's the very reason why AI technologies can be useful in augmenting human intelligence; they see problems in a different light, can find alternate solutions, and generally just don't think like we do. There are many paths to a correct result and they needn't be isomorphic. Think of how a mathematical theorem may be proved in multiple ways, but the core logical implication of the proof within the larger context is still the same.

Statistical modelling doesn't imply that GPT-3 is merely regurgitating. There are regularities among different examples, i.e. abstractions, that can be learned to improve its ability to predict novel inputs. There is certainly a question of how much Copilot is just reproducing input it has seen, but simply noting that its a statistical model doesn't prove the case that all it can do is regurgitate.

One way to look at these models is to say that they take raw input, convert it into a feature space, manipulate it, then output back as raw text. A nice example of this is neural style transfer, where the learnt features can distinguish content from style, so that the content can be remixed with a different style in feature space. I could certainly imagine evaluating the quality of those features on a scale spanning from rote-copying all the way up to human understanding, depending on the quality of the model.

Imagine for a second a model of the human brain that consists of three parts. 1) a vector of trillion inputs, 2) a black box, and 3) a vector of trillion outputs. At this level of abstraction, the human brain "pattern matches and replicates" just the same, except it is better at it.

Human brains are at least minimally recurrent, and are trained on data sets that are much wider and more complex than what we are handing GPT-3. I have done all of these standard though experiments and even developed and trained my own neural networks back before there were libraries that have allowed people to "dabble" in machine learning: if you consider the implications of humans being able to execute turing complete thoughts it should be come obvious that the human brain isn't merely doing pattern-anything... it sometimes does, but you can't just conflate them and then call it a day.

The human brain isn't Turing-complete as that would require infinite memory. I'm not saying that GPT-3 is even close, but it is in the same category. I tried playing chess against it. According to chess.com, move 10 was its first mistake, move 16 was its first blunder, and past move 20 it tried to make illegal moves. Try playing chess without a chessboard and not making an illegal move. It is difficult. Clearly it does understand chess enough not to make illegal moves as long as its working memory allows it to remember the game state.

>The human brain isn't Turing-complete as that would require infinite memory

A human brain with an unlimited supply of pencils and paper, then.

Hmm... but a finite state machine with an infinite tape is Turing complete too. If you're allowed to write symbols out and read them back in, you've invalidated the "proof" that humans aren't just doing pattern matching.

> The human brain isn't Turing-complete as that would require infinite memory.

This is wrong, this is not what Turing completeness is. It applies to computational models, not hardware.

https://en.wikipedia.org/wiki/Turing_completeness

How so? The page you link offers three definitions[1], and all of them require an infinite tape.

You could argue that a stack is missing in my simplified model of the human brain, which would be correct. I used the simple model in allusion to the Chinese room thought experiment which doesn't require anything more than a dictionary.

[1]: https://en.wikipedia.org/wiki/Turing_completeness#Formal_def...

Turing completeness applies to models of computation, not hardware. Otherwise, nothing would be Turing-complete because infinite memory doesn't exist in the real world. Just read the first sentence of what you linked to:

In computability theory, several closely related terms are used to describe the computational power of a computational system (such as an abstract machine or programming language)

Thank you for pointing that out, I was indeed wrong to assume it was used to classify hardware rather than a model.

Human thought isn't anything like GPT thought - humans can spend a variable amount of time thinking about what to learn from "training data" and can use explicit logic to reason about it. GPT is more like a form of lossy compression than that.

Reply on news.ycombinator.com