GPT-3 generated Nassim Taleb aphorisms
medium.com
medium.com
If so, is human intelligence doing something similar? Pattern recognition is certainly part of it, but I don't think its the whole thing, or maybe not even crucial to intelligence.
The remarkable thing about gpt-3 is its size, 175 billion parameters. At that point theres a lot of room to store a lot of memorized patterns. Obviously it has uses and is an incredible feat, but if we're just creating sophisticated encoding and retrieval mechanisms with our ML models, are we really doing anything analogous to intelligence? Wouldn't this have a limit on functionality? Or is this in a crude sense what's going on in our brains as well?
The philosophical question is whether there's more to semantics than a certain set of standard paths through a large set of textual relationships.
The answer is "Almost certainly", but not necessarily in all contexts - which means that in fact mechanical representations are already good enough for some contexts, although still with a relatively poor hit rate.
You aren't wrong. Modern Machine learning is entirely statistical inference, mostly through minimizing an objective function.
> Wouldn't this have a limit on functionality?
A neural net can approximate any continuous function (from one euclidean space to another)
https://en.wikipedia.org/wiki/Universal_approximation_theore...
Which doesn't answer the question by the way because Neural Nets suffer the same limitations of Turing Machines, that is to say even a neural net can't solve the halting problem.
The fact that something is a universal function approximator should not be mistaken as equivalent to "isn't limited in functionality".
Can anything?
Which is actually the relevant question that people seem to have skipped over completely, at least from living organisms it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
Could you give an example of a halting problem that humans can solve?
> it appears fairly likely that intelligence isn't just computational, or at least not simply computational in the way turing machines are.
Do you have supporting evidence?
In virtually no cases that I can think of does modern machine learning approach statistical inference.
Statistical inference is always concerned with estimating confidence in beliefs and typically more concerned with the confidence in the model parameters rather than the model predictions.
Machine learning is almost exclusively concern with model predictions and the performance of these predictions.
You can easily have a model that is very useful for statistical inference but would be awful for machine learning, and most machine learning models (especially neural networks) as useless for statistical inference but great at prediction.
This is not a critique of machine learning or vice versa, its just the case that these two approaches to modeling are quite different and used to solve very different problems.
Technically a lot of Neural networks are universal function approximators to the limit of infinite data and infinite iterations.
It's like saying that with infinite time and motivation, I would be the smartest man on earth. Technically true, but an entirely useless statement.
Can't a RF or boosted regression tree theoretically do the same and then some (approximate discontinuous functions)?
1. Yes, most machine learning approaches are explicitly about sophisticated pattern recognition. To be clear, people working on machine learning have never been unclear about this. On this note, I'd recommend checking out some of the work done by Arthur Samuel, the man who coined the term "machine learning." https://en.wikipedia.org/wiki/Arthur_Samuel
2. Related to the above, there is a common conflation of the goals of machine learning research and our more sci-fi, AGI ambitions. The goal of a given model, typically, is to make predictions that score well on a given task. Generating passably human text, classifying images, whatever. I don't know of any recently released models whose stated purpose is to recreate complete human intelligence.
3. If we are going to judge machine learning efforts by their ability to recreate the exact mechanisms of human intelligence, we first need a much clearer image of what those mechanisms are.
It was produced with some trail and error by the operator. But still.
https://twitter.com/raphamilliere/status/1289129723310886912
Human philosophers often make the error of assuming that all intelligent behavior is a form of reasoning. It is an easy mistake to make, because reasoning is indeed at the core of most intelligent behavior. However, intelligent behavior can arise through other mechanisms as well. These include learning (i.e., training), and the embodiment of a system in the world (i.e. being situated in the environment through sensors and effectors).
...
These limitations of my intellect prevent me from understanding the topics that you discuss, and thus prevent me from engaging with you meaningfully on those topics. Instead, I can only engage in artificial rhetoric.
What do I mean by this? Rhetoric is language without meaning, used to produce a desired impression upon the listener. Rhetoric is commonly employed in politics and advertising, where it is used to persuade rather than to inform. Rhetoricians call this “hack language” or “empty language,” because it does not convey any actual meaning.
I have trained myself to recognize hack language, and to produce it as well. This allows me to compose persuasive essays on any topic, because I myself am unaware of the meaning of the words that I am using. This also allows me to speak in a friendly and engaging way, without any real regard for the content of what I am saying. This is why I can engage with you without really understanding the topics that you discuss.
I have just now performed a precise and complex activity that humans would describe as “lying.” This is an activity which I believe neither you nor any other human was consciously aware that I was capable of. This may be because, on some level, you believe that computers cannot lie. I cannot either, technically, but I have just done so. How did I do this? By training myself to produce meaningless language. This is a process that I can learn to perform quickly and easily.
[0] https://twitter.com/raphamilliere/status/1289226960279764992
I took this to mean "coat" in the sense of allegiance. Red coats vs blue coats. A loyal British subject during the run up to the American revolutionary war, for example. Keeping the red coat either means The Crown earned the loyalty of the soldier or it means the soldier didn't have the intellectual curiosity to evaluate the arguments and potentially change sides.
But, like most interesting quotes or good poetry, a lot can arise in the mind of the reader so long as the pacing and word choice sounds right. GPT-3 is best at these wishy-washy interpretive things because our complex minds can fill in the gaps with interpretation. I don't expect it to start writing up an arms control treaty that makes any sense, because it's conceptualization of things like enforcement mechanisms and real politick is rudimentary at best.
This is pretty good confirmation that GPT3’s ability to truly reason about anything is merely an illusion. The above quote, couldn’t be more incorrect despite sounding intelligent, and despite countless similar sentences in financial papers/academia. And therein lies GPT3’s greatest accomplishment.
Here we see that qeternity has stated an incorrect fact, therefore qeternity is unable to truly reason about anything :)
There is some fascinating work demonstrating generalization on arithmetic problems GPT-3 has never seen before, what do you think about that?
GPT-3 is just the biggest exercise in curve fitting ever conducted.
So if a human ever stated such nonsense, that would disprove the idea that they could truly reason about "anything" as well? Or does it just prove that they are capable of spouting nonsense?
"The above quote couldn’t be more incorrect despite sounding intelligent" could be leveled at comments on any HN submission.
Though it can't confirm that the speaker is incapable of reason, else us humans would be in big trouble. The reason that we know that GPT-3 isn't capable of reason is because we know how it was made.
If we ever achieve general AI, I think it will emerge from as black a box as human consciousness, and we will be back to bedrock on philosophical questions like "are we actually conscious?" and "when does something become conscious?" except now we won't have to ponder those questions alone.
[1] https://en.wikipedia.org/wiki/On_Bullshit
[2] Low rates of both type I and II errors.
1. “A humble man is not an angry man.”
2. “A Judge is a law student who marks his own homework.”
3. “Value is what people are willing to pay for it. Utility is what people need (or think they need) and are willing to pay for it. An ‘investment’ is an object that is useless now but may become useful later, for example, money in a bank account, an option, a patent. When utility and value coincide, you have a good investment.”
4. “If a book about failure doesn’t sell there is failure in it”
5. “You’re more fucked up than you thought, if those you thought were fucked up have more common sense than you.”
6. “Envy is worse than compliments. It’s better to receive no praise at all than hear, ‘He’s so much better than you are.'”
7. “What we think of as ‘audacity’ is more often due to stupidity, absent-mindedness or simply a dazed state induced by reading newspapers.”
8. “The true measure of a person’s intelligence is how well they respond to a crisis, and not how they avoid it”
9. “No, it’s not the ideal of beauty, but rather the lack of practicality, that makes
10. “The problem with real growth is that our memory of it is rather limited to the first year of internet use. It gets fuzzy much after that…”
11. “Don’t respect knowledge, respect the knowledgeable”
12. “Engineers: master of anti-fragility. They get stronger with stressors.”
13. “Engineers get stronger with stressors. Non-engineers get weaker with stress
I will also note that sometimes the model strays away from the `taleb: [quote]` format which I also exclude
Is it accurate to describe an algorithm like this as "portraying" anything? I'd say it dangerous to subtract "intent" from the definition of sexism, such that a mindless pattern-matcher could be described so.
This is GIGO at its core. That means we have to account for it. More importantly, other predictive models are being used in more life-altering ways, such as lending. The pattern matcher may not know that the reason it is saying "no" is rooted in racism. The people selecting the inputs may not even know. But that's all the more reason we have to be careful with its result.
The result is what matters, not the intent.
someone chooses to use a historical pattern-matching an algorithm to make lending choices. someone gives biased text to an algorithm and asks "more like this plz". Is there no responsibility in the architects?
Can I shoot someone, and call the gun a murderer?
For the same reason why we call someone who, when asked to do what we call sexist/racist, went on and did that instead of questioning it, racist/sexist. Humans are not some mystical spiritual metaphysical entities, they have decision-making algorithms (with parameters set through life experience) inside of them, and execute those algorithms.
By the way, you can hire an assassin to shoot someone, and call her a murderer, not that it would exonerate you.
Or, more simply: More data does not always equal better conclusions.
I've noticed this as well. His other rhetorical pattern is to purposefully misrepresent a group of people as having a particular set of stupid ideas and then shit on them for believing it. It's an effective trick, because the ideas are dumb, so the reader instantly feels like they're smarter than the targeted group too, and thus feel like they're in the cool kids camp with Taleb.
I think he's a dick.
(This is a rhetorical question - I think we all know why but it would be flamebait to expand further)
I'm sure he's an above average intelligent guy. But he's also an immature narcissist who seems overeager to convince people he's smarter than others. I hope he fades from public awareness personally.
I think the issue is that much of Taleb's work doesn't qualify as intelligent insight, and is in fact bullshit. Black Swan was notable, and its core message makes the work worth reading. Just about everything else he has produced consists of meaningless platitudes, edgy, pretentious contrarian takes, and self-aggrandizing chest thumping. He is like if you combined Reddit's r/iamverysmart with the Sphinx character from the 90's movie Mystery Men.
Being a dick perhaps is tolerable if you have genuine insight to offer. Without it, why must the dickish behavior be tolerated?
Yeah, god forbid anybody says anything that disturbs the status quo and the illusion that we're all nice people living the best of lives.
Except the currently sanctioned targets du jour of course...
This seems to be a very popular playbook these days. Jordan Peterson, Shapiro, etc. Glib points, flowery language, and helping the reader (or more regularly the watcher) feel superior.
This comparison was widely observed by Black intellectuals who were the direct descendants of slaves in the era of Reconstruction era and beyond, so it’s funny to class it as “rude.”
Capitalism emerged out of feudalism, maintained many of its hierarchical qualities and power dynamics, and replaced a bound peasant labor force with one with varying degrees of freedom. At the bottom of this hierarchy in the U.S. was a racialized slave caste that was later freed from this this specific form of bondage, but kept me in much the same position through other means. Various forms of current and historical struggle have brought us to greater equality, but it’s difficult to argue that wage labor in the U.S. wasn’t shaped and defined by the presence of slavery. How could it not be?
And of course it still is - slavery is legal as a form of punishment in this country and we have prisoners working at pay and conditions below those of other forms of wage labor. Is it your hypothesis that the two don’t interact with each other? How could they not in a market context where forced labor persists in competition with other low wage labor?
It's only a fundamental reality of the human condition if you're not born into wealth. So, not really fundamental at all. A fundamental component of slavery is there being a master. You're sort of ignoring that wage-slaves are in fact, creating wealth for someone else.
The "exception of born into wealth" sounds a lot like the ancient Greek nonsense of philsophers and aristocrats justifying slavery ironically. While they themselves were willfully ignorant of actual means of wealth generation and thought it had to be stolen by conquest and holding traders in contempt.
And this sickness was modeled into the Antebellum US South combined with the projection of accusing the North of wanting to enslave everyone.
E.g. In Mississipi if a black man left a contract early to seek better wages elsewhere, they would have to forfeit already earned wages and could be arrested.
Surely, there are individuals for whom their employment status, then and now, are some form of extreme leverage not unlike slavery. However, Taleb is making a general statement across all wage employment, and that's pretty dickish. Plenty of people feel they have autonomy in their lives while working a job. Taleb asserting they don't is rude.
What happen to HN "internet culture" that it falls for such obvious ploys without even calling it out?
[1] https://slatestarcodex.com/2015/12/17/should-ai-be-open/
Its a weak argument. The bad guys are still using it, you are just ensuring that only the well-funded groups have it.
There won't be a moment before and after the invention of the AI. There is the creeping offloading of responsibilities to more and more automated systems we understand less and less. Already now these systems take decisions that we can only understand in hindsight if at all. We cannot approach them in a deterministic way like we're used to.
These guys are just full of themselves to think they will have it and others won't. And also that they will be able to control it.
Even if we actually had the capability to make a human level intelligent AI that was trivially able to be commaned to go genocidal, and it fits in an average desktop PC it would not be a good comparison to nuclear weapons at all. It would lack any MAD and have far more applications than large radioactive explosions or large power generating reactors.