raydiatian

Emergency emergent emergence

I agree that emergent behaviors are real, and important.

I am skeptical, although not completely unconvinced, that LLMs like GPT are going to produce truly emergent phenomena, such as true first-principles logical reasoning. The limitations of the underlying transformer architecture itself are, in my opinion, the problem. The first problem is that the embedding space of the transformer needs to grow much, much larger, and it's already huge. This matters because you need to model the order of neurons in the brain. The second problem is that you're never going to train an LLM (as they're designed today) that is going to produce a truly good 'emergent-phenomena' answer without multiple network traversals. This is because the human mind constantly and autonomously refines its thoughts.

Perhaps a good counter-argument is that emergent phenomena are fundamentally a space-time domain concept.

I am aware that things like Conway's Game of Life are a fantastic counterargument to my "the transformer architecture doesn't support it" argument. But I agree that the definition of "emergent behavior" when it comes to machine learning is too easily corrupted to be novel rather than rigorous.

“Emergent anything” is probably the most obnoxious buzzword in all of machine learning.

I dropped acid one time and turned a coffee grinder and an arduino into a system that opens my front door. I showed it to my friends but they laughed at me. But it worked, so I don’t give a shit.

Stir my f*ing soul!

Cus Windows fuckin sucks from the subcutaneous layer down to it’s bone marrow.

Click

Bait

The problem is that at the inter-governmental scale of civilization, it’s essentially Stone Age tribalism. Survival of the fittest. As such, countries like the US and China are incentivized to disregard AI ethics in the interest of self preservation.

Mankind wins reality if we can install an AI as a one world order system with lasting world peace. It genuinely doesn’t matter if it is a communist or capitalist invented system, it just needs to out-function our current paradigm of warring to maintain status quo.

Mankind loses reality if we build genocidal AI death systems.

Ted Lasso! Relevance!

Knees be jerkin'

I mean that's what happens when you signal for two decades that web is the highest paying salary for the lowest upfront investment. Furthermore, the ability for companies to do their compute in the cloud just means more HTTP traffic.

> Instead, Olshausen and others argue that information in the brain is represented by the activity of numerous neurons. So the perception of a purple Volkswagen is not encoded as a single neuron’s actions, but as those of thousands of neurons. The same set of neurons, firing differently, could represent an entirely different concept (a pink Cadillac, perhaps).

Unless they’re talking about embeddings, this is how 99% of people think about representation.

And even if they are talking about embeddings, embeddings are, quit standardly, higher dimensional floating point vectors. Even the original Transformer paper used IIRC 512-dim vectors.

This article seems like yet another signal for the death of quantamagazine.

Isn’t that artificial super intelligence?

> It's not cheating for people so asserting that it's cheating for machines just seems like goal post shifting more than anything.

I genuinely appreciate this argument, and was considering it myself. In which case, I’d almost argue that we “have” already achieved AGI, and maybe it’s just not that thrilling.

What you’re saying gets to the core of why I would call it AGK and not AGI. Training a transformer on known answers to problems and then observing that it can successfully answer questions related to those problems is cheating.

I think the way that Ilya suggests that the “test for consciousness is to train a model with an absolute absence of any training example remotely referring to the notion of a self or of feeling, and then ask it questions about feeling. If the model can do it, congrats, you’ve discovered consciousness.” Similarly, if you train an architecture on exclusively the building blocks of a particular class of problem, and also avoid training it on any sort of problem where it could just reason by analogy and get a correct answer (isolating first principles thing as the only option), then if it can solve the problem you have a genuine problem solving architecture.

Right but if it needs its hand held, that ends up being a transcription task rather than a logical reasoning task. Like if you _tell it_ the solution to a coding job in detail, it can build you the complex entity you’re looking for. But if you just say for instance “write me a Python script that generates random chord changes (ex A#dim to Gmaj9b5)”, first of all it will just dump code without asking for clarification on requirements, and second of all even if you do give it further clarification on requirements the code won’t work without you explaining in depth the algorithm.

Although, that’s just a personal anecdote.

I have been thinking for a few weeks now that we need another term for large language models trained on colossal datasets: AGK, artificially generally/globally knowledgeable. It can mimic a likeness of problem solving because the corpus it was trained on is full of problem/solution pairs in the abstract. But task it with any novel problem solving challenge outside of its training that is of sufficient complexity and it will balk, thereby precluding it from being AGI, because humans are by nature problem solvers.

Furthermore, I just don’t feel like the transformer architecture is suited for problem solving. Like I may just be a charlatan but self attention over the space of words does not seem like it’s going to be enough, and praying it falls out in emergent behavior if we can just add more parameters is… unscientific-ish? Now, if you could figure out a way to do self-attention over the space of concepts? Maybe you’ve got something.

I feel like AlphaGo ideas and some variation on MCTS is more likely to produce a solid problem solving architecture.

Well, I mean it’s pretty fair accusation. ChatGPT was demonstrably bad at math. I think it was only recently mentioned that GPT4 was trained on math. Furthermore, consider what it means to apply the transformer architecture to math problems. I think the tool is a mismatch for the problem. You’re relying on self attention and emergent phenomena to fake computational reduction as symbol transformations. It can probably do some basic maths (all the way up to calculus even) because, in the scope of human life, the maths we deal with are pretty boring. But that’s what they made the wolfram plug-in for as well.

I really think people attribute powers beyond what GPT really is: a colossal lookup table with great key aliasing.

Unfortunately, I don't think Marvel is capable of that level of depth.

Nice, that's probably closer then. I just plugged in the mass of the milky way for the mass.

For those wondering, the radius of this black hole (assuming a Milkyway’s worth of mass), based on the Schwarzschild radius, would be anywhere from 10-1000x larger than Sagittarius A, the supermassive blackhole at the center of the Milky Way. I don’t think the article stated this.

I think AGI needs to be compared against the species in the aggregate. The vast majority of humans also cannot practice law or pass LSAT/GMAT, but some of us can. If we’re setting the bar for AGI as “anything you can do I can do better”, AGI simply would not meet the bar.

To pull that thread further, I think a core feature of intelligence is problem solving. Problem solving is the derivative that creates difficult to obtain knowledge. GPT-4 might look like a problem solver, but it isn’t actively solving new problems with its outputs, it’s sharing existing knowledge of prior problems solved.

The test for problem solving ability is similar to Ilya’s test for consciousness: subtract everything but the fundamental building blocks of a certain problem from the agent’s training set. Finish training, see if the agent can solve the problem from the building blocks. If it can, and it’s not a reason-by-analogy but a logical first principles answer, congratulations, you’ve created artificial general intelligence. If it operates on fast enough time scales, it should probably even be categorized as super intelligence, because it might be singularity:35pm on a saturday.

Artificially Globally Knowledgeable*

GPT-4 is not quite AGI. Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. It is, to perhaps coin a term, AGK, artificially generally knowledgeable. As a language model trained on an absolutely colossal dataset, it’s basically just a giant snapshot of human knowledge taken in superposition. Sure it’s probably at least 90% of the standard knowledge, but intelligence is a different thing.

I also think agency is wrapped up in AGI. Intentions & thoughts are meaningless until acted upon. Agency is not all or nothing either; Stephen Hawking had multiple augmentations, community and technological, which allowed him to continue to impact the world of physics After he lost his god given agency.

> GPT-4 has nearly aced both the LSAT and the MCAT. It’s a coding companion, an emotional companion, and to many, a friend. Yet it wasn’t programmed to be a test taker or a copywriter or a programmer. It was just programmed to be a stochastic parrot.

I disagree, it was absolutely trained to be a test taker. It’s been a second since I read the original GPT paper but there’s literally a multiple choice auxiliary learning task, where they use a separator token-embed to organize "question, context, options a, b, and c". As far as being a friend to many, is there evidence of this? I tried to talk to ChatGPT about some emotional problems to see if it was a cheap therapist, and I got flagged.

This.

Will try this, thank you!

This looks rad.

nodejs alpine

> It seems these godlike AIs are already frighteningly cheap and easy to replicate.

Who writes this shit?

Meanwhile, taking LSD hints at LSD being definitely hallucinogenic and holy f** why is your face wiggling