Complex systems emerge from simple rules
tecnica.substack.com
tecnica.substack.com
* Hidden Order by John Holland. Especially if you are technical, then read it from the perspective of "how would I build one of these"? (See RIP).
* Signs Of Life: How Complexity Pervades Biology. An overwhelming book. At the very least you should come out aware of how little we understand.
* What is money? Really. We all know the mythical person month perspective, but according to that New York City can not exist.
* The RIP routing protocol.
* A New Kind of Science.
Gestalts, created from complex adaptive systems, surround us. (Inside joke). I think this is an example of the old adage "I won't see it until I believe it" which is just a simplified version of Sapir Whorf.
[edit: add Wolfram's "A New Kind of Science" ]
As it happens, I don't believe this argument if taken all the way. Sure, emergence has the property that, in some sense, one knows more about the emergent system if it is described at its own level rather than at the level of the underlying system (eg, if you tell me you are hungry I'm much more likely to correctly predict you will order pizza than if you give me a catalog of the state of all your neurons), but your behavior still supervenes upon the neurons inexorably. Even the constraints that maintain the emergent system ultimately are expressed in terms of the underlying system.
Emergent is that produced/provided (as a potential) of a system which is greater than the sum of its parts.
Absolutely everything new you can do by arranging a few independent things for an outcome which could not have been achieved otherwise (literally creating potential in the universe) is "emergent."
So driving great distance and speed in metal chariots emerges from our technology for cars.
Flight emerges by airflow physics and the trimtab.
Life emerges by eating and crapping dense energy, propagating the continuity of existential being.
As soon as some perfection is found, it crumbles and reconstitutes in the minds of those who express and those who perceive.
Seeing "potential" as the underlying and the "things" (which we value) as meta symbolic liberates my definition of a dozen contingencies.
There is a great mystery, yet this is a very useful definition.
Emergent properties are those potential created by a system of which such potential does not exist in any part otherwise.
A product emerges from an assembly line.
All that other stuff you said must be scrapped off one meaning, and further distilled toward whichever mystery you actually want to solve.
... which may be consciousness or meaning or edge cognitive technology.
On the other hand: For quark systems, the total mass seems completely unrelated to the mass of the constituents. A pion is 2 quarks, 139 MeV. A proton has three, same type of quarks. Mass 938 MeV. The mass is emergent, from the interactions of the quarks and gluons. It's not predictable easily from the constituents.
Nevertheless, we can put QCD on the lattice and brute force it.
It is not clear to me if this is fundamental. I.e. if there are systems where you can't predict the behavior short of a full simulation. I suspect that there is a fundamental statement about nature here, similar to computability arguments. But I also think there are still a lot of techniques to develop to get closer.
Emergence in some esotericly romantic specialized science means "the symbol derived." In existential Reality without confusing theoretical limits emergence is any new potential created by a system, whether planned that way or a surprise to you.
Granted, surprises are also "emergent."
> The mass is emergent, from the interactions of the quarks and gluons
I argue the mass emerges as a standing wave against the mass relationship with all other bound potential. A hyperdimensional shape [knot] causes constructive/destructive interference to whatever potential is bound (here miniscule yet accumulative.)
The spin characteristic of this hyperdimensional knot of essentially light. Will be such that it will have a discrete mass (as full spins are at the speed of light, half spins are dimensional vectors causing fundamental force interference.) this relationship between the speed of light and half spin vectors may account for mass as the particle comes to rest. So a light speed particle is slowed down to mass rest by spin disposition (which was formed in the particle collider of infinite space and time.)
A small bias in a standing wave has pseudo-deterministic influence in everything it interferes with.
All of existential reality emerges by constructive and destructive interference of otherwise uncommitted particles.
Additionally, abiogenesis is well accepted in academia as a prime example of emergence.
If you use "emergence" to mean "complicated stuff in a simple system" then I guess it is emergence, but that definition is philosophically useless or at the very least uncontroversial. It's trivially clear simple rules give rise to complex behaviors at this point. The question is the causal role of the roughly conserved elements of the complex system in question.
If life evolved here on Earth, then somehow, CHON+trace all self-organized into all of Earth life today, including us, and all we humans have done and will do in the future.
Now: say we could go back just before life evolved. Even with very very good data, and with whatever talent (AI, science, anything) and technology from today, would we be able to truly describe the emergence paths for those CHON+trace atoms?
Impossible. This would be far beyond our level of science and technology. We can't even do this for much simpler systems and shorter time horizons. You'd have to predict the emergence of life, the full properties and behavior of all forms of life in the last 4 billion years, and last but not least, humans and all that humans have done and thought since then.
Yet clearly nature emerged all this from perhaps small amounts of 11 elements or so. Complexity is one of the greatest unknowns in our present civilization.
So it doesn't matter if the ChatGPT is designed by simple rules, the minute we try to control the randomness, it becomes complex.
LLMs aren’t simple systems either, they are density estimators pretrained on terabytes of text, then fine tuned with human feedback. The simple fact that the author believes this suggests he doesn’t know much about LLMs.