Ah, but the trick is, as soon as you can get a machine intelligence smart enough to work on programming tasks, you can get it to help you design a better one. And so on, etc.
The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on.
There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever.
A computer with five times the mass, five times the volume, and five times the power requirements of the human brain, that can run an uploaded copy of a human brain at merely half the speed of a human brain, would still render Pluto and Mercury trivial to colonise with machine intelligences.
And if it cost a $100k (inflation adjusted) to build, and lasted 80 years before you had to melt down the hardware and reforge from scratch, it would still be cheaper than Musk's target for a human to Mars.
Mere COVID disrupted that here; imagine what a few hundred million miles of space would do.
Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well.
https://www.researchgate.net/publication/322275580_Electroly...
> Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well.
If we really needed those things (why would we even want to use wood as a structural element on a space habitat?), we can build bioreactors from metal, glass, and water.
Oil and gas do occur non-biologically e.g. Titan, but they can also be made from algae grown in a transparent tube exposed to sunlight and provided with the necessary minerals and CO2, which is trivial to make.
IIRC the two limited resources if you needed biology and couldn't do it all with a clanking replicator are nitrogen and phosphate, everything else is easy to find basically everywhere.
Note that I am extremely skeptical of your 1T parameters claim - there is much much more to AGI than natural-looking natural language processing + image recognition.
Natural selection is
- random
- blind - gradient descent is prone to getting stuck in local minima, has no foresight, no ability to go back to the drawing board, no "understanding" of what it's doing.
- not even optimizing for intelligence, except instrumentally - to the extent that increasing intelligence interferes with survival, it has to be sacrificed
- subject to hard constraints like the limits on size imposed by childbirth
The synapse is an incredibly complicated structure and there is a lot of 'computation' occurring within the butons. We're not very sure of all the processes that occur at this time. It's not just a 0/1 kinda thing. Also, our current computers aren't even running the hardware that a synapse is. The most analogous electrical structure to a synapse is a memristor, something we can't make at scale right now. The synapse is also not the only structure that causes computation to occur, many things effect the firing of a neuron and that modulation.
Great lower bound calculation though. I'd say that's the right ballpark number.
Another is that Neurons are incredibly slow. Lets say you can make a machine only half as intelligent as a Human but that can think 1 Million times faster, this is even a lowball since a neuron can fire at like 1Hz and modern CPUs operate in the GHz range.
>In all likelihood we are never going to create a server with a whole city of intelligences in it, ever.
Think this is a really bad take. If one assumes even a .1% improvement in "Machine Intelligence" per unit time then it is literally only a matter of time until you reach Human level and then pass it.
That's like saying "even if there's only a 0.1% improvement in materials strength per unit time eventually we'll have structures strong enough to bounce off an incoming rogue planet." There is an upper bound to computational performance per unit mass and unit energy. We're nowhere near it but it is there, lurking.
This comparison doesn't make much sense - computers have a very small number of cores, let's say 10, and brains have 86 billion neurons. 86 billion things operating at 1Hz is also in the GHz range. This is leaving aside the issue that a CPU cycle and a neuron firing are doing completely different sorts of work - comparing them is kind of nonsensical in the first place.
>This is leaving aside the issue that a CPU cycle and a neuron firing are doing completely different sorts of work - comparing them is kind of nonsensical in the first place.
Maybe, but the op comment was explicitly using raw transistor count to compare to Neuron count as a proxy for sapience.
you jump in the air at high noon. the sun is (some) distance away, and for some time, you're getting closer to it.
if we make an ai that's able to make a better ai than we are, then it does stand to reason that it'll kick off exponential growth for some time, but it'll reach limits just like anything. those limits may fall short of magicing/smarting away the obstacles to colonizing space.
and we'll reach limits too. it might be that the amount of time between now and when we make a sufficient ai is more time than we have.
A human jumps, fails, learns the underlying physics of what’s holding them back, and consequently build rockets to sit in. There are no hard limits yet, as far as we know, and we’re just bags of mostly water.
Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them.
Computers might be a better analogy. We started designing computers without the aid of other computers. We made very rapid progress on architecture and semiconductors. We’re now completely dependent on computers to help us design future ones, and despite all our efforts and hundreds of billions of dollars in investments, our rate of progress has slowed to a crawl and we seem to rapidly be approaching the end game. Dennard scaling has been done for 15 years. We can make wires that are smaller but just proportionally worse. We can use more silicon area, but then latency and power consumption get proportionally worse. A super clever hyper optimizing AI might push a couple things forward a little bit (or more likely wind up at the exact same destination slightly earlier than we other wise might have), but there’s no compelling reason to think it gets to design computers that face different constraints.
Can you share your crystal ball? I need to check next week lottery numbers
They may, or they may not, which is why I would not opine on the subject. I was only addressing one narrow part of the comment I replied to.
We have no idea what that limit is so it seems crazy to make any assumptions about where it is. At minimum it lies above Human Intelligence and even imagining what a civilization could do with millions of beings only as intelligent as say John von Neumann is frightening.
>those limits may fall short of magicing/smarting away the obstacles to colonizing space.
We can basically already colonize our Solar System with current tech. You don't need FTL or anything close to that to eventually colonize the entire Galaxy.
Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.
I'm not sure what's difficult about this. Are we not able to test what humans can run the fastest without being able to run just as fast? We can equally test which humans are the best spellers, or the best chess players, or the best problem solvers.
Once we move to more general forms of intelligence, like math and other forms of problem solving, then you apply more general tests.
For instance, if we had an Einstein AI v1.0 that could infer General Relativity using X0 bits of information in time T0, and Einstein AI v2.0 managed to infer GR using X1 << X0 bits of information and/or could infer it in time T1 << T0, then v2.0 is clearly superior to v1.0.
I agree that sometimes the parameters for such tests aren't always clear at first, but this has always been the case in science, which is we we iterate and progressively refine our tests as we learn. Any true AGI must be capable of such abductive and inductive reasoning to qualify as a general intelligence.
The amount of brain matter does not say all that much about the level of intelligence. Some kinds of birds are as intelligent as 7 year old children, and that's not something you'd guess from the size of their brains.
Why do people keep acting as if this was a god given truth and just a matter of time ?
It's making the assumption that human brains are simply very complex neural nets. Even ignoring religious/spiritual arguments about souls, the simple fact of the matter is that we don't really know a lot about how the human brain/consciousness actually works on a fundamental level (and anyone who tells you otherwise is misinformed or lying).
How human brains work is irrelevant, we didn't need to solve the Navier Stokes Equation to build Airplanes and we don't need to solve intelligence either. Consciousness is totally outside the bounds of this discussion and also irrelevant to building an intelligent AI.
If they gave a specific time range, it would be.
Plenty of people have done so.
At what point do you think computer technology will reach a theoretical limit and cease to progress?
Indeed, we have AGI already, they are called humans, and the progress we see in the development of AI is not exponentially explosive. The only exponential growth we've seen (which is necessarily logarithmic growth after some inflection point, actually) is Moore's Law.
Actually it is. Human population growth has been exponential, as has been our resource use and consumption of our habitat, and the pace of our innovations. We've now created forms of machine intelligence to further augment our own, and machine intelligence could itself start advancing soon, without the messy constraints of evolution and biology. The possibility is definitely there, we'll see if it works out that way in the end. If it does, it will likely blindside us.