Hacker News Reader

Also, the limit of one agent's intelligence is kind of missing the point. Humans are powerful because of societies of people loosely working together. AI agents will be able to communicate and interact at speeds that are we cannot imagine. A network of near human IQ AI agents would be much more powerful than a single super smart AI agent.

I think that there is very compelling evidence (but not formal proof) that there is exponential falloff in terms of the ability of parallel networks of intelligent agents to have heightened overall intelligence.

Imagining that there will be near-human IQ AIs that think orders of magnitude more quickly than humans puts the cart before the horse, as well.

Would it not appear as though it was more intelligent simply based on the speed at which it makes decisions?

The Actual Intelligence may be the same (or a little higher) considering the Average Intelligence, but the effect the higher speed would likely be a measurable difference.

Yes, that's what I'm saying. If you can make an AI that's roughly human IQ (even like 80 IQ), but thinks 100x faster than a human, then that's something very much like, if not identical to, "a superhuman AI."

So when you say, "Here's how we'll get superhuman AI: We'll network together bunches of 80 IQ AI's that think 100x faster than a human," it's kind of assuming its own solution.

Why would you network them? Just add more ram.

Ask the guy who came up with this scenario. But, obviously, we do at some point find it easier to set up side-by-side computers with (relatively) high-latency connections (like, say, ethernet) than to try to put more computing resources together with low-latency connections (like motherboard busses), for physical reasons.

Reminds me of the story about the galaxy spanning AI in the Dune series. With a latency of 10000 light years between it's various instances.

How did they physically get the machines so far apart in the first place? No wormholes??

They had FTL capable ships. The AI used to sync updates using that.

Same reason why culture matters more than individual IQ. Most of human progress is cause because there are many people learn from each other and making millions of tiny improvements over time.

The 100x faster scenario is not really clear to me. Computers already "think" orders of magnitude faster for some things and orders of magnitude slower in other things. I am not sure this will be all that much more useful than the status quo. Unless they modeling a human brain than I see 100x faster being a useful thing to measure. Otherwise, I don't see speed increases being all that critical to making a superhuman AI.

I think networks of AI can become super powerful even without being nearly as smart as individual humans. AI will only need to approach certain aspects of humans IQ. It can still be much dumb than us in most ways. Most of human progress has been build up because of vague interactions and feedback loops via markets, religions, and nations.

You said that the AIs would be able to "communicate and interact" much faster than humans. If they can't think much faster than humans, then they can't communicate and interact (meaningfully) much faster than humans.

You can think much faster than you can express yourself. You can express yourself much faster than most people can understand you. You can only have a back and forth communication with a tiny number of people at once.

I don't think it will have heightened measurable intelligence. It will be dramatically more powerful, unpredictable and innovative.

will they?

Adding people to a late project makes it later.

But if the AI is fast enough, the project will be late much more quickly.

Perhaps more relevantly, I think there's a more basic questionable assumption - the idea that Church-Turing is the best computational basis for AI.

I strongly suspect this is not true - because even after all this time, our best abstract models of computation are still extremely brittle and fragile.

In a hand-wavey sense, as soon as you start allowing for arbitrary inputs and context awareness, you're no longer dealing with a simple Turing architecture but with something else entirely.

The TMs you hear about in your undergrad studies are not even good models for the webserver that served you this comment. Church-Turing is about computing functions over the natural numbers. You can (and usually do) have a Turing complete model of computation in which you can't program Tetris.

"Adding people to a late project makes it later."

Not all projects are beholden to the rules of the mythical man month.

And even for those that are the primary reason they are is basically the lack of a fast, efficient way to transfer built up institutional knowledge, which may be far less of a factor for AI collaboration.

If I've been working on a project for the past year as the sole developer and I could copy a snapshot of my brain to a portion of your brain that you can access without interrupting me, adding you on as a collaborator is very unlikely to make the project later and very likely to make it go much faster as you could very quickly 'hit the ground running'.

A lot of the problem with the current limits of human intelligence is really bandwidth problems with getting information transferred in and out of our meat brains effectively. Our bandwidth for knowledge import/export is DREADFUL.

AI collaboration (even assuming that bandwidth limitation goes away or is vastly reduced) would still introduce the issue of increased communication requirements, but compared to the initial transfer of acquired knowledge that one isn't as big a deal in my experience and only becomes problematic at great scale.

> I'm not a historian but I have read that some scientific discoveries are made through pure logic

What you are talking about is scientific hypothesis/theory which obviously needs to be tested using experiments to make sure it works. Without experiment there is nothing much you can do in real world.

Einstein's work was based on a lot of evidence that seemed very strange. Light somehow had a fixed speed in all reference frames WTF?

It's a problem perfectly suited to a logical AI, really. Take in the experimental observation data, and produce a solution that fits the parameters.

Human intuition was the major thing getting in the way, and even Einstein's intuition got in the way of his acceptance of quantum mechanics.

Sure. Question is: how many other discoveries await today such that we, collectively, as a species, already have all the puzzle pieces, but haven't yet put them together? An AI as smart as your average grad student, but which somehow could retain in mind at the same time all of our accumulated scientific knowledge, might be able to quickly draw all sorts of seemingly brilliant conclusions. Now imagine its reasoning process works 6 to 7 orders of magnitude faster than ours, even if not qualitatively different in its logic or biases.

Dunno, I don't really believe we are that close to building that sort of AI, but it doesn't seem fundamentally impossible, and it does seem like it could do things that to us would look as "more intelligent" than us. It might in the end be better at creating scientific knowledge in the way current computers are better than us at solving arithmetic: faster and capable of holding more in memory, rather than following any qualitatively different process. But even that would be enough.

IMO, few major ones. We have billions of man years of research and things are better than before that period, but diminishing returns are real.

Don't get me wrong I think it would be useful, just that the gap from human level AI to 1,000x human AI is simply not that huge. Let's say you moved someone from 2006 to 2016 or even from 1996. Yea sure there is real progress, but not really. We have better drugs in terms of AIDS for example, but we are worse off in terms of antibiotics. Game graphics have improved, but quake is from 1996 so we already had real 3D first person shooters and gameplay is not that different. Hell FTP is arguably worse. Further that's 20 years so we are talking literally millions-man years of effort and trillions of dollars worth of R&D for not much.

In terms of machines the SR-71 is still the fastest manned aircraft, no supersonic passenger aircraft. Tallest building is much taller, but lacks much space on the top floors making it more monument than utility as the sears tower has more useable space and a smaller footprint.

Invention proceeds because of game-changing insights.

Maxwell's equations were a game changer. So were Newton's laws. So were Relativity and QM.

Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since.

There's been a lot of application, but no game-changing deep theoretical insights.

Quantum computing may - should? - eventually throw up something new.

It's going to have to. If it doesn't I think we're going to be stuck with much less progress than we're expecting.

> Church-Turing was also a game changer. But I don't think there's been anything equivalent in computing since.

Quantity when the difference is large enough becomes quality. The 9 orders of magnitude or so that computers have gone through in storage capacity and speed definitely count as a game-changer.

The insight that all problems in NP are basically the same was a game changer in complexity theory (it basically started the whole field).

Except that a lot of our accumulated scientific knowledge is simply wrong due to experimental errors, fraud, selective publishing, and random false positives. See the whole reproducibility crisis.

It's conceivable that someone could construct a special-purpose AI which isn't superhuman but uses advanced ML and NLP algorithms to mine the scientific literature for previously unnoticed implications and relationships between seemingly unrelated articles. But I expect that would only give a suggested starting point for human researchers to formulate hypotheses and design experiments.

There's tangherlini relativity which doesn't make these assumptions and is consistent with existing observations! The math is crazy hard.

I agree that the fear of "super AI" is overblown, vastly in my opinion. Discussion on this topic is tantamount to arguing over the attributes of imaginary entities.

Simple artificial life of some kind seems much more likely in the foreseeable future. I've often wondered why no-one has written a virus or worm that is capable of mutation or even cross-over (sexual reproduction of a sort). Under selection pressure it might evolve into something resembling life. The Tierra experiments were a fascinating experiment in this area.

> The idea that humans could, just by chance, be pushing the limits of intelligence strikes me as silly

It's the wrong argument entirely. It has nothing to do with intelligence but everything to do with knowledge. Smarts are useless if you have no knowledge. And that's exactly where computers have the edge. Even a 'regular' human level intelligence with all the knowledge that currently exists at its beck and call would be a formidable adversary. Remember: Knowledge is power.

Without further experimentation the best one can say is that you have a theory which covers that which has already been observed. This is not a useful theory, however: useful theories make predictions about future observations. Really useful ones make correct predictions.

An AI can certainly produce a theory which matches observations, but no matter how powerful it is it won't be able to magically produce only correct useful theories any more than a human can.

We had over 200 years from Newton's laws of motion to produce relativity from pure logic alone, but we didn't manage it. We had to reason from the evidence, and even then we didn't accept relativity as "true" until it had been experimentally verified.

Relativity stems from the Michelson-Morley experiment.

The ideas of relativity predate Einstein - his particular view/model still has serious flaws and in the modern rendering leads to non-reality conclusions. In a very funny way, Einstein's two theories of relativity have become religious dogma. Try to discuss this and the usual conduct of "true-believers" is NOT to show where your thinking is wrong, but to simply call you an idiot.

This approach does nothing to convince anyone who believes differently that they are wrong, all it does do is convince them that the "true-believers" are themselves deluded.

This does nothing to advance the understanding of society of the universe around them.

I mostly agree with you, but what are the 'non-reality' conclusions?

Gravitational singularities. If one puts thought into it, one will recognise the shell game played when describing how something that requires infinite time to form as defined by the model, forms in finite time in the real universe. One then recognises that any two such entities will take infinite time to merge in the real finite timed universe.

I am also looking at a paper that purports to say that Einstein's GR which is used to calculate the precession of Mercury uses a couple of simplifications, which if not used, gives GR calculating the precession of Mercury at twice (or thereabouts what is measured). To test the veracity of the paper, I'll have to do some serious brushing up of my engineering mathematics.

Another example is related to the speed of light c. This value is inversely proportion to the square root of the (electric permittivity of the environment times the magnetic permeability of the environment). The normal assumption is that outside of the atmosphere of the earth, the environment is a perfect vacuum. It is assumed that the speed of light is therefore a constant in a perfect vacuum.

For many areas, this simplifying assumption is adequate, but this assumption may not be true. It is only so assumed. It may turn out that due to variations in both the permittivity and the permeability of the extraterrestrial environments, that this assumption is not inadequate but grossly inadequate.

When doing any experimental work, one should keep in mind and have written down all the base assumptions, otherwise, one may garden path oneself.

Modern cosmologists certainly don't assume that the heliosphere and interstellar medium are perfect vacuums.

That may be the case. However, they still assume c is a constant, in the same way as they also assume that gravity is the prevalent force even in areas that have high plasma reactions. The normal assumption (in all the papers I have seen to date by the "conventional" cosmologists) is that the universe is electrically neutral and hence the only significant force is gravity.

This has led to the idea that there has to be additional unseen, unmeasured matter (aka dark matter) and unseen and unmeasured energy (aka dark energy) to explain the anomalous measurements of galactic movement that is not explained by the "gravity is the only significant force" model.

Now, it may be that there is such a beasty. But, I have my serious doubts because electromagnetic fields have more intense effects than gravity, by some orders of magnitude (about 36 orders of magnitude).

In other words, some environments may be completely neutral and gravity is the major effect, in others, the environment may be very electrical and gravity is the least effect.

Our problem is that we can observe only at vast distances and are not able to take direct measurements to confirm our indirect modelling measurements.

Science is no different to any other aspect of people's lives, they will invest great amounts of energy in following their specific model of the world. When challenged, they mostly look at other as imbeciles who are incapable of seeing "the truth". I've seen this occurring in areas like string theory, black holes, dark matter, sub-atomic physics, climate science, immunology and vaccinations, medical surgical techniques, religion, politics, economics, sociology, finance and business, education, archaeology, evolution, etc., etc., etc.

Our models and theories are just that models and theories. We hope and believe they give us insight into the universe around us, but they are not reality. Each has limits and all use simplifying assumptions that are not accurate outside of the limits imposed by those assumptions.

If on the basis of our assumptions, we have to keep adding more and more "strange" entities to make our model fit the observations, then, maybe, we should go back and look at the underpinnings of our models and see if another model will work better (as in the kind of change that eventually occurred when going from the earth-centric epicyclic model to the sun-centred heliocentric model).

The fun is being able to learn more and more about how our universe operates and what we can do within it.

What are your issues with relativity you allude to, I wonder?

> Try to discuss this and the usual conduct of "true-believers" is NOT to show where your thinking is wrong, but to simply call you an idiot.

They say "give me evidence", because so far they have an enormous mountain of evidence that SR and GR describe reality well.

That doesn't excuse people being jerks -- science should be welcoming, not alienating -- but it does mean "well, have you considered X" is almost always coming from a place of ignorance, not well informed skepticism. Some impatience with the cranks of the world has to be excused.

A tiger is dangerous even if its not selfaware.

A mathematical theory (á la Einstein) without experimental confirmation is just a theory. Or perhaps economics.

And the idea that all forms of computation, like Turing machines, lambda calculus, etc., etc., also seems pretty darn silly at first glance, too.