Hacker News Reader

> Every time a major advance is made in AI the computational neuroscientist say: "why didn't you talk to us 15 years ago? We could have told you that!".

I'm not sure which advances you are talking about, but the modern successes of Deep Learning are primarily due to backpropagation and convolution. Neither of the two is considered to be biologically plausible.

Some people are actively trying to come up with alternatives to backpropagation that would be biologically plausible though.

See for example: Bengio et al., "Towards Biologically Plausible Deep Learning" https://arxiv.org/abs/1502.04156

Actually convnets were inspired by Fukushima's Neocognitron, which was itself inspired by visual cortex.

> Actually convnets were inspired by Fukushima's Neocognitron, which was itself inspired by visual cortex.

That doesn't contradict what I wrote.

ConvNets require the synchronization of weights between neurons, which is not considered to be biologically plausible. Some aspects of the architecture (the receptive fields, in this case) may well be plausible, with the complete architecture still implausible.

You're focusing too much on the implementation details and assuming that because they're different that it's not equivalent. The secret sauce here is that network topology (and threshold rules), not the implementation details, are largely what determine the functional properties of that network. Show an electrical engineer the circuit diagram of a 4 bit adder and they'll know it's function immediately. Artificial Neural Networks and Artificial Gene Regulatory Networks are the same. The problem with ANNs is that we fail to see the circuitry driving the function because we assume that every w_ij != 0 is functionally relevant. Once you start to strip away non-spurious interactions you start to see topological patterns. I did a lot of work on this problem -- this is not an uneducated guess.

I didn't see a maths major in your background but I'm surprised you seem to dance around how important back-propagation is, and how different it is from what you could find in nature.

The major advances are the implementation details, and I think many would consider the network topology research constrained not by our imagination but by our implementations.

There are some similarities, but ConvNets really remind me of LeCun's airplane vs bird analogy. There are similarities such as learning very similar intermediate representations (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5288363/), but they are also very clearly not doing the exact same thing that a human is. It's a technological emulation, and that's good enough for me.

> Artificial Gene Regulatory Networks

I've been following AI for years but this is the first time I found such a concept. So basically a cell is like a small neural net with as many neurons as genes, each gene having (chemical) input and outputs signals. That means a cell's DNA is much more dynamic than I previously imagined. It's a self-replicating m.f. computer, that's what it is. We can only dream of similar accomplishments.

Previously I was aware of the huge workload carried out by DNA - for every protein in the body, DNA replicates the blueprints - an amazing amount of fine detailed work. It's not just sitting there waiting for reproduction. Seeing it not just as a factory, but also as a neural net is another level.

>Every time a major advance is made in AI the computational neuroscientist say: "why didn't you talk to us 15 years ago? We could have told you that!"

Can you name one time that has happened? I can't think of any.

Hindsight is 20/20 there.

Bayesian brain theories of perception. The neuroscientists and cognitive scientists are still waiting for the AI theorists to stop obsessing over that one paper with the cat's visual cortex and get off the deep learning train of sacrificing correctness for cheapness to compute.

this is not a case that happened

As a matter of fact, the mainstream AI community hasn't adopted probabilistic programming, but it does exist.

Feel free to link to any Kaggle blog posts showing victories using probabilistic programming.

Is your opinion independent of evidence?

you re going way off topic . this is about AI discoveries which in retrospect were found to have been discovered by neuroscience. you 're complaining that a potential theory (which is only partially supported by evidence) might one day hypothetically be found useful in AI.

Background: Some guy on the Internet with an unrelated Bachelors degree he didn't study that hard for and has picked up a little Python and Javascript somewhere along the way.

I know of no biological forms that have evolved wheels. But wheels have turned out to be a hell of a lot faster than fins or legs. I see no reason "intelligence" has to follow the pathways or limitations of biology or neurology at all. Although certainly it may be a place to look for some ideas.

Fair point, but you are omitting the fact that brain itself (that invented the wheel) was evolved through biological means.

That said, evolutionary optimization is generally considered unreliable(convergence) and inefficient.

The wheel existed before the mind did. It was just sitting there waiting to be noticed :)

Not for the function the brain made a use of it.

Wheels are useless if you don't have roads.

EDIT: Please explain your downvotes here? Don't you people see that this, coupled with there being no useful intermediate stage before developing a full wheel, prevents evolution from "inventing" it.

Well that's not true at all.

See: Wheelbarrow; amongst many wheeled devices I can think of immediately.

Wheelbarrow creatures? I don't see how that would be advantageous at all.

Also, try running around with a wheelbarrow in a dense forest.

Wheelbarrows are a prime counterexample - historically, in environments that had decent man-made trails, wheelbarrows were useful and used; in environments that did not, pack-carrying animals or even people did the job even though the concept of wheelbarrows was known; also, the availability of roads (as opposed to narrow trails) was the prime reason for differences of usage of wheelbarrows (especially the efficient middle-wheel variety e.g. https://www.google.lv/search?q=wheelbarrow+chinese&tbm=isch ) versus two or four wheel animal (or man) pulled carts, which were popular only where roads had been made and were maintainable, so not in jungle-like environments where it takes more work to keep a road from overgrowing.

I would also point to heavier-than-air flight. For thousands of years people tried to mimic flapping wings of birds, but in the end it was the unnatural propellor and fixed wing design that gave us flight.

Fixed-wing flight isn't unknown to the natural world.[0]

Flagella use propeller motion for propulsion.[1]

Lots of animals use jets.[2]

I'd consider the combustion engine to be the most novel 'human' design with regards to aircraft propulsion -- or the way we achieve jet propulsion; that's pretty unique.

[0]: https://en.wikipedia.org/wiki/List_of_soaring_birds

[1]: https://en.wikipedia.org/wiki/Flagellum

[2]: https://en.wikipedia.org/wiki/Jet_propulsion

... and a new energy source, the combustion engine.

> I know of no biological forms that have evolved wheels. But wheels have turned out to be a hell of a lot faster than fins or legs.

On certain terrain. And much slower - sometimes useless - on others. Wheels aren't simply better legs, they're a different way of moving, with its own benefits and drawbacks.

Likewise, if we're talking about computational ability, we already have things that are faster than the human brain in certain areas (say, numerical computation). When we talk about AGI, we're talking about human-like intelligence (at least to a certain degree).

So we already have the wheel, but now we want to develop something that does what the leg does. Now, maybe that doesn't have to be an actual leg, but that's usually what people usually end up with when they try to find something that will have a similar functionality as a leg.

Not a wheel but a rotary motor: topologically the same. https://en.wikipedia.org/wiki/Flagellum

The wheel, by itself, is not faster than legs. It does not adapt, repair, or feel. You will always need more than your legs to create a system that properly utilizes wheels.

There is nothing intelligent about artificial intelligence. You might as well call it artificial stupidity, but that doesn't make you sound sophisticated.

Memorizing the results of a pre-defined concept faster due to a lower barrier of entry does not make you smarter or more intelligent. You can process all of the traffic signals in the world in a blink of an eye only to get stomped by a dancing traffic director.

You can't have AI without a human, and you can't have AI with a human.

Apparently philosophers have pondered on the matter, it's called the multiple realizability problem.

>I know of no biological forms that have evolved wheels.

Silly to reduce all rotary motion to wheels.

Less complex to roll as a ball -- the benefit of rotary motion without facilitating the additional complexity of adding a steering mechanism.

> But wheels have turned out to be a hell of a lot faster than fins or legs.

Nature doesn't somehow cares about hyper-optimizing for a single characteristic like 'speed', rather the characteristics arise from individual (species) successes between genetics lines.

https://en.wikipedia.org/wiki/Rotating_locomotion_in_living_...

https://en.wikipedia.org/wiki/Category:Rolling_animals

It's not about building better tools (i.e. the "artificial" wheels vs. nature's wings and legs), it's about building a thing that would be just like us, humans: intelligent and especially a thing that has feelings.

I know that the "feelings" part is most of the times neglected in these types of presentations (you cannot put them into any abstract representation, like you can draw a wheel using geometry), but it's what most of us, people who are not in this field, expect from AIG. Great Science Fiction works (novels, movies) have talked about this subject (AI and human feelings) long enough, but it somehow never makes it into academic presentations.

Just think for a second of the self-driving cars problem. Someone on HN (or maybe reddit) explained a couple of days ago that even if we 'd manage to devise such a system that would be considerably better compared to humans we'd still not be ok with these machines killing us on the roads. Now, imagine these machines possessing the feeling of "guilt", or of "compassion", I'm pretty sure that the percentage of humans that would become ok with some AIG driving our cars for us will rise dramatically (I know I would stand on the AIG's side). We don't mind the fact that machines kill us, accidents happen, we mind that machines kill us and they don't "realize" what they've done and they don't support the consequences.

Re-reading my comment I feel like it sounds a little pop-sciency, didn't mean it to go into that direction, it's just my interpretation on how I think things are. I for myself would love, really love, if we'd somehow "make" an AIG capable of telling genuine, funny jokes.

You can't really evolve a wheel in the sense that a half-evolved foot is useful already, whereas a half-evolved wheel: not really. Therefore obtaining a functional wheel has to be done in one jump, which is contrary to how the gradual process of evolution works.

Wheels are way more basic. Anything will skid and slide on a spread of rubble, doesn't even need to be animated.

If you consider rotary motion as wheels, there are some bacteria that have flagella that rotate to propel itself, complete with molecular bearings.

That said, biological evolution tends to get stuck on local maxima very easily (see convergent evolution of eyeballs), and wheels are kind of hard to evolve because of the difficulty of making large bearings biologically.

Not to mention wheels kind of suck unless you're on pavement (look at tanks, etc)

> neuroscientist say: "why didn't you talk to us 15 years ago? We could have told you that!".

Interested to hear about these. I can honestly not remember any.

There's been this tension over whether achieving apparent human levels of artificial intelligence requires thinking like and following processes associated with human learning and intelligence pretty much forever. Today, computational capability and data have gotten us to a place where there's been a big step forward while largely ignoring how it relates to human-level thought. (After all, we got powered flight by largely ignoring how birds fly.)

In fact, cognitive science largely split off as a separate field but there's a school of thought that algorithms and big data are only going to take things to a certain point and you're not going to get to things like fully autonomous vehicles under general conditions without different approaches that involve better understanding the physical world.

Actually, previous to the Wright brothers, the dominant theories of flight involved studying how birds flew and theorizing what parts were important and trying to replicate them. You could get a degree in this area of study.

I do agree that different approaches are needed.

In the case of flight it was the realization that the airfoil is important and the rest is window dressing.

Am I right in thinking the study of fish/sharks and fluid dynamics helped with the breakthrough?

No, it was bicycles that guided the intuition that control should be based on roll rather than rudder yaw. That was the key innovation that the Wrights contributed at the time of their first flight.

Not sure why the downvote, but this is what I was thinking about:

https://en.m.wikipedia.org/wiki/Fluid_dynamics

Nothing to do with origins of flight but interesting and related to why flight works.

Actually before the Wright Brothers came a guy called Otto Lilienthal who figured that out.

https://en.wikipedia.org/wiki/Otto_Lilienthal

From what I've read, the Wright brother's main innovation was that 3-axis control was needed to maintain flight, which they also copied from birds.

Which means that copying birds is fine, you just have to know what parts to copy (mechanisms to control direction in all 3 axes) and which to ignore (flapping wings isn't practical given the power sources and construction materials we have, but soaring on stationary wings is a good place to start).

Which in turn means that saying "Just copy nature, they've been doing this for 1bn years" or "don't bother copying nature, we need different stuff" both aren't very useful.

>Which means that copying birds is fine, you just have to know what parts to copy

The analogy between acheiving AGI and achieving flight is invalid on its face and yields no useful information. Nothing can be drawn from the latter about the former.

Flight is mechanical. AGI is not. However, until now, ML and other AI techniques have focused largely on mechanics (stuffing data into an algorithm), which is why such analogies with nature are tempting. But, that approach won't suffice for AGI.

The breakthrough for AGI will come through an approach that backs away from brute force and attacks the problem from a meta-perspective; building on the core blocks of what intelligence is. The approach will be relatively simple, once the core foundation of the integrative learning process is understood, which is the much more difficult task. Execution will then involve establishing a base case that implements, not learning, but the capacity to learn. This will then be replicated across generations, compounding exponentially until what we recognize as AGI emerges.

As well, the required computing power is likely to be significantly less than is commonly believed. This raw brute-force ML-style approach again misleads us here.

Why are planes called wings? And while the propeller or turbine is largely missing from birds, it contains many wings as well. I'm not convinced.

Don't know why you feel so compelled to give your background while writing an anonymous comment on the internet. It only makes me suspect your argument doesn't hold up without some expectation to respect your authority. All the theory in the world is irrelevant until it you can apply it to predict something new -- its almost trivial to create a theory that describes some phenomenon accurately, after-the-fact. See the tremendous bifurcation in physics with regards to string theory. Similarly there are loads of academics in various fields claiming to have been the origins of artificial intelligence.

Hand-engineering and hand-assembling evolved components will fail, way too complex. We struggle with making bug-free word processors. This will only be solved by fully evolving AGI, or some other form of automated search of high-dimensional design space.

I'd love to read your thesis. Would you please a link?

What do you think the best way of attacking these problems is? It seems to me that we aren't even at the point where technology can do much to help us (in terms of recreating what we know about the brain) and we just need to understand the brain better.

"Just" that. Maybe.

The problem is that, at the risk of trivializing a lot of good research, science related to the brain/how we think/etc. has seen a whole lot more money spent and the effort of smart people expended than it has seen useful results. There are still debates going on about a lot of aspects of learning and other aspects of cognitive science that were probably already happening when computers were made from vacuum tubes or even earlier.

Can you link to some of your work, this is very interesting stuff

Hmm, reading your experience clarifies to me (a Maths guy) why current deep learning efforts are producing real fruit from their labor. Linear Regression and Vector/dot products are just constructs to re-create the evolutionary biology seen in nature. When looking at the slow process of evolution, it's the nuances that matter and over time, they compound exponentially. Pretty neat. By breaking down a task into abstract constructs and sifting problems out with a sigmoid function.

I don't think you're wrong that AGI will have (something like) those components in its inception.

I just disagree that we have to intentionally create them -- I think there are a lot of places we can experimentally bruteforce the implementation instead of having a solid theoretical understanding if our only goal is creating an AGI.

I don't think we need any theoretical breakthroughs, per se, but rather, a lot of computing power and time. No one seems willing to take a million nodes and run them for a decade to brute force some of the mechanisms -- everyone wants results on the short timelines that grants or Wall St operate on. I get why, but that bias towards quantifiable short-term gains fundamentally limits the search algorithms we're implementing in our quest for AI, and likely means that we won't get there in a meaningful way, because intentionally implementing the necessary requirements is a fool's errand of complexity.

In short: we're not smart enough to do the simple, but slow thing, so we're trying the highly complex one with demonstrable incremental results. I expect this approach to continuously fail to develop AGI even while it demonstrates results on discrete problems.

I think you are right that we could, in theory experimentally brute force intelligence. But if we try and do that, we are basically trying to simulate an entire universe and wait for intelligence. But the amount of computing power that that would require is beyond even what we can imagine. And even if we could find some ways of simplifying it, it still seems like a waste of our intelligence.

Like when it got cold for humans, rather than just dying and waiting until they evolved fur, they just made clothes. Evolution is inherently able to do great things but is very inefficient. We have the power to reverse engineer nature and use that knowledge to our advantage, and if our goal is to create general AI, then not trying to understand the one sitting in everyone heads and just trying to throw more computing behind it just seems like a waste of a tool that has allowed us to get this far.

I'll preface this by saying that my views don't necessarily represent a mainstream position here:

Humans already accidentally invented AI at least a few times, sort of. To talk meaningfully about this topic, I think it's better to step away from the topic of "intelligence" and instead speak about "reasoning ability". It's still fuzzy, but seems to sidestep a lot of the "can submarines swim" arguments.

Things like societies, governments, corporations, economies, etc all show similar features to machine learning algorithms and reasoning animals in their ability to dynamically problem solve -- the main difference is that they run on human computers instead of electronic computers. (Or photonic, quantum, etc computers.) Well, in modern times, they're actually sort of cyborgs, but that's a side issue.

The quest for an AGI is more about translating the notion of a corporation (or government, etc) meaningfully into fully electronic computer terms than it is in replicating the human mind. We know enough about the supra-structure of the corporation to meaningfully bound the search, and it's much more prudent to spend a decade on bruteforcing each component of a corporation than it is to navel gaze about reinventing minds. The truth is that human brains (or animal brains) aren't the only general intelligence that we know -- so reinventing them isn't the only path to AGI, though it's a worthwhile project in its own right.

I just worry that many of the meaningful contributions of the people working on reinventing minds will be lost on the birth of the actual first AGI through corporate cyberization. (And that many of us will fail to recognize corporate cyberization for what it is -- an AGI. Any (successful) corporation which can act fully by algorithm must be a general intelligence.)

1. What are the other general intelligence that we know of besides life?

2. How is it that you know that the reasoning abilities of the structures that we create are because of the structure, and not because we are the pieces that make up the larger structure?

1. Sorry if I was unclear -- the examples I listed were meant to be examples of "general intelligences", but I think corporations are a good example.

2. Well, this is somewhat more complicated to answer --

Obviously the reasoning abilities are (partly) because we're part of the structure. But in a way that we know can be decoupled from the reasoning ability of the structure -- see modern CS on which problems are tractable to mechanize and the success of ML at individual tasks -- because modern corporate theory has formulated a system of corporate governance and execution that only depends on individual people for one or a few tasks. The fixation of modern corporate management on replaceable, cheap, low-skill workers makes it a perfect template to replace components one at a time with bruteforce discovered ML components (and in fact, we already see this happening -- because it is part of the reason that corporations are designed that way).

After all, from the perspective of the corporation as a reasoning entity, increased cyberization is increased efficiency, which ultimately means increased competitiveness (and hence survival).

Electronic computers were designed to replace human computers in the military, and we have a wealth of information on their numeric and string processing capabilities. They're specifically designed to emulate in another medium some of humans' reasoning abilities, particularly those around logical deduction and numeric calculation (which in some senses are similar). Since modern corporations have reduced much of their operation to machines (eg, factories), string manipulation and calculation, it seems perfectly conceivable to have a corporation control all of its "core functions" via electronic computer after a period of increasing cyberization, with contracted delivery services and on-site technicians viewed something like a doctor.

Corporate cyberization is, if you take my first claim that corporations represent a "general" reasoning structure of made of specialized components as true, an open and direct path to an AGI that we seem to be stumbling down.

Some very good points.

I mean, what is the problem we are trying to solve here? Fully emulate a human? That's worthless, we have an overabundance of humans already and it's easy to make more.

Or are we trying to come up with something that performs our tasks without us having to think or do any work and presumably makes life richer and more pleasant? Well that's a different problem, and yes, we have been automating that forever and are arguably getting better at it. (I say arguably because "richer and more pleasant" are relative as it turns out).

The argument over what constitutes "intelligence" is not an insignificant one either. My dog likely thinks I'm stupid because I don't eat butter all day and shave and go to work. And come to think of it, he might be right. Point being, we often conceive of intelligence as "having better ways to do or get what we would". But then there is abstraction, longer term planning, higher order thinking, which in the end almost certainly will be incomprehensible to humans.

>The quest for an AGI is more about translating the notion of a corporation (or government, etc) meaningfully into fully electronic computer terms than it is in replicating the human mind.

That's an astoundingly terrible idea, considering what powerful corporations and governments have done to the world.

I didn't mean it as a normative statement, just one about the technology required -- I share you concerns.

Perhaps an important warning that AI could go the same thing...