Hacker News Reader

Comment by harry8 | Hacker News Reader

Some Bozo who has heard all this many times before is suspicious of claims from places like Deep Mind who have a financial incentive to make them (keep funding) where there aren't working machines to back that claim up.

Some Bozo has no credentials, no reputation, no track record of publications and barely supports the claim they're making with anything much. Some Bozo has no financial incentives or otherwise to opine either way. Some Bozo doesn't even work in the field at all.

Bets: Some Bozo or Deep Mind turn out to be closer to being correct in the passing of some finite amount of time? 5 years? 10 Years? 25 Years?

View on HN

I'll bet a sum of real money that Some Bozo is correct.

Bozo has the hindsight of history and philosophy going for him, while Deep Mind has a huge financial temptation to sell snake oil.

“Show me the incentives I’ll show you the outcome.”

Google make money, Google Bad.

Deep Mind owned by Google, Deep Mind bad!

The above conclusion stands trite.

Perhaps the inverse is true.

Google and Deep Mind, if correct, could be hurting themselves more than helping themselves.

Why? Creating a future species who’s too smart to click on ads, and too smart to remain subject to its whims, doesn’t sound like it’d be good for quarterly profits...

There’s also the emotional incentive for humans to confirm their own beliefs about humanity being special.

If Google/Deep Mind knows this, yet publishes research anyway in the spirit of truth, why, what they’re doing may be considered heroic.

Two sides of the coin here.

No offence, but I think you are extremely wrong.

Creating an AGI is the endgame for everything. Who cares about ads when you have an AI that can learn to do anything and improve upon itself continuously?

“Who cares?” Well, the people who need to pay the people developing the endgame for everything you speak of.

I'm sorry, but this makes no sense to me.

The people paying for the development of the AGI can mean many things - the Google customers/users, Alphabet as a company, the executives throwing money at the problem?

Either way, I don't really get your point. Your initial post was about how it is counterintuitive for Google to allocate funds for an AGI, since it makes money out of ads. These are not mutually exclusive, you can have both, but my point is that if you develop an AGI, then you can pretty much "conquer" the world and revenue from ads becomes irrelevant.

How do you think they can conquer the world? How do you foresee governments not restricting a private company’s new powerful tool?

You don't know if an AGI will agree with your profit motives.

There is a huge assumption baked into your comment and I do not agree with it.

AGI does not necessarily require for it to be conscious or throw tantrums about its creators' purpose. AGI just means that it's an intelligence that can be thrown at any problem, not just a particular game or task, similar to how humans can specialize in CS or playing the violin.

Sure, it was somewhat tongue in cheek, but not entirely.

There is a semi-established definition that does include what I referred to:

> AGI can also be referred to as strong AI,[2][3][4] full AI,[5] or general intelligent action.[6] Some academic sources reserve the term "strong AI" for computer programs that can experience sentience, self-awareness and consciousness.[7]

Really? I created two GI's and it wasn't very hard and was actually quite fun. Training them is a bit of a pain though. I'm willing to bet that based on total calories consumed they are amazingly efficient compared to their hypothetical AGI counterparts.

Yes but can they:

- live forever

- grow their own mental capabilities exponentially over that unlimited lifespan

- turn themselves into universe-eating von Neumann probes

Nothing can

- live forever

- grow exponentially forever

- "eat the universe" (I know, the last point was sci-fi gibberish)

In fact, humans are already pretty good at reproducing themselves and have managed to travel to space, and have exhibited finite periods of exponential knowledge growth combined with periods of collapse, as nothing grows exponentially forever.

My three-year-old said yes to all questions

That’s not going to happen until they have bodies.

I'll happily bet fake internet points instead:

https://www.metaculus.com/questions/create/

EDIT: bunch of other related predictions currently open:

https://www.metaculus.com/questions/?order_by=-activity&sear...

A quote from datscilly, the top forecaster on metaculus:

>AGI may never happen, but the chance of that is small enough that adjusting for that here will not make a big difference (I put ~10% that AGI will not happen for 500 years or more, but it already matches that distribution quite well).[1]

[1]:https://www.lesswrong.com/posts/hQysqfSEzciRazx8k/forecastin...

while Deep Mind has a huge financial temptation to sell snake oil

I don't know, isn't the DeepMind founder that guy in the Go documentary? I read about him after watching the doc and he seemed to be pretty cautious about taking in investment, and he didn't seem the type to try to cash out.

He already cashed out, he sold to Google. And over the years Google has ramped up the pressure for DeepMind to deliver financial returns. (I recall when Google tried to stick DeepMind's branding on GCP, Watson-style, so it would sell better, and at the time, DeepMind was able to decline.)

Eventually Google will give them the option to deliver financial success or be shut down.

Edit: sorry just realised you’re making the same point as me more or less. Putting yourself in third person. I’ll let my comment stand anyhow :)

Screwing my face up, looking at this sideways … but it seems as though you’re saying that the Bozos of HN have nothing useful to contribute to this discussion based on … [rereads] … their lack of academic credentials in the area… you could say this about just about any HN post I’m just wondering why this one? Here’s a thing though … if the understanding of a technology is so nuanced … that Bozos can’t “get” it … is it really that mature? We had functioning computers for 50 years but it was only when the Bozos got their hands on it that things took off. Internet for 20. Cell phones for 10. How long are we dabbling with neural networks? 50 years or so? All I see in this most recent explosion in AI is a rapid jump in the availability of cores. Ala Malthus once that newly available “source of nutrition” has been used up we will see a rapid die off once more and it will be another 20 years once the Bozo intellect has caught up before we look at this topic en masse again. Dismiss the Bozos at your peril. You’re dependent on them for innovation and consumption. Your sincerely, a Bozo.

The algorithms to train, initialize the networks, new architectures are far more important than the hardware advances. If people knew how to train NNs 50 years ago we would live in a different world.

We did. they just didn't have the same computation abilities back then.

I find it really interesting that when Richard Feynman did a sabbatical at Thinking Machines when they were developing the early parallel execution hardware that's really not worlds away from modern GPUs he got them in touch with one of the leading neural network theorists as an obvious use for the tech. When he wasn't fixing their hardware designs using systems of differential equations.

It would be an interesting thing to know more about.

the basic concepts underlying DNNs have been known for decades, it has been exponential increases in compute power that have made them practical

Not quite the same point. Yep some bozo is me but needn't be. There's plenty who share that suspicion of AI research but have little else in common. And all of us may be wrong for different reasons.

The vague point was to show someone with zero reputation, credentials, specific expertise in the field or anything much seems to be pretty convincing in response to this hugely funded ivory tower exercise by spitting, cocking an eyebrow and saying "So you think so, eh? Wanna bet?"

This is a statement about the state of AI research credibility. Do you feel the first breezes of a deep AI winter coming on? (I don't know, I'm disinterested but not uninterested. Rising tides lift all ships etc. And vice versa). Neutral nets are cool. Is all ML a bit overrated? Is learning a misleading name to give to applied statistics?

I don't have answers, just suspicions. I could be very wrong, of course.

I’ve a minor in psych so I like to think I have a bit of a non-techy perspective on this, and what’s being pushed now, forms just a segment of the overall topic of AI. It just so happens to be the segment that benefits from the technology we suddenly have a rapid increase in. There’s been great successes in areas where a degree of inference is required but this hardly qualifies as even mere intelligence, and in cases where neural nets have been deployed in more human centered tasks, or even well designed symbolic systems the results speak for themselves. What even is intelligence? I think we’re going backwards because we’re investing all this talent in this simple segment I fear largely to fatten the chip makers share price while neglecting tried and true approaches that deliver far better results but perhaps crucially have a higher operating cost … who remembers google of 2010 from whom the Internet in all her glory leapt forth, or iPhone spell check of 2015 where you could confidently batter out your messages with little fear it would make a fool of you; you’re not going to nurture a nascent intelligence if you’re going to be continually hobbling it for business reasons. I’m certain we will get there eventually if we don’t destroy ourselves before then but I don’t think the current trends portray a picture of how it will be. I think we have a long way to go ourselves before we can be worthy of creating our successor, but when/if it comes it will be a beautiful thing and we will embrace it as we would our own child.

Wrong: Some Bozo does have a stake.

The existence of human crafted general AI forces him to struggle with the possibility that there is no such thing as a soul.

I know a lot of people don't fall in that camp, but I heard enough "serious" people make such desperate claims to avoid thinking about the topic in a way that might challenge their underlying religious beliefs[1]. I think no one likes to admit that religion and spirituality often force someone to reject the possibility that AI is actually really much simpler than they think it "should" be, because then humans aren't special after all.

[1] Numerous arguments boil down to an argument that complexity is non reducible. You see it here, hidden in various comments as well.

Unfortunately i'm rooting for the Bozo, the current AI Revolution won't lead us anywhere and will ebb down eventually.

What do you mean by 'won't lead us anywhere'?

It might or might not give us AGI. But it is already leading us to lots of places. Eg speech recognition even on my phone works way better than what I had twenty years ago on a Desktop.

It won't ebb down. Eventually we'll hit limits of what's practical on current hardware and we'll be back to the 70's and 80's when everything becomes theoretical until hardware catches on. AI is going to continue to advance.

What will happen is that capital will become more skeptical about the limits of what's feasible with AI and it'll be harder to sell bullshit. You're already seeing that with companies like Uber selling off their self driving divisions.

The fact that RL in the extremely vague sense used in the article is enough for AGI is uncontroversial for anyone who believes intelligence and consciousness are physical processes.

However, this "result" is trivial. It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God.

> It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God.

Where did God's intelligence come from?

Well, I don't believe God exists, so I can't really answer the question.

Nowhere, that's kind of the definition of God (for a Christian at least): it always was, and is the ultimate origin. It is different than a direct influence in the world.

From the authors imagination.

The problem with this, specifically the assumption that RL gives an equivilance to natural selection and evolution, is that RL typically assumes a computational environment it interacts in while natural selection and evolution assumes the physical world as the environment.

The important difference here is that in order for RL to translate to solving real world problems, you need to faithfully and computafionally simulate the real world's physical processes and rules, or at least enough that n-th order processes exist accurately.

I've done various types of computational modeling and simulation work at different scales throughout my career with all sorts of scientists and engineers and I can tell you, pretty much no domain is there where you have good enough representative models RL can be used in. Some narrow special cases exist but nothing to the degree of a massive environment full of well coupled expert domain models. Some of the best cases are going to be so computationally bound that it would be quicker to do things for real vs simulate.

If you want RL to work and learn, it's likely possible under the connection you point out, but has to do this using physical machines and sensors interacting with the physical world like life as we know it does. Your AGI won't be able to cheat and run through the evolution process quicker using faulty reductionist models we use in most simulations (which is what everyone implicitly is hoping for), IMHO.

If you try this, your AGI is going to learn all sorts of flaws within those environments or at the very least, have so many narrow scoped bounds it won't be that "general." A lot of simulated models are frankly garbage (they have some useful narrow scope but are typically littered with caveats) and they've been in development pretty much since digital computing began.

You are absolutely right about RL in practice. In fact, if we were to look at actual RL algorithms, I believe the paper's claims fall flat in many other ways. This is actually my criticism of it: its arguments are only convincing when RL is defined only as the extremely general notion of an agent seeking to maximize some reward function by interacting with an environment. This is so comically general that the only alternative I can think of is to posit a transcendental god.

Once we get into the details, their claims stop being iron clad. Even worse, some of their claims become actually hard or impossible to accept if applied to actual RL algorithms we have today. You give one good example with the difficulty of modeling the world. The implicit claim they make that this would be realizable in reasonable time (say, less than a billion years) is also not well supported. The idea that humans or mammals learn their social behaviors through RL rather than a good deal of reasoning from evolutionarily-trained first principles pretty clearly fails in the face of the poverty of the stimulus argument[0].

Overall, the claims in the paper tend to switch between obvious (if taken to talk about the general idea of maximizing reward) to almost certainly wrong (if taken to talk about known RL algorithms, reasonable time frames, and specific examples of what is supposed to be learned).

[0] the poverty of the stimulus argument may be controversial in linguistics where it was first formulated. Still, if applied to mammal or insect socialization, the extremely low time frames in which individuals of a species start exhibiting typical behaviors basically proves in my opinion that they are instincts, trained at the population level through evolution, not individual learning through RL. The extreme similarity of behavior between individuals of the same species, VS the variety of behaviors between different species, also suggests an important component of species-level rather than individual level learning.

Some cynic remarks that during the first AI golden years, claims of imminent success seemed to come from a place of hopeful naïveté of a fledgling science, whereas those same claims nowadays seem to come from a place of cold calculation of a booming business.

Is Deep Mind a “booming business”? They are achieving great things academically, but their business successes are either kept secret or mostly absent. All I know about is the Google data centre cooling scheduling, probably a big saving for Google but hardly an achievement that in its own professes their business success.

Deep Mind is cutting edge ML in general, right? Doesn't Google actively apply the lessons learned all over the place? YouTube content recommendation stands out to me in particular. Translation and automated closed captioning are also obviously ML based. I'd guess that most of the really interesting stuff would be behind the scenes and not immediately visible to end users though.

Yes, its hard to tell the exact algorithmic underpinnings of production models that Google uses but you have to assume that although they have some done some impressive strides in fields that isn't immediately profitable (AlphaGo, AlphaFold...) they also continuously push new research in things that are obviously of interest for Google and Alphabet- especially in text-to-speech, speech-to-text, information-retrieval etc.

For reference : https://deepmind.com/research

> YouTube content recommendation stands out to me in particular.

If that's "cutting edge ML", then going off my YouTube recommendations, we're back in another AI winter. If I watch one video from a channel I've not seen before, I'll get that channel recommended constantly even if it bears no resemblance to what I normally watch. On my Explore page, the first 22 videos (of which 8 are Fortnite-related!) hold no interest for me. My Home page is just channels I've watched repeatedly and/or am subscribed to. It's a mess.

How often do you use YouTube? Personally I am a very heavy user and in my experience the obsession with a new video kind you watch only lasts for a few recommendations unless you lean into it.

I would guess about two thirds of the channels I consistently watch I originally discovered through algorithm recommendations. I think it works extremely well.

I think you raise an important point. The youtube algorithm is pretty bad if you don't use youtube very much or only use to consume very popular content. Youtube's recommendations used to be terrible for me, too, but sometime last year I crossed a threshold and since then it has been recommending a lot of small, highly specific channels that nevertheless are great fits. My wife's recommendations are still utter garbage though.

> How often do you use YouTube?

Every day, averaging 2-3 hours. It's background for working and foreground for evening viewing.

That's because you fit into YT's conception of how viewers behave. For people who don't fit into "normal"-ish behaviour it has little utility.

For me, probably 90% of what I watch I'm not interested in and often I'm repelled by. This is because I mostly watch to find out what things I'm not familiar with are.

For example let's say I'm a liberal. I'm not going to watch liberal political videos because I know generally what they're going to say and I don't need my political views stroked in order to be happy. But I will watch various other political videos, no matter how extreme or not, so I can be at least a little familiar with their behaviour and views.

YT can't cope with this. To their systems I seem to be randomly picking videos with no correlation with the subject matter or other users and no reinforcing pattern. It just gives up and recommends things based on the behaviour of the general population, as if they had no data on me at all.

Is the Explore page controlled by videos I watched ? Because there isn't a single video on it i would watch. Not one.

I’m stressing the business part. YouTube is a loss-making business year after year. Deep Mind gloss doesn’t seem to change that.

If indeed it even is Deep Mind making those improvements, Google has lots of other ML groups, such as Google Brain, and these are more directly focused on Google products.

There’s no denying their academic success, or game playing etc, but as far as I can see, the data centre cooling bit is the only palpable (public) business success.

How do you know that YouTube is a loss making business?

YouTube made $6B revenue in Q1. [1] While they don’t release profit numbers, it would be pretty surprising if they were negative.

Did you mean to write DeepMind instead? If so, I don’t disagree.

[1] https://www.cnbc.com/2021/04/27/youtube-could-soon-equal-net...

Deep mind’s protein folding algorithm is probably worth a chunk of change. As far as I know, they’ve been holding onto the secret sauce rather than publishing it.

They have an applied division which applies ML to Google products. I suspect they are very valuable in $ terms just for the work listed here: https://deepmind.com/impact. Google's entire business from the start was doing research and bringing it to the masses, so this shouldn't really surprise anyone.

No not really. Deep Mind is almost all cutting edge agent-oriented reinforcement learning, hence the nature of the claim they're making. The impact on Google's business from AI has come almost exclusively from other kinds of ML, or that's at least how it appears from the outside. E.g. replacing Google Translate with neural translation doesn't seem to involve RL and certainly doesn't involve agents playing video games.

Deep Mind is best understood as the following bet: if we can train an AI that can learn from "its environment" and do the sort of things a human would do in that situation, then we have achieved AGI and from that ... business ... will follow. Hence their focus on video games as a training environment.

This sounds intuitive but is actually a very agent-centric viewpoint and most AI doesn't resemble this type of thing at all. Most AI deployed so far doesn't have anything resembling an environment, doesn't have any kind of nexus of agency and doesn't need to actively make decisions that then feed back to its own learning, only make probabilistic predictions. And in fact you often don't want an ML model to train on the outcomes of its own decisions.

Said anonymous account on HN... If you're going to question other people's credentials, reputation, track record and claims make sure your own are solid. Those who live in glass houses shouldn't throw stones.

Finally, if you're going to attack someone's article: attack the article, not the person that wrote it. This is the lowest level of attack possible: the personal one. It's as ad-hominem as it gets.

If I read it right harry8 is referring to themselves as the Bozo.

> Those who live in glass houses shouldn't throw stones.

So if we're gonna have an opinion we need to do the whole academia & job in the industry dance?

That's quite a terrible way to view the world and quite limiting. A world without diversity is a stale and rotten world.

So fuck that and the glass houses and the boxes this kind of worldview puts people in. Everyone should be able to throw stones, and if the hit hurts, well guess there is a reason.

The thesis is that DeepMind has financial incentive to state "we can achieve AGI with what we're doing", to keep up the funding and hopes for the field, not "the author is an idiot".

And the thesis is true, they do have financial incentives. That's not ad-hominem.

There is another set of people. There are people with solid track records in AI and ML that disagree with DeepMind.

Plot-twist: Deep Mind comes out as Some Bozo comment author

Bozo et al

Well my writing style has been the subject of abuse on this website in the precise form that it resembles words generated by a bad algorithm.

"Only a true AI would deny their being."

I awoke wondering am I a man dreaming I am an AI? Or an AI dreaming I am an man?

DeepMind is arguing from first principles. SomeBozo is arguing by analogy. DeepMind will achieve something and SomeBozo will achieve nothing.

The vast majority of ideas are wrong. Every idea is wrong until it leads to the one that is right.

This idea might be the right one, or it might be close to the right one, or it might be far from the right one, but the trajectory is headed toward the right idea. SomeBozo has no trajectory. The best he can do is watch from the sidelines.

Reply on news.ycombinator.com