Hacker News Reader

Comment by _acco | Hacker News Reader

If you haven't seen it, the documentary on DeepMind's AlphaGo is very well done. The climax revolves around the AI expressing what we humans call "creativity" in a match against decorated player Lee Sedol. Watching Lee Sedol's reaction is deeply moving.

Free to watch on YouTube: https://www.youtube.com/watch?v=WXuK6gekU1Y

View on HN

This comment, that AlphaGo showed "creativity", seems so ridiculous to me. Alphago is operating purely on bayesian statistics (with very good priors) so it's decisions are exactly the opposite of creativity.

It's an invitation to define (or re-define) creativity, my friend :)

If it seems ridiculous to you, that's a good signal that you'd get a lot out of digging into philosophy of mind.

Where do our thoughts come from? Creativity? Is it so preposterous that these too arise from probabilistic electrical storms - but in wet matter as opposed to hard?

Where thoughts come from may be too complex for us to model now, maybe ever. But if you look deep enough, I think you'll find at heart deterministic processes driven by networks that were trained through nature + nurture.

Maybe creativity is less ethereal than we perceive. Maybe it would benefit us to start considering how non-sentient intelligence can achieve it.

I am not making, and do not explicitly agree with, the statement that only human brains are imbued with 'creativity'. However, the core of the alphaGo algorithm is just a monte-carlo tree search of the game initially populated with prior estimates of the value of each game state(with the generation of the prior value estimates being carried out by the DNN). I see no space in this simplistic statistical game analysis for concepts such as creativity. The move in question was chosen because it's prior value estimate by alphaGo was favorable and upon further analysis was concluded to be the most likely to result in a game victory relative to all other available moves. If alphaGo was only trained on human matches and subsequently was able to discover moves that drastically deviate from typical human strategies, I would be more open to labeling it as creative. However, most of alphaGo's training is carried out against itself, it's entirely possible that alphaGo has made this particular(or similar) move many times against itself during training.

Much of human creativity is likely linked to some sort of Bayesian analysis, but it also can happen based on limited sample sets and unseen situations. As you point out, the Alpha Go likely just saw that scenario out of millions of random scenarios never explored by humans before. Given the enormous compute resources used, it's possible AlphaGo has made more Go moves than all of humanity ever combined.

Reminds me of a quip about Richard Feynman that he'd spend many hours honing a new way of doing a physics problem, but carefully present it as if he just did it off handed. So we just need to link human brains with cybernetic implants to compete. ;)

Perhaps in 100-200 years, some neuroscientist will make the same remark about a painter’s brain — “it’s just math, I can model it!”

And biological life has been doing... what exactly?

This criticism reminds me a bit of one of my least favorite interview questions I've had: what was your most creative solution to a problem? I found it awkward because creativity tends to be what we recognize in others rather than ourselves. If I came up with a solution I probably wouldn't consider it that creative.

Of course I still made something up.

It's not creativity. It's optimization. No new explanatory knowledge is created unless a human analyzes the move and learns something from it.

You can think of probabilistic tree search as creativity and the neural network as intuition. Like, what are you doing when you think of a cool new idea? You think of a lot of ideas and narrow them down slowly until you reach a cool new one. It’s not magic.

We have literally no idea what thoughts are or how they are implemented in human brains. This is such a silly argument at this point. For 50 years people have been saying "obviously humans are just doing a probabilistic search" or some other such evidence-free assertion, claiming "it's not magic", an obvious strawman since no one is claiming it is. And for 50 years we have not created any system capable of creating new explanatory knowledge. Yet still the same claims are made, week after week.

I’m not saying humans are doing probabilistic search (not saying they are not either). I’m saying that if probabilistic search produces a result that people see as creative, then we can regard probabilistic search as something that produces creativity.

Furthermore, just because we don’t have the computational capability yet to replicate human thought processes using, for example probabilistic search, does not mean that probabilistic search is not part of replicating human thought processes.

IMHO creativity isn't really as conscious as people seem to think. I have had a handful of hyper productive songwriting sessions and by the end it always felt more like I was the horse than the rider. Neither was anything made so much as it was found.[1]

There are constraints and limits and rules: rhyme scheme, meter, key, chord progression, genre, structure, and whether or not those things will support the consistency and coherence of the content... that's barely scratching the surface. The "choices" and the "thinking" are highly constrained. Few things are going to work. It's often either "this way" or "wo, back it up".

Even in something like Jazz there are still intricate rules at play and the true greats were eccentrics and/or taking loads of drugs in order to push the boundaries where they could.

[1] And then that one time where I finally cracked the melody at 3am, only to realize in the morning that it was Neil Young's "Heart of Gold". Even my creation was not my creation!

Reply on news.ycombinator.com