AlphaGo documentary (2020) [video]

If human players do the new AI move, an attack to cancel out the better fighting position is unavoidable. We've learned from AI how to attack and defend in this situation, so even human players are prepared and know what to do. So it is fair game for the two human brains again.

The player who gets out of this fight with the worse position is on a rather high disadvantage for the rest of the game.

So you can think of this new strategy as doubling the stake of the game very early on. For players who are really good at fighting and handle the complexity that will arise from this fight this strategy is a good choice.

> It is common to see casual players copying the AI style and digging their own grave while doing so.

I think this point is greatly overstated. Amateur go style has always been modelled on professional play even though amateurs have no idea of the subtleties of it, and this was rarely bemoaned much in the past. It's easy to find examples of amateur players getting it wrong and turning a slightly-good position into a bad one, but this is true of just about anything and I think there's a lot of confirmation bias in associating it with AI moves in particular - it doesn't in itself mean they've disadvantaged themselves by trying.

It also greatly overstates how bad it is to play the 3-3 early and not fight aggressively. Even if you don't worry about the fighting variations we're talking about a small number of points, much less than amateur games tend to swing via mistakes every few moves. This is also true about those professional-like lines that have always been mimicked but played imperfectly, and is probably a big reason the lack of amateur understanding of them never mattered too much. Plus let's not forget that both amateur players in the game are relatively bad at fighting, it's just about as hard for the defensive player to really get it right as for the offensive player to utilise their theoretical advantage.

Overall I think the modern focus on AI-like play is much more of a fashion in line with other general shifts following what's professionally popular than a mistake. Players do pay attention to how well they do and associate it with specific moves and lines they recognise they don't understand well, but AI-inspired lines don't seem to have attracted any widespread cautiousness.

> Then came AI and it found out, given enough fighting power, it can mostly cancel that later game advantage.

Also worth noting that the AIs play the 3-3 invasion in a specific different way to what was the accepted human pattern, and this alone does make a big but understandable difference to how the shape commonly continues. It isn't that they only turned around the evaluation of the standard position through superhuman reading, they revealed a line that humans had undervalued but could then learn from.

Move 78 was not a lucky break. At least in my mind.

There are not many movies I cried to, in this movie I cried twice, once when Lee Sedol was apologizing for the loss, and once when he made move 78 and Alpha Go's probability to win dropped with 10%.

Now for me move-78 means hope.

The move in question should be move 102, which is an invasion that human would not try.

I feel like the emphasis of your comment misses how the game has changed for humans since AlphaGo.

"hey, China built the most extensive high speed railway network in the world in the last 15 years and the fastest operating trainsets, we can and should do better" they do something else entirely.

https://hackernewstitles.netlify.app/

https://en.wikipedia.org/wiki/AlphaFold#SARS-CoV-2 :

> AlphaFold has been used to a predict structures of proteins of SARS-CoV-2, the causative agent of COVID-19 [...] The team acknowledged that though these protein structures might not be the subject of ongoing therapeutical research efforts, they will add to the community's understanding of the SARS-CoV-2 virus.[74] Specifically, AlphaFold 2's prediction of the structure of the ORF3a protein was very similar to the structure determined by researchers at University of California, Berkeley using cryo-electron microscopy. This specific protein is believed to assist the virus in breaking out of the host cell once it replicates. This protein is also believed to play a role in triggering the inflammatory response to the infection (... Berkeley ALS and SLAC beamlines ... S309 & Sotrovimab: https://scitechdaily.com/inescapable-covid-19-antibody-disco... )

Is there yet an open implementation of AlphaFold 2? edit: https://github.com/search?q=alphafold ... https://github.com/deepmind/alphafold

How do I reframe this problem in terms of fundamental algorithmic complexity classes (and thus the Quantum Algorithm Zoo thing that might optimize the currently fundamentally algorithmically computationally hard part of the hot loop that is the cost driver in this implementation)?

To cite in full from the MuZero blog post from December 2020: https://deepmind.com/blog/article/muzero-mastering-go-chess-... :

> Researchers have tried to tackle this major challenge in AI by using two main approaches: lookahead search or model-based planning.

> Systems that use lookahead search, such as AlphaZero, have achieved remarkable success in classic games such as checkers, chess and poker, but rely on being given knowledge of their environment’s dynamics, such as the rules of the game or an accurate simulator. This makes it difficult to apply them to messy real world problems, which are typically complex and hard to distill into simple rules.

> Model-based systems aim to address this issue by learning an accurate model of an environment’s dynamics, and then using it to plan. However, the complexity of modelling every aspect of an environment has meant these algorithms are unable to compete in visually rich domains, such as Atari. Until now, the best results on Atari are from model-free systems, such as DQN, R2D2 and Agent57. As the name suggests, model-free algorithms do not use a learned model and instead estimate what is the best action to take next.

> MuZero uses a different approach to overcome the limitations of previous approaches. Instead of trying to model the entire environment, MuZero just models aspects that are important to the agent’s decision-making process. After all, knowing an umbrella will keep you dry is more useful to know than modelling the pattern of raindrops in the air.

> Specifically, MuZero models three elements of the environment that are critical to planning:

> * The value: how good is the current position?

> * The policy: which action is the best to take?

> * The reward: how good was the last action?

> These are all learned using a deep neural network and are all that is needed for MuZero to understand what happens when it takes a certain action and to plan accordingly.

> Illustration of how Monte Carlo Tree Search can be used to plan with the MuZero neural networks. Starting at the current position in the game (schematic Go board at the top of the animation), MuZero uses the representation function (h) to map from the observation to an embedding used by the neural network (s0). Using the dynamics function (g) and the prediction function (f), MuZero can then consider possible future sequences of actions (a), and choose the best action.

> MuZero uses the experience it collects when interacting with the environment to train its neural network. This experience includes both observations and rewards from the environment, as well as the results of searches performed when deciding on the best action.

> During training, the model is unrolled alongside the collected experience, at each step predicting the previously saved information: the value function v predicts the sum of observed rewards (u), the policy estimate (p) predicts the previous search outcome (π), the reward estimate r predicts the last observed reward (u). This approach comes with another major benefit: MuZero can repeatedly use its learned model to improve its planning, rather than collecting new data from the environment. For example, in tests on the Atari suite, this variant - known as MuZero Reanalyze - used the learned model 90% of the time to re-plan what should have been done in past episodes.

FWIU, from what's going on over there:

AlphaGo => AlphaGo {Fan, Lee, Master, Zero} => AlphaGoZero => AlphaZero => MuZero

AlphaGo: https://en.wikipedia.org/wiki/AlphaGo_Zero

AlphaZero: https://en.wikipedia.org/wiki/AlphaZero

MuZero: https://en.wikipedia.org/wiki/MuZero

AlphaFold {1,2}: https://en.wikipedia.org/wiki/AlphaFold

IIRC, there is not an official implementation of e.g. AlphaZero or MuZero with e.g. openai/gym (and openai/retro) for comparing reinforcement learning algorithms? https://github.com/openai/gym

What are the benchmarks for Applied RL?

From https://news.ycombinator.com/item?id=28499001 :

> AFAIU, while there are DLTs that cost CPU, RAM, and Data storage between points in spacetime, none yet incentivize energy efficiency by varying costs depending upon whether the instructions execute on a FPGA, ASIC, CPU, GPU, TPU, or QPU? [...]

> To be 200% green - to put a 200% green footer with search-discoverable RDFa on your site - I think you need PPAs and all directly sourced clean energy.

> (Energy efficiency is very relevant to ML/AI/AGI, because while it may be the case that the dumb universal function approximator will eventually find a better solution, "just leave it on all night/month/K12+postdoc" in parallel is a very expensive proposition with no apparent oracle; and then to ethically filter solutions still costs at least one human)