Zacharias030

With all due respect, what does it say about us that „famous researcher voices his speculative opinion“ is an instant top 1 on hackernews?

Should be the top comment.

MSL is not only those few high profile hires.

Impressive setup :)

I think Alexandria.

Thanks! mmd.

touché

Must have been a program without much trickle down into gov tech

some of it are cultural products too.

Do you know of a systematic study of the AG - Lee Sedol match with contemporary KataGo? I’ve never seen this written up.

The tasuki who put Cho Chikun sensei‘s problems online without solutions as pdfs?

Thank you, Sir! I have learned so much from these pdfs.

I appreciate the discussion as well.

After having checked the positions you mentioned, I had put you somewhere in the 4-10 kyu range. For me this was the time when I found out how important it is to play efficient and fast and play big spots. In order to improve further, I had to learn to appreciate thickness more because of the downstream benefits. As the fighting of both players becomes stronger, thicker shape makes your side of the fight much easier and prevents everything from crumbling.

Take your first example where you suggested 1-point jump or keima instead of nobi. The main point here is about 1) liberties and 2) having your group‘s „head“ run ahead. There is also this concept of a hard head or a soft head and the nobi creates a wonderful hard head ahead of your opponents stones that can not be bullied. If you play any other move, a stronger opponent will (locally) immediately and gladly play that hane without thinking, which takes a liberty and forces you to play again to defend your stones. Depending on your moves, the opponent would turn your group into a big dumpling and collect very nice stones on the outside in the process.

Perhaps you would find the games of Kitani Minoru or Lee Chang’ho inspiring.

why is this downvoted?

I don’t share that sentiment.

Yes, at some point when people are somewhat able to take a decent lead home the fuseki becomes important. Before that, beginners really need to understand how to „move“ their stones, how to defend and connect their groups and how to cut and capture.

If you see a strong player win against a weak player with a large handicap it always goes down the same way: the strong player places stones all over the board such that eventually many many skirmishes appear all over the board and then she is patient to take small advantage after small advantage, manifesting groups and territories out of what looks like thin air to the other player.

At a somewhat higher amateur level and above the fuseki again loses importance and the distinguishing factor is fighting skills and judgement, fuseki and prep just becomes table stakes.

That time when I reverse engineered JRR Tolkien‘s Lord of the rings from symbols engraved on dead trees. Took me three summers…

I think people often interpret a bit too much. Perhaps it’s just some researchers who got enough freedom to run and publish interesting work within apple. For a company like apple it makes sense to have a research lab with considerable freedoms even if protein folding is not a core interest, which is why you see it published but not the formula for the new Corning Gorilla glass…

ostensibly it does.

Yes, but the discussion is about Multi-Token Prediction (Gloeckle et al. 2024) which is only incidentally useful for speculative decoding.

There is no reason that it couldn’t be beneficial for training though.

yes, if you know the sequence of tokens ahead of time you can verify them about as quickly as you can generate one more token because of the parallelism benefits.

If you don’t know the future tokens though, then you can’t, and blind guessing of tokens is infeasible because the vocabulary contains circa 100k possible different tokens.

When Swe-Bench was new in 2023, it was — with all due respect — a bit of a niche benchmark in LLM research. LLMs were so incredibly useless at solving these tasks that I think you could find a bit more empathy for the original academic authors. I don’t think the Theranos example applies. Even the flawed benchmark was good enough to get us from ~GPT4 to Claude 4‘s coding ability.

Thanks for that! where can I find your writing?

You realize that this problem in SWE-Bench was discovered and publicized by people within those FAANG corporations?

should be the top comment.

I agree mostly. They are all that you say, but if you think about the conditional distribution that you are learning, there is nothing preventing us in principle from mapping different contexts to the same responses. It is rather a practical limitation that we don’t have sufficient tools of shaping these distributions very soundly. All we can do is throw data at them and hope that they generalize to similar contexts.

We have observed situations where agentic LLM traces on verifiable problems with deterministic (greedy) decoding lead to either completely correct or completely wrong solutions depending on the minutes on the clock which are printed as coincidental output of some tool that the LLM used.

I think there may be some mild fixes to current models available , for example it is worrying that the attention mechanism can never fully disregard any token in the input, because the softmax will always assign a > 0 weight everywhere (and the NN has no way of setting a logit to -infinity). This directly causes that it is extremely difficult for the LLM to fully ignore any part of the context reliably.

However Yann LeCun actually offers some persuasive arguments that autoregressive decoding has some limitations and we may need something better.

While I get that this is how LLMs work, I think you should think backwards from the user / from what AI as a field is aiming for and recognize that the „naive“ way of the parent to ask for reliable responses no matter what the „context“ is, is exactly what a good AI system should offer.

„The context is the input“ betrays a misunderstanding of what (artificial) intelligence systems are aiming for.

It is consistent with what the commenter was saying.

In any case, for Go - with a mild amount of expert knowledge - this limitation is most likely quite irrelevant unless in very rare endgame situations, or special superko setups, where a lack of moves or solutions push some probability to moves that look like wishful thinking.

I think this is not a significant limitation of the work (not that any parent claimed otherwise). MuZero is acting in an environment with prescribed actions, it’s just “planning with a learned model” and without access to the simulation environment.

—-

What I am less convinced by was the claim that MuZero reaches higher performance than previous AlphaZero variants. What is the comparison based on? Iso-flops, Iso-search depth, iso self play games, iso wallclock time? What would make sense here?

Each AlphaGo paper was trained on some sort of embarrassingly parallel compute cluster, but all included the punchlines for general audiences that “in just 30 hours” some performance level was reached.

unhealthy fumes in the garage!!

I strongly believe that our concept of intelligence is like the „god of the gaps“ [0]. Intelligent is only what we haven’t yet explained.

Chess computers surely must be intelligent, but then deep blue was „just search“.

Go computers surely must have intelligence because it requires intuition and search is intractable, but then it’s „just CNN based pattern matching“.

Writing essays surely requires intelligence, because of the creativity, but then it‘s actually just a „stochastic parrot“.

We keep attributing intelligence to what is currently out of reach even as this set is rapidly shrinking before out eyes.

It would be better to say that intelligence is an emergent phenomenon and that behavior that seems intelligent is intelligent.

[0] https://en.m.wikipedia.org/wiki/God_of_the_gaps

how though?

Wouldn’t you say that now, finally, what people call AI combines subsymbolic systems („gradient descent“) with search and with symbolic systems (tool calls)?

I had a professor in AI who was only working on symbolic systems such as SAT-solvers, Prolog etc. and the combination of things seems really promising.

Oh, and what would be really nice is another level of memory or fast learning ability that goes beyond burning in knowledge through training alone.