10xDev

An authoritarian regime is not your friend and will pullback the moment their own models become highly capable.

Why wouldn't they want their product called super intelligence?

Use compression sleeves if you mean knees / elbows. If you mean rotator cuff, do face pulls. This isn't medical advice, just speaking from experience.

And just like that Elon Musk shifted the overton window away from "unthinkable" on yet another topic.

Anything you do as a university / person reflects your values. This shows a lack of integrity.

Hey buddy, not everything is about cold calculation. Some of us value culture and integrity. These are going to matter even more in the age of AI not less.

You are in the warmup phase for one of the greatest take offs in human history. Enjoy the ride.

You realise tokens are just data and data can represent anything.

I believe this is alignment working as intended.

So once we have online learning in LLMs, what do you think you will have that makes you intelligent but not LLMs? Better learning efficiency? That will be improved as well.

I think we need to start moving on from the term LLMs because it clearly confuses people since they started modelling more than just language.

AI is more like an ongoing research project than it is a product. GPUs are currently the product that is profitable and Nvidia will keep funding labs to keep buying GPUs.

Zugzwang is everywhere.

Not exactly scientific but it is at least entertaining and some things do sound plausible https://www.youtube.com/watch?v=Gw_hnD7m00M

Risk/reward. Isn't the reward worth the risk?

The idea that they are entirely trained on human intelligence is already outdated. Yes, the earlier models relied heavily on RLHF and human curated data but we have since moved on to synthetic data produced by the models themselves and reinforcement learning with verifiable rewards (RLVR).

Eventually the bottleneck might be people themselves.

Compute will always be the bottleneck even if this were true.

If Astra can actually reason in 3D space, I would like to see it play The Witness.

This statement is behind times, go and watch astra play Pokemon: https://www.twitch.tv/gpt_plays_pokemon

You generally forget things you don't retrieve. It is not really a bug but a way to declutter for efficiency. That's not the same as it not fitting your context window because it was at the beginning.

Unless you infinitely increase the context window, its memory will always be limited. And their latest ARC3 result with and without harness demonstrates how important not discarding memory is for learning.

It will be AGI once it can update its own weights. It can't be "general" intelligence if its weights are frozen and requires to be updated manually.

It is about memory retention. No heavy lifting done on the reasoning side so I hardly see anything misleading here.

Edit: update from fchollet https://x.com/fchollet/status/2095598451115614371

No, the unsolved problem of AI is continuous learning. We never stop learning, we don't have a "training phase". You are always updating your world model even when you sleep. Also more quality training data does lead to greater learning efficiency as you have more priors to work with.

That’s why you have a private dataset.

It is remixing what the industry created, not advancing past it. OpenAI even ended Sora in under a year. Anthropic doesn't even bother.

Not sure where I implied they are "raw base models" and not sure what "various adjustments" means here or how "ton of RLHF" contradicts anything. If we look at research for open source models, "adjustments" usually come in the form of efficiency gains which directly contributes to the ability to scale or synthetic data pipelines to increase the dataset and increasing the context window.

Coding models are not the destination. Coding models are just part of the bootstrapping process towards general intelligence.

Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.

This is putting Nintendo on a pedestal. Major Japanese studios in general are consistently publishing great games and increasing hiring count / raising salaries. Switch 2 is also essentially just a spec bump from switch 1 which came out in 2017.

https://www.gamedeveloper.com/business/atlus-to-increase-sal...

https://www.videogameschronicle.com/news/sega-is-raising-jap...

https://gameworldobserver.com/2024/03/06/capcom-salary-raise...

https://automaton-media.com/en/news/sonys-game-division-anno...