Hacker News Reader

I also wonder about JS only, Python only, etc models.

Maybe the future is a selection of local, specific stack trained models?

There is some recent work on modularizing knowledge in LLMs.

It might be possible to train a big generalist that is a composition of modules, some of which can be dropped dynamically at inference time, depending on the prompt.

Cool. Thanks for sharing. I am thinking about creating a series of smaller models for specific purposes and then orchestrating them so they mirror the human brain which is a bunch of subsystems that give multiple opinions about the same stimulus

Interesting direction. I’ve also been thinking about modular / subsystem-based approaches for specialized tasks in small AI systems.

These models being able to generalise at coding will likely get worse if you remove high quality training data like all of python.

Reply on news.ycombinator.com