Hacker News Reader

Comment by gamegoblin | Hacker News Reader

Emad (founder of Stability AI) has said they already have video model training underway, as well as text and audio. Exciting times.

View on HN

And copilot-like code, possibly Q1 2023.

"Generate the code base for an advanced diffusion model that can improve on the code base for an advanced diffusion model"

The road to Grey Goo is paved with artificial general intelligence.

Oh no you forgot the important term “but do NOT start turning the universe in to paperclips”.

Salesforce CodeGen (particularly the 16B-multi and 16B-mono models) is pretty good already and can be used with FauxPilot [1] to get an open Copilot-like experience with local compute :) I am also very excited about the upcoming BigCode project though, which is maybe what you're thinking of?

Disclaimer: I am naturally biased since I made FauxPilot ;)

[1] https://github.com/moyix/fauxpilot

[2] https://www.bigcode-project.org/

Is this going to end up into a single model, where its trained on text and images and audio and videos and 3d models, and it can do anything to anything depending on what you ask of it? Feels like the cross-training would help yield stronger results.

These diffusion models are using a frozen text encoder (e.g. CLIP for Stable Diffusion, T5 for Imagen), which can be used in other applications.

StabilityAI trained a new/better CLIP for the purpose of better Stable Diffusions.

Probably not. We're actually headed towards many smaller models that call each other, because VRAM is the limiting factor in application, and if the domains aren't totally dependent on each other it's easier to have one model produce bad output, then detect that bad output and feed it into another model that cleans up the problem (like fixing faces in stable diffusion output).

The human brain is modularized like this, so I don't think it'll be a limitation.

Reply on news.ycombinator.com