Hacker News Reader

Comment by naillo | Hacker News Reader

Probably only 6 months until we get this in stable diffusion format. Things are about to get nuts and awesome.

Emad (founder of Stability AI) has said they already have video model training underway, as well as text and audio. Exciting times.

Is this going to end up into a single model, where its trained on text and images and audio and videos and 3d models, and it can do anything to anything depending on what you ask of it? Feels like the cross-training would help yield stronger results.

These diffusion models are using a frozen text encoder (e.g. CLIP for Stable Diffusion, T5 for Imagen), which can be used in other applications.

StabilityAI trained a new/better CLIP for the purpose of better Stable Diffusions.

Probably not. We're actually headed towards many smaller models that call each other, because VRAM is the limiting factor in application, and if the domains aren't totally dependent on each other it's easier to have one model produce bad output, then detect that bad output and feed it into another model that cleans up the problem (like fixing faces in stable diffusion output).

The human brain is modularized like this, so I don't think it'll be a limitation.

And copilot-like code, possibly Q1 2023.

"Generate the code base for an advanced diffusion model that can improve on the code base for an advanced diffusion model"

The road to Grey Goo is paved with artificial general intelligence.

Oh no you forgot the important term “but do NOT start turning the universe in to paperclips”.

Salesforce CodeGen (particularly the 16B-multi and 16B-mono models) is pretty good already and can be used with FauxPilot [1] to get an open Copilot-like experience with local compute :) I am also very excited about the upcoming BigCode project though, which is maybe what you're thinking of?

Disclaimer: I am naturally biased since I made FauxPilot ;)

[1] https://github.com/moyix/fauxpilot

[2] https://www.bigcode-project.org/

Isn't Imagen a diffusion model?

From the abstract: > We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models

"Stable Diffusion" is a particular brand from the company Stability AI that is famously open sourcing all of their models.

Pedantically, Stable Diffusion v1.4 is the one model where weights were open sourced and released. Stable Diffusion v1.5, announced September 8th and live on their API, was to be released in "a week or two" but still has yet to be released to the general public.

https://discord.com/channels/1002292111942635562/10022921127...

SD 1.2 and 1.3 are open source too

Even more pedantically, SD weights are in fact not open source, they're under a source available license.

* If weights are copyrightable in your jurisdiction (who knows!)

Reply on news.ycombinator.com