It’s really crazy how Stable Diffusion seems to be very on par with DALL-E and you can run it on “most” hardware. Is there an equivalent for GPT-3? I don’t even think I can run the 2M lite GPT-J on my computer…
You can invite the bot to your server via https://discord.com/api/oauth2/authorize?client_id=101337304...
Talk to it using the /draw Slash Command.
It's very much a quick weekend hack, so no guarantees whatsoever. Not sure how long I can afford the AWS g4dn instance, so get it while it's hot.
Oh and get your prompt ideas from https://lexica.art if you want good results.
PS: Anyone knows where to host reliable NVIDIA-equipped VMs at a reasonable price?
The next thing I considered was just buying up a ton of 3060 12gb cards (saw a few new ones for $330) and just hosting a server from my house. This might be a good option if you don't care about speed but care about throughput.
RTX 3090s are also decent in terms of price per iteration of Stable Diffusion. If you want to build a fast service like Dreamstudio I think it's the only option to be able to do it at a reasonable price. If you want to host these in the cloud using consumer RTX cards, you'll have to go with less reputable hosts since Nvidia doesn't allow it. I don't want to name any since I can't vouch for them, but there are some if you search. The cheapest option will be to buy them and host it yourself.
I'm still researching what the best price/performance is for hosting this so if you have any findings please share.
I‘m not really affording this, to be honest — I’m looking forward to switch to a Spot instance tomorrow, which could bring costs down to about $0,20 per hour, but even then I will have to switch it off in a couple of days.
I‘m working on a significant speed improvement — if that works out, and users get a result in under 1 minute if they are first in line, then maybe it‘s possible to make the bot finance itself through a credits system.
I quickly polished things and created a useful README - hopefully it's all correct. If not, let me know!
Also can run an 8 bit quantized version pretty easily. This takes ~6gb RAM.
The results seem far off from GPT-3 but apparently it can get good results when fine tuned.
Bigger models like OPT 66B can run on cloud machines (or a really big local system)
OPT 175B weights are not open but can be applied for.
175B would require something like 500GB RAM if not quantized. That's a lot, but it's possible to build that locally if you have a couple 10's of thousands of dollars.
Wait a few years and 175B on a GPU will be no problem.
Apparently it doesn't affect the results significantly.
More info:
Plus, there are huge gaps in training. Ask it to draw something simple, like "a penis" and you get nightmare fuel....