Some of our favorites and more prolifically used were OpenAI API, ChatGPT, Midjourney, and Jasper.
Curious to hear what other AI tools (and frameworks like LangChain etc) y'all are heavily dependent on
VoltaML for stable diffusion imagegen (as AITemplate is really fast, and controlnet and LORAs are super useful)
Llama.cpp for textgen... I use the koboldcpp frontend (and sometimed AI Horde) for experimenting with models, though I am still figuring out the exact stack I want for productivity. I am watching this PR closely:
https://github.com/ggerganov/llama.cpp/pull/1773
As well as Starcoder/Openllama finetunes for summarization, codegen and such.
I've been hearing a lot about AITemplate on Twitter. Seems the consensus is it's hella quick.
I think its overlooked in Stable Diffusion land because its (last I checked) linux/wsl only and HF diffusers based.
But the best test is personal experimentation. Prompt engineering and subjective preference have a massive effect on finetune performance.
right now i'm trying to prove to the company i work that you can turn the endless rules and daily information stream into something people can find easily using sematic search (vector DB) and an AI to summarize categorize, generate embeddings and auto update the DB, while also interpreting the content of the search result and giving a more digestible answer and a source link if the end user needs more information.
Keyword based search is a pain when different products have similar names or the internal search doesn't filter words like "of" and "and" from the keyword search.
After that, if the tests work and i can push the idea forward, it'll probably be OpenAI + a local vector DB (Chroma?) + custom made search page for internal usage
This stack is evolving so quickly but we are convinced it goes back to a distributed systems/infra problems when it comes to complexity, unless you’re training your own foundation models.