ajhai

> The only way I can think of is to provide a comprehensive list of everything the tool was trained on.

There are some startups working in the space that essentially plan to do something like this. https://www.konfer.ai/aritificial-intelligence-trust-managem... is one I know of that is trying to solve this. They enable these foundation model providers to maintain an inventory of training sources so they can easily deal with coming regulations etc.

Enterprises with Azure contracts are using embeddings endpoint from Azure's OpenAI offering.

It is possible to use llama or bert models to generate embeddings using LocalAI (https://localai.io/features/embeddings/). This is something we are hoping to enable in LLMStack soon.

We have a cloud offering at https://trypromptly.com. We do offer enterprises the ability to host their own vector database to maintain control of their data. We also support interacting with open source LLMs from the platform. Enterprises can bring up https://github.com/go-skynet/LocalAI, run Llama or others and connect to them from their Promptly LLM apps.

We also provide support and some premium processors for enterprise on-prem deployments.

Explicitly calling out that they are not going to train on enterprise's data and SOC2 compliance is going to put a lot of the enterprises at ease and embrace ChatGPT in their business processes.

From our discussions with enterprises (trying to sell our LLM apps platform), we quickly learned how sensitive enterprises are when it comes to sharing their data. In many of these organizations, employees are already pasting a lot of sensitive data into ChatGPT unless access to ChatGPT itself is restricted. We know a few companies that ended up deploying chatbot-ui with Azure's OpenAI offering since Azure claims to not use user's data (https://learn.microsoft.com/en-us/legal/cognitive-services/o...).

We ended up adding support for Azure's OpenAI offering to our platform as well as open-source our engine to support on-prem deployments (LLMStack - https://github.com/trypromptly/LLMStack) to deal with the privacy concerns these enterprises have.

Adding to the list:

- https://github.com/trypromptly/LLMStack (build and run apps locally with LocalAI support - I'm a maintainer)

Check out LLMStack (https://github.com/trypromptly/LLMStack). It allows you to build LLM apps and chatbots and provides you with an API for integration.

It should be possible to write a simple script to link LLMStack's app with your IRC channel. API documentation for running the apps is at https://llmstack.ai/docs/apis/introduction#apps. Let me know if you need help with this.

We were using OpenAI's codex model with a bunch of few shot examples for natural language queries over blockchain data coupled with a finetuned completions model for plot the results when we were working on makerdojo.io.

Without the ability to fine tune Codex, we had to employ a bunch of techniques to pick the right set of examples to provide in the context to get good SQL. We have come very far since then and it has been great to see projects projects like sqlcoder and this.

Pieces of the pipeline powering makerdojo eventually became LLMStack (https://github.com/trypromptly/LLMStack). We are looking to integrate one of these SQL generators as processors in LLMStack so we can build higher level applications.

Shameless plug: recently launched LLMStack (https://github.com/trypromptly/LLMStack) and I have some custom pipelines built as apps on LLMStack that I use to transcribe and translate.

Granted my use cases are not high volume or frequent but being able to take output from Whisper and pipe it to other models has been very powerful for me. It is also amazing how good the quality of Whisper is when handling non English audio.

We added LocalAI (https://localai.io) support to LLMStack in the last release. Will try to use whisper.cpp and see how that compares for my use cases.

Thanks for the issue. Will take a look. In the meantime, you can try the registry image with `cp .env.prod .env && docker compose up`

> Also do you have plan to support llama over openai models.

Yes, we plan to support llama etc. We currently have support for models from OpenAI, Azure, Google's Vertex AI, Stability and a few others.

There is a generic HTTP API processor that can be used to call APIs as part of the app flow which should help invoke tools. Currently working on improving documentation so it is easy to get started with the project. We also have some features planned around function calling that should make it easy to natively integrate tools into the app flows.

A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data.

Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at least in the initial days. To this end, we just open sourced LLMStack (https://github.com/TryPromptly/LLMStack) that we have been working on for a few months now. LLMStack is a platform to build LLM Apps and chatbots by chaining multiple LLMs and connect to user's data. A quick demo at https://www.youtube.com/watch?v=-JeSavSy7GI. Still early days for the project and there are still a few kinks to iron out but we are very excited for it.

This status page is for WhatsApp Business API (https://developers.facebook.com/docs/whatsapp/cloud-api/) and not for the consumer product that most of us depend on.

We have just added support for ElevenLabs. https://twitter.com/ajhai/status/1659642782607372288 is a quick demo of the platform if interested.

We allow users to build LLM chains at trypromptly.com as apps. Once an app is built, it can be integrated into other applications as iframe embeds or can be called via our APIs so custom frontends can be built.

We also have Zapier integration (https://zapier.com/apps/promptly/integrations) so apps on Promptly can be invoked from zaps

For fun, I whipped a quick one using our platform at https://trypromptly.com/app/65573e71-c05b-4567-87e8-49688c82.... (Initial chunking of PDF takes a bit of time because of rate limits with my account)

We let users build similar apps at https://trypromptly.com without writing code

When YouTube urls are used, we extract audio and transcribe it so users can either save as datasource or pass that content to other LLMs.

Ask YC (https://trypromptly.com/app/bd0a3f40-44a9-4789-b3d6-ec185134...) is an app built with content from YC startup library and some of them are YouTube videos

Yeah we store user provided data in a vectorstore and allow retrieval as a processor. Character storage is essentially number of bytes stored in our db. Since we extract text from uploaded content and persist that, we went with number of characters for storage. There are limits on the total number of requests at account level.

Congrats on the launch, Andrew and Ani. This is most of the backend stack any LLM app needs. We built similar infra when we first started using Open AI's completions API in our product last year.

With respect to GPT-4 and image inputs, how do you see search working with such inputs on the platform?

Nice. We started this route and ended up building https://trypromptly.com, a no-code platform to build LLM apps and chatbots.

Congrats on the launch. We are building something similar at https://trypromptly.com that allows chatbot building on user's own documents powered by GPT. We also allow users to build AI powered apps without writing code.

For example, https://trypromptly.com/app/9da10a4f-6d20-431e-98f7-048fab81... is a chatbot built on Coursera video transcript. Similarly https://trypromptly.com/app/d478594d-2082-46c9-bee7-f057f4bc... is a web app that generates resume based on user information and the position they are applying to. Both are powered by Open AI message completions API and built using Promptly's no-code builder. Another fun app we built allows end user to provide a audio file, pipe that to Whisper and use that in the chain below to extract summaries etc.

Ps: We are still updating our landing page with app builder videos and other content

> In the hours before we took ChatGPT offline on Monday, it was possible for some users to see another active user’s first and last name, email address, payment address, the last four digits (only) of a credit card number, and credit card expiration date

This is a lot of sensitive data. It says 1.2% of ChatGPT Plus subscribers active during a 9 hour window, which considering their user base must be a lot.

Letting users pick a comparison metric of their choice is a good option till something better comes along. Good luck with Vellum!

Congrats on the launch! I'm glad to see all the tooling come up in this space.

Regarding tests, how do you evaluate the generated completions for tests? Allowing users to execute a set of tests against a prompt and show completions for visual inspection is a good start but imho doesn't scale when the app is in production with a large corpus of tests. Something we are exploring right now is to generate a similarity/divergence score between generated completions to make this easy at scale.

Disclosure: We are building something very similar at Promptly (https://trypromptly.com) out of our experience using GPT-3 at MakerDojo

Oops. Thanks for the correction.

We are still figuring out pricing. Is there an email I can reach you at? Would love to chat about your use case if you could send me an email at ajay[at]trypromptly.com

We have added them to our playground at https://trypromptly.com if you want to check them out. https://twitter.com/ajhai/status/1631020290502463489 has a quick demo

gpt-3.5-turbo is missing from Open AI's playground. For anyone looking to play with these models, we have now added them to Promptly playground at https://trypromoptly.com. https://twitter.com/ajhai/status/1631020290502463489 has a quick demo.

We have added it to https://trypromptly.com's playground. https://twitter.com/ajhai/status/1631020290502463489 is a quick demo

These adversarial examples are a great way to form a mental picture on how the model responds in general and can help with prompt engineering when using LLM models.

If anyone is looking for an easier way to manage their prompts in production (versioning, templating, caching and testing), please checkout https://trypromptly.com. We built this out of our experience using Open AI's completions API at https://makerdojo.io which is now powered by Promptly. (Testing and sharing are WIP)