A new way to build with Large Language Models

Neat, this is like a SaaS version of Langchain[1]. For those who haven't already seen it, Langchain is a framework that gives you the ability to do this as well as integrate out of the box solutions for other common problems like memory.

[1] https://langchain.readthedocs.io/en/latest/

What's your guidance concerning the risk of prompt injection with applications built on this kind of platform?

If developers on your platform are building code that accepts prompt input from untrusted users do you have any measures in place to avoid them tricking the system into executing functions in an inappropriate or unintended way?

One thing I'd find useful would be a limit on the number of functions that can be called during a single execution - and rate limits and budget calls on functions generally might be smart as well.

Relevant to this discussion is the Toolformer paper that was just released today from Meta AI. To me the exciting result from the paper is that it appears that much smaller models can be used to translate human intent into actions: https://arxiv.org/abs/2302.04761

If you ask an existing LLM for a stock quote, it will hallucinate one, or give you a price it found in the snapshot of the web that it was trained on. But the intent here is to build a system that knows it should call an external service to get the right answer. It seems that there's a risk, though: the platform might use the correct service and get an accurate answer. Or it might make up a plausible answer. Who knows?

Wow, it seems like a fully frightening idea to have an ml/dl system with no clue if its right or wrong about anything (as it knows literally nothing, including that fact) to have any kind of role in autonomous agency.

Neat. I was just thinking that one thing you can do that current large language models don't do is use a calculator to check your work.

So, the first thing I'd want to try is hooking up some basic arithmetic to see if its math gets more reliable. And if not, why not?

Seems like someone would have tried this already?

Looks interesting, can you show us more use cases on the landing page? The stock price example is okay, but I want to see what I can use this for that I can't currently do easily. What problems does this solve?

Could you use this to provide better support chat bots? That is, support bots obviously need to be scoped so that they only give information the user asking has rights to, e.g. "Show me the stocks in my portfolio". Assuming the user has logged into the interface showing the chat, is there a way to pass that user's unique access token through to one of these Fixie Agents to ensure that the data being accessed is scoped by that access token?

if I’m understanding this correctly this is a way for LLMs to call external APIs by showing them special key words that call functions

I assume the output of the LLM is monitored by fixie and the result of the function inserted allow the LLM to respond again?

This looks great. I really think a good use case for the current generation would be to be able to extend it with your own data.

Having internal users ask questions then have a ChatGPTesque system answer with injected data would be nice. Would be very tightly controlled (or it couldn't answer what it doesn't have).

Something between a rigid chatbot that's exists now and ChatGPT that just makes up plausible answers.

Is this OpenAI based, or do you host your own LLM? Can I provide my own fine tuning so that I can teach the AI to call my code better?

The next stage: you guys carefully curate a gigantic list of extremely high quality agents and I dont have to make any agents for my specific use case.

Stage after that: GPT, or some other AI constantly adapts / curates the agents for me based on my notes, conversations, etc. (Although a privacy nightmare)

I've wanted something like this for years! And just started building it myself last month (mostly for home automation). I like the multi-agent model, and the "thought" steps are pretty interesting. But how do you deal with context length with all the back and forth?

This would be better to discuss when there's something people can try, rather than just a waitlist.

Another big step would be to give LLMs access to computational sandboxes (like wasm) to do some calculations based on what it deems worthwhile calculating. (without access to bash/aws)

For a team of such apparently impressive people, their website doesn’t work on phones with a large font size for accessibility.

Is matching prompts to agents a popular research problem? Does anyone know of some related work?

nice. I've been playing at openai for the last couple weeks, experimenting with extracting commands and queries from user prompts trying to gauge how to do pretty much exactly this. this kind of thing is going to be awesome.

> You can also build your own Agents that are hosted on your own infrastructure, using the tools and examples in the open source Fixie SDK.

What exactly is running on my infrastructure? Do I need to buy GPUs? What state needs to live on Fixie cloud infrastructure?

Should this be called SLM (Small language models) ?

I read a NY Times article by Ted Chiang today in which he made a kind of "stochastic parrot" argument for chatGPT - https://www.newyorker.com/tech/annals-of-technology/chatgpt-.... I believe, on the other hand that chatGPT may eventually be able to generate AGI, but that this will occur emergently and spontaneously. In other words, it will be difficult to predict.

One of the conditions for this, is for chatGPT models to start being able to write their own code in order to produce models of themselves that are more accurate and more efficient. Given this ability, and some fitness criteria, genetic algorithms may be used to create new LLMs. This sounds like science fiction, but once the compute requirements come down for these models (by a couple orders of magnitude), I believe this may be possible.

To what extent does your model allow for semantic models to create semantic models that are themselves more efficient in relation to some fitness criteria? Can I tell a model "You (model) I want you to reproduce using interaction with these other models (some collection of other models) and have the child model offspring be more efficient according to this criteria [for example the resultant models will create short stories that are more likely to receive high ratings on a subreddit devoted to short stories]".

You would need to get around the "model pollution" problem in which LLM models pollute the space for which the models generate data because other models are producing web artifacts (Ted Chiang's Xerox of a Xerox problem). I call this the problem of alpha (direct experience). One of the ways I've thought of to fix this is to have models trained on direct user input (such as cell phone video and pictures from a single user) - I have to admit that I got this idea from Neal Stephenson's Snow Crash (see Gargoyle). If your platform can integrate with visual processing this may have a high information density - object detection in daily videos demonstrating how objects are related to each other in the real world of the user and correlating these into a semantic network.

I'd also suggest that Obsidian integration might be useful.

This is exciting, thanks for making the Fixie SDK public.