We've been testing and developing a few openai based applications. Relatively simple stuff. I topped up my api credit with 10$ a few months ago. I just checked, we burned something like 61 cents over this period. About 126 requests over the last month. That's just us testing but it shows me that this stuff is relatively cheap. The reason we don't have a lot of requests is that most of this stuff pretty much works on the first try. This is not hard to build.

I think AI is an underused capability in applications. People come up with all these big plans to build agents. But even a simple button to "calculate a color palette that takes into account these symbolic names the user entered" is a simple prompt that comes back quickly and doesn't require a lot of reasoning. We use that with Luna and we give it some more context.

Image based stuff is more expensive but still quite reasonable. We do some things with Sol that work on images. It's not fast but the result is worth waiting for. These are features that save the user time and feel a bit magical. They just click a button, wait 20-30 seconds and then magic happens. Beats them clicking around for half an hour.

Local model hosting would force me into higher cost and overhead for less intelligence. That's only a good tradeoff if you are really going to use lots of tokens or you need this for other reasons (e.g. data security). But otherwise you just end up paying for underutilized GPU capacity.