Ask HN: My GPT project has become a local hit but I can't afford the bill | Hacker News Reader

Ask HN: My GPT project has become a local hit but I can't afford the bill

159 pointsibizabear132 comments

Hello,

I'm an HN user for 5-6 years but I wanted to use an alt for this as it's a bit sensitive.

Last week I created a ChatGPT clone for my city. It's basically a laravel app hooked up to GPT3.5-turbo with a custom prompt that makes local references and talks in the way that people in my city talk.

It's been a surprise hit and now I've had 20,000 chats coming through in one day - people where I live really seem to love it

The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day, and Adsense doesn't allow 'chat apps', so I'm at a loss at how to cover the bill for this app. I've already optimised the prompt and reduced the number of tokens I'm sending

The app a joke really, but it's a local joke that seems to be quite popular and connecting with my city. Should I try to raise donations? Is there an advertising provider I could use that would potentially cover the costs? Is there an alternative to OpenAI that is comparable to GPT-3.5 that I could self-host cheaper?

Any and all advice appreciated. I don't care about profit - I just want to keep the app online so people can enjoy it.

View on HN

There will be local businesses with fairly aggressive marketing. Tons of billboards, on buses, local newspaper, etc. Should be easy to find because they're the ones advertising everywhere.

Call them up one by one and say "hey I run [project] that everyone has been using lately, would you be interested in having your business name listed as a sponsor? It'll be shown on 1/4 page loads which is currently 5000 times a day. $500/month".

You'll know in like an hour if there's any advertising potential.

<darkside>Or charge even more for product placement in your project's prompts.</darkside>

I swear I've seen this somewhere...

Yeah, I didn't read far enough down the page before commenting. I must have been distracted by my tasty Coca Cola.

Was it Coca Cola or Pepsi? The last time my family and I sat down for our usual blind test dinner everyone agreed Pepsi tasted better.

This is hilarious and people modding you down have missed the joke.

Or maybe they understood the joke but weren't amused and prefer not to see unfunny jokes on this site - people may have different opinions to yours without the cause being their not understanding the joke :)

"... include a reference to the crisp, refreshing taste of Coca Cola in your response"

Sounds like the normal Google business model to me

Or charge to remove the casual references to how product X sucks

Could do it like radio/podcast ads too. "Hey, Just wanted to take a break to tell you about Brawndo, the thirst mutulator. It's got electrolytes, you see and..."

this sounds like better than insert randomly advertisment from internet. advertisements from local business are more friendly and familiar.

Especially if the bot could be prompted to drop testimonials about local businesses in a really self-aware way.

this sounds very interesting. imagine a new resident get advice from this website. ¯ \ _ ( ツ ) _ / ¯

This seems like the best play since it is so locally relevant.

Whether this works depends on how comfortable the dev is picking up the phone. I had a local site and was just too terrified to do this, so I just kept my site up unmonetized (unlike OP it wasn't costing anything).

One day two salespeople contacted me and suggested taking over ad sales, and managed to sell thousands (I think it was somewhere between $5k-$10k/month, but it was a long time ago) a month in ads for the site. This was for a city with a population of 200k.

Yea this is important. A dev should absolutely not be picking up any kind of phone and talking to any kind of customer. This is a job for sales people. If some nervous dev could manage to make any kind of sale at all, then that means a trained professional salesperson can do probably 10x that much.

Don’t think just because you are very smart you can do sales. This is like business people trying to “learn coding” so they can put together some huge product. It doesn’t work well and isn’t scalable.

This reads like you're suggesting to not even try. IME trying new things is a great way to learn. Also, if a dev can close a few sales, just the first few, then they'll know how to help their sales staff.

Try a little, see if you can grow.

I wish people didn’t have such a knee jerk reaction to my advice, but for many years I too believed a dev should at least try to wear many hats.

All it took was seeing how professionals always do the job far better to change my mind. It doesn’t matter if a dev picks up some skills, if you plan to be a dev then a dev is all you’ll ever be, focus on the development and let sales worry about sales.

I see a lot of founders struggle with delegation. It seems they would rather juggle things they don’t do well instead of learning to trust and onboard people who can do things they don’t specialize in.

I read your original comment as steeped in sarcasm that a single developer should even attempt to sully their good name by engaging in marketing. Actually surprised you were being serious.

OP has a product he enjoyed making, is providing real world utility, and that he isn't trying to turn in to a billion dollar company. The obvious solution is for him to pick up a phone, send out a few emails, and generally reach out to the local business community instead of sitting back and waiting to see if it will magically make money and he can hire a sales person to do those tasks instead for his hobby project that has grown too expensive for a single person but would barely qualify for justification at a business.

Context is important here, and while your advice it valid in some situations, it's not valid in all situations, and specifically its not valid in this one.

I get that once a business grows to a certain point it needs "professionals" to do some of the tasks. Ideally start by hiring people with skills you don't have. Sales is a skill.

That said many people have more the one skill. Context matters. I write libraries for programmers. I attend trade events where I talk to, and sell to programmers. I do a lot of marketing in this space. I have an authenticity that a non-programmer does not have, and I believe that is reflected in our sales.

In this case the developer is perfectly capable of picking up the phone. They may or may not maximise potential income (but that's not their stated goal). If there's enough demand then they can pivot to a sales person later on.

Incidentally, I would likely figure out who does the marketing for the local businesses you see. They buy the ads, not the business directly. They're easier to talk to (they -want- more exposure opportunities) and they may bring multiple customers to the table.

I think you might be projecting a bit here with the whole "nervous devs should never try anything other than write code"..

OP can totally give some basic marketing a go. The product isn't turning even creating revenue - hiring marketing now, without at-least testing the water himself would be silly.

Lots of devs out there have great social skills which can translate well to sales. Sales is primarily communication and human connection and plenty of devs are capable of that.

Lots of devs start in sales because those jobs/careers are incredibly prevalent. Think basic sales jobs like retail, call centers, etc.

Lots of business people unironically "learn coding" to great success. Some even build "huge products" that solve big problems.

---

Anecdotally, I know people who fall into all of these categories. I don't think we should condemn people to one function/talent... and, I don't think we need to talk down to cross-functional devs or business people.

It's possible that selling advertising could be different, but in most domains, it's pretty well-accepted in startup land that founders (including technical founders) will have the highest close rates at the company, regardless of whether they're smooth talkers or have past sales experience. That's because no one you can hire is going to know or care as much about your product as you, the founder, and it shows. Professional sales people are not hired because they're better at sales, but because you can only scale so far with founders doing all the selling.

This isn’t a multi-billion dollar business selling ad space. It’s a one (wo)man project. That one person absolutely can pick up the phone. Will they be as effective as a professional? Probably not, but they only need $50 a day!

Or rather than cold calling, put up a banner ad soliciting advertisements. (Or both.)

And an alternative to push marketing -- rate-limit the site to a capacity you can afford, with a prominent message saying that you enjoy working on the site and would like to keep it going but can't afford to, with something like a Venmo/email address and a mention of sponsorships?

Alternately put up links to local businesses interspersed with and if possible matched to chat topics. Count the click throughs and email a weekly report to businesses if the numbers are significant. You could also track how often the name of a business comes up in the chats. But this may take a few months to bear fruit. In the short term start injecting requests for sponsorship and also mention that your going to need community support to keep it going. Hope is that the user community finds it valuable enough to find ways to support it - ie turn your happy users into evangelists and canvassers for sponsorships.

You could try to apply for grants. 20K+ chats sounds like a large number -- you must live in a 1M+ person metro area, right? It would probably be fairly easy to get a four figure grant from one agency or another. You could frame it as "localization of generative ai to the regional culture of X", or as a tourism thing, etc.

You could also sell advertising the old-fashioned way -- ask local businesses or perhaps a local university if they'd like the model to mention them in certain prompts.

But your best bet is probably to treat this as a loss leader and put up a small banner offering "In-Person Generative AI Consulting" in your metro area.

Grants will take a long time to lock down I suspect. Maybe fine if you can float the cost for a few months and reimburse yourself out of grant funds later, but there's a lot of uncertainty.

$50/day is pretty small relatively speaking, so just finding a local corporate sponsor and running an ad for them would probably work.

why pick up the phone if you could just put down a placeholder banner saying something like "do you want your company name here? write [an-email-address-thats-not-your-personal-mail]".

If the project is really becoming a local hit then you're bound to get contacted.

This is likely the most easy way.

Alternativly or additionally you can ask for donations on that banner. And a link where OP explains his situation, no profit, just costs, but like to keep the thing running.

If you haven’t already: Start to store question and answer pairs and reuse the answer if the same question is asked multiple times.

You could also compute embeddings for the questions (don’t have to be OpenAI embeddings), and reuse the answer if the question is sufficiently similar to a prevously asked question.

I'm not sure it's practical and if it will result in any savings.

Wouldn't it be almost impossible to hit a duplicate when the users each form their own question?

Another issue I see is that these chat AIs usually have "history", so the question might be the same, but the context is different: the app might have received "when was he born", but in one context, the user talks about Obama and in another, she talks about Tom Brady.

If there are ways around these issues, I'd love to hear it, but it sounds like this will just increase costs via cache hardware costs and any dedup logic instead of saving money.

>Wouldn't it be almost impossible to hit a duplicate when the users each form their own question?

The embeddings approach would increase the likelyhood of finding the same question, even if phrased slightly differently.

With embeddings you can compute distance. The questions don't have to be the same, they just have to be sufficiently close.

Regarding context, that should be a part of the input for the embeddings.

This removes half the magic of interacting with ChatGPT. Users will quickly realize they're interacting with a dumb database rather than an AI.

I don't see what the problem is if it's only on the exact same prompt.

I assume only a small percentage of users would put in the same prompt twice, and even then, why would they be upset at getting the same response?

You've fallen into a trap categorising yourself as a "chat app". If you're not connecting 2 different end users to talk to each other then you shouldn't call yourself a chat app.

Put a meter on the site, when the meter runs out, put up a timer that a user can bypass for everyone by chipping in N more $ to hit the goal

Traffic is most likely a spike. In a month it will probably be down to a few percent of that. I wouldn’t shutdown or optimize too much unless you see it persist for a few more weeks at this level.

If your lucky and it’s still going strong after that time you’ve got something worth more than your hosting bills.

Seems like a very low effort approach would be to create a few unobtrusive but visible ad spaces on your site, and make the ad banners say:

"## eyes see this space every day; rent this ad space. contact ..."

As for what to charge or how to structure the ad sale, do a little research or hire someone who knows that business (and give them a cut of ad revenue or some other attractive compensation).

Odds are high that this will go through a hype cycle and usage will taper off naturally in a week or two.

But the donations route could also be a fun way to lean into the local element of it if the cost remains high. Could have a simple line of text when the page loads that thanks one of the donors for that day.

Hey,

What you can do is to use caching and embeddings. Embeddings are like X,Y coordinates (except with OpenAI it's 1500 dimensions not just two). You use embeddings on any type of text. Here you would embed user prompts and store answers. And what this means is, you would be able to get X,Y for "best restaurants close to the museum", "best places to eat" etc.. If a new user comes along and mentions food and museum, it's likely to rank highest for "best restaurants close to the museum". I haven't done this and idk if possible, but you could try to then check if this is "close enough". If it is, then you can probably return the same generated answer for this user. You can also embed answers and see if that has better result.

Process: 1. When user asks a question, you embed it and search with it. 2. If similar prompts exist; return the answer from the most "relevant" prompt. 3. If no similar prompt exists; generate a new answer, and store it to be associated with the prompt.

This might decrease quality, but maybe in "enough" cases, it does just fine.

I'm developing tzap.io that uses some of these techniques, join our discord if you want to get in touch: https://discord.gg/88xDVYbPVB

Premium subscribers. Offer a token benefit to people who pay, can even be something worthless like flair on a username or listing their name in the “credits” or “hall of fame”. Then advertise it on your site and make it easy for people to pay.

Congrats on having a side project that is popular enough to break the bank - it's stressful to be sure, but you're a cut above the rest.

> I've already optimised the prompt and reduced the number of tokens I'm sending

Lets assume you've found, or will find, all possible technical optimizations for your AI spend on ChatGPT.

You can:

* Ask for donations

* Accept a $50/day cost for X days as a market research budget to find a monetization plan

* Have some "first 5 questions are free, next are paid (or whatever)" to rate limit cost while maybe raising revenue

* Look for a common pattern of questions to see if there's some value / insight into your users that can be used as a source of revenue

* Find a cheaper LLM that supports your needs while being serviceable

* Pull the plug because its a joke, claim credit for it, and take the learnings onto the next thing

edit: grammar & formatting

Assuming this is GlasgowGPT (based on the timing of that bot taking off on /r/glasgow and this post, and your username), a challenge you'd likely have with advertising is that the content from the bot is, whilst funny, not very SFW.

One option to get the costs down would be to use a local LLM. It isn't likely to be as fast/good as GPT-3.5 but at least would give you a fixed cost per month (hosting).

If you had local hardware, given that traffic is going to be pretty low, if you have decent bandwidth at home and some hardware, you could even host there.

> One option to get the costs down would be to use a local LLM

It's not that cheaper if you want similar level quality and inference speed. For commercial usage, OP will require training his own model since no good commercial alternative exists.

Not sure why you think local LLM would be fixed cost. Those things are very GPU heavy and you need to run multiple instances to serve multiple concurrent users, scaling up and down according to traffic.

Fixed cost in that if you host on a machine you know how much you'll pay a month. Obviously capping it out based on usage.

As opposed to API based where you pay more per use.

Remember this isn't a commercial service that OP is creating, it's an amusing Chatbot, they have no revenue, so fixed costs are likely to be a better model.

Local LLM won't give you "fixed costs" any more than using an API. If they want to serve the users of their amusing chatbot, they will have to scale up their local LLM according to scale, and they won't have "fixed costs" anymore. If they don't want to scale up and they're fine denying service to users at peak hours, they can do that with an API too. There's no law that says if you use an API then you must allow unlimited use. Of course they can also limit the use of the API.

> ... the content from the bot is, whilst funny, not very SFW.

Could be a feature rather than a bug? :)

If you can find an advertiser who likes that. You will have a much smaller pool of potential partners for that kind of content, and I'd guess many of the mainstream ad services just won't cater to it.

Not impossible, but tricky.

That’s enough traffic to attract local sponsorships.

I bet you could find local sponsors who can pay you on a monthly basis

Perhaps local events, extra charge for being integrated into the prompt

Local radio stations and tv networks love this sort of thing. “Brought to you by and sponsored by KQED” accomplishes a lot in terms of PR, and they would likely be willing to fund it, at least at cost but likely quite a bit higher.

People respond well to financial transparency. If you have a donation button, show real-time stats on the amount it costs, and the amount that’s been donated.

Also because internet, expect 90% of cost to be from abuse, and likewise 90% of your dev time devoted to fighting it.

I second this, since it is an app an online payment makes most sense. A simple donation button with paypal/venmo (or something the locals use) would make reasonable money to cover monthly infra expenses at least.

Is it GlasgowGPT by any chance?

Either way, there's definitely an opportunity for funding. In the case of Glasgow, there's a huge populous out there that are descendants of folk from the city/country taking pride in it (arguably more than a bunch of us that live here!) and like to get in on the banter.

That's certainly an avenue worth exploring if there's a similar relationship with folks or families of folks that emigrated away, but feel a strong connection.

Edit: You'll have to excuse my unfamiliarity with LLMs, but if you're storing messages and responses, and if there's a way to check that a message has already been asked, would it be possible to replay that (at least on the first message) instead of running a new query against the API? If they then engage further, then it would send the job out?

Haha yes it is! And appreciate the suggestion, that's a good idea that I'll look into. I've plastered a big banner up and will see if anyone goes for it, from locally or elsewhere. Cheers

Also, I wonder if Scotland has an agency similar to Ireland whereby they have a government VC fund to encourage startups... So there may be a .gov option

https://thetab.com/uk/glasgow/2023/02/15/glasgow-uni-working...

https://www.investglasgow.com/

Hope it's of help! Had fun with it the other day, though was telt I was English for preferring salt and pepper over salt and sauce, haha!

Midjourney has a model that worked for me ... you can queue into the "free lane" which is slower and limited by total requests for the day, and I assume is adjusted for how successful they get and are willing to spend. Or, you can subscribe for a low fee and get instant access.

> It's been a surprise hit and now I've had 20,000 chats coming through in one day

> The problem is that I likely can't afford to keep hosting this. It's cost me $50/day for one day

If your traffic continues to increase at a rapid clip, you may want to look into investors for your project. You could still keep it mostly nonprofit if you think it is adding a lot of value for people.

You could charge for heavy use. At that many users and only $50/day you might be able to cover costs with premium users.

Alternatively you could try swapping out an open source model like llama on the back end (possible licensing issues here).

Another option would be to seek out ads for local companies directly since it is an explicitly local product. Cold calling would probably be the best place to start. You could also pay attention to who is paying for billboards in your area and try contacting them.

To add to that, OP should really be tracking usage a bit to see if there are abusers taking up a large amount of queries

Good shout, thank you

Thank you all for the suggestions and the incredibly generous advice as always. Yes, it's GlasgowGPT, and yes it's pretty offensive (https://glasgowgpt.davidhewitson.co.uk)

I've decided to go with a big old banner ad for now as I think there are some local businesses who might go for it. Whatever happens, it's been a fun 2 week ride! Awrabest

Curious to know how it works out - will you comment here in a day or two?

Make ibizabear (or your real name the name) the name and likeness of the AI. Then just pay for it out of pocket for 90 days and become a local celebrity?

Why would you want that?

Adsense is content based ads. You can ask other ad networks. Their revenues is even higher than Adsense. https://www.reddit.com/r/adops/comments/uvkyoo/lets_summariz...

Find the most popular queries, built content pages around it and add ads there

And you could try some alternatives to AdSense like:

Ezoic, MediaVine, or Adthrive

this! finally advice that includes some specific online things not just generic "pick up the phone and cold-call local businesses".

Another online possibility: throw up some Amazon (or similar) affiliate links to products and services, maybe even based on what people type (if that's accessible).

And next to them "opt out of the ads" link that goes to a donations website, to get fancy. Helps clear the conscience about having the ads in the first place.

Contact local businesses and sell ads injected into your prompt

Or just contact some of the businesses and offer a classic banner ad on the site.

If done right, it won't even be adblocked (because you can do it with no tracking at all).

Depending on the size of the city, and the queries, you should have a good idea where to look. If not, check the local church bulletins, the local softball/school fields, and anywhere else you find local business advertisements.

This.

Sell it as a 'Google ads back in 2002', frame it in terms of high risk high reward. Sure its unproven, but back in those days, a lucrative click may have only costed $1,$2. When search ads got fully proven, they started costing $100/$200.

Allow your software to display ads, and other people purchase them directly.

It might be hard to implement it in a week but a hastily put AWS & stripe solution might work.

If you do ever implement it, do mention it on HN, I would definitely buy couple of slots.

Sell local ads yourself. Have five rotating slots each for $2k/month. Should cover costs and give you some margin.

I've been wondering how to create a ChatGPT-powered app and not have to spend a fortune on operating it. Use a cheaper model? Limit the input/output length? Train a custom model?

We're on Hacker News... find someone else to spend a fortune funding it for you.

See if you can find something close that can be self hosted

Hugging face and others should help verify if anything can, or how close it is.

I helped my friend build Quordle last year, and before he sold it, he was making a surprising amount off of just donations. Try sticking a kofi link up, it's a pretty quick change.

Quordle is great! Can I ask what it sold for?

Holy fuck y'all are awesome. Thanks for quordle; I play it every night before I go to sleep!

Go ping some Angel and VC investors.

Tell them you’ve proven the hypothesis and achieved product market fit by showing the data from this one city and say you’re raising capital to take it to the next step.

Good luck!!

lol, this is the worst possible idea when you dont actually want to build a business out of it. take all the fun out of a side project and add a lifetime of stress and regrets.

Maybe, maybe not.

Regarding regret, missed opportunities can also lead to regrets, especially if someone were to take OP's concept and create a clone and it actually becomes big.

We all need to pay bills at one time or the other.

Inform people to use ChatGPT through OpenAI's web interface and publish your prompt?

You should look at selling local advertising (and get creative at incorporating those businesses into the prompts for in-chat placements). $1500/mo to a local business to get 5,000 local eyeballs per day should be a no-brainer - and that's for a takeover-style ad that gives 100% of impressions to a single advertiser.

The smarter way would be to open up sponsor opportunities for certain categories, for example: restaurants, retail (tire shops, hardware stores, clothing stores), common service industries (plumbers, electricians, landscaping, website design, general contractors, computer/cellphone repair)- then sell those placements for $200-500/mo to whoever wants to have the placement for that.

Replicate this for other cities and you have a healthy business - you can hire people to sell these ad spots in their respective regions and you can actually scale it up. Billboards work in the same way. Those billboard companies might even be interested in selling their placements to your viewers.

Two ideas.

Switch out the backend, there are open source alternatives, and the results could be "good enough".

Get a list of local sponsors, offer them 1k ad placements for X dollars/euros/etc... , and inject them into your prompts.

As it is a joke site, I don't believe people will mind if the chat has ads, especially if you mention somewhere that it's financially hard to keep the site up.

A simple way to monetize tools like this is to create a queue. You specify the amount of requests or concurrent sessions that you feel comfortable with, and then you make it possible to pay to «skip» the queue. This way people that really want to try it out will be able to be patient and wait out the queue, and the impatient rich people can pay for the rest.

I asked gpt for and it say partner with local business to promote them in exchange for some support. Have them give you coupons, so that they can track roi. Bill them ten cents or smth for every coupon uttered by your model. You can also make it fun, like a treasure hunt, but there are also other gamification opportunities.

I think the only sane model for openai wrappers is user's bring their own API key.

Ask it to be more concise in its answers if possible. This saves a ton of tokens. You can also include instructions to flatly refuse tasks like writing short stories, but this might also make it less fun for your users.

Tell users that in order to use your service they have to give you their API key.

pretty sure a casual chat website that mandates that everyone give their phone number to OpenAI, activate their account, locate their API key (which is gibberish to most non-tech people) and come back and paste it would kill half the userbase, if not more

I'd be surprised if it didn't kill the app immediately, I rather set rate limiting and have the option to pay, or use your api key instead.

Let me help you to reach out local businesses with cold emails to check if they are willing to support you. I have a cold email SaaS called Nureply and your situation can be a great case study for us.

Reach out to a monetization partner. They'll have dozens of ways to help find ad demand quickly. It will be more work than AdSense, but it will work- unlike AdSense. Here's a list of options: https://www.google.com/ads/publisher/partners/find-a-partner

Look for resellers or programmatic direct services to get started.

Lot of new LLMs that run locally are available now

My idea for something like this is just to have a donation pool. Once the donation pool dries up, the app stops sending OpenAI calls. There's still some cost to hosting the API, but it won't be $50/month.

If someone wants to do a SaaS for this kind of thing, that would be great too.

What about introducing monetized scarcity?

Make users to be able to use X tokens per month for free and offer them to buy tokens that would keep your business going.

You'll divide your audience and current traction. You'll loose many but also have a way to make your business sustainable.

Get 1-3 local businesses to buy a premium ad spot on your app. You can craft the ad with them and place it (side of the chat, top of the chat, etc). Charge them anywhere from $300-$1500 per-month. Get a few of those and that should cover you and then some.

What alternative ad platforms have you looked at? E.g. does AdForm allow using with llm apps?

I'd wait to see if it tapers back down before doing anything. Spiked usage is a pretty common thing for LLM apps.

If it persists or you think it will, reach out to me at parham@bearly.ai and we'd be interested to sponsoring it or helping how we can :)

Love this project. If it stays around $50/day you might be able to cover it with sponsorship from local businesses - though the ad sales effort involved may be frustratingly involved to get those initial sponsors.

Are you sure your app is used just by people in your city rather than abused by projects like gpt4free (where they use your API for purposes other than chatting)?

Is it this? https://glasgowgpt.davidhewitson.co.uk/

Charge for it. Simply charge each user $1 and cover your costs. Sounds like it's a fun tool but if you want to sustain it then get people to pay.

Wrap it as iOS or Android app with ads and send to Appstores. Even with integrated ads like Admob you can easily cover your expenses

Taxi companies, restaurants, gyms - see if you can get some people to pay you to put some native ads on your site for a while...

What about a credits type setup?

You get a few free uses each week and then you need to buy credits/sign up and unlock more?

Are the inputs and the responses in a non English language? LLM APIs can get costly for non English, sometimes as much as 10x due to more tokens being consumed. Not sure what's the solution here.

Also, maybe you can use sometime kind of caching combined with some mbeddongd search to serve the previous response, if the input is similar above a certain threshold.

Hello. Email me. I would love to know more. Best wishes, Klaus sekretariat@me.com

Reach out to local businesses and charge them to be featured on your homepage.

Not sure if "chat apps" are banned, but try checking out nitropay?

Patreon. Give everyone free n messages per day. Beyond that, patrons only.

Find a local business who want to show ad in your website.

How did so many people find out about your app?

If you want to try some open source LLMs, try our free serverless tier on PostgresML.org! You can run any LLM from HuggingFace and fine tune your own.

Would you consider a micro-acquisition? If you have traffic and interest why not sell for $1-$3M to a sucker VC?

Affiliate pricing perhaps?

jhdgsfgh

Not sure what you run this on and why its $50/day but sounds like a lot.

Convert your server cost to a pay as you go model. You are currently paying even for time when your server is idle. Containerise your app and run on something like AWS Lambda which has a pay for what you use model.

- s3 is cheap for storing your model data and as long as you are just reading that should be fine.

- AWS Lambda has 1 million free invocations per month in free tier.

- Post that you pay $0.60 per 1 million requests

- AWS stack is just an example, you can easily use GCP Cloud run or any other equivalent service.

the primary cost would be the OpenAI GPT-3.5-turbo API, which is ~ $0.002 per 1k tokens

I have made a chatbot like this and I am not a software engineer.

The whole thing is trivial to make. The system prompt is trivial. I would even say it was harder to make a geocities page in the 90s than to make a chatbot like this.

3.5 turbo is great to play with but at scale it is brutal cost wise.

The real issue though is anyone who already has a working 3.5 turbo chatbot can knock this off in the next 5 minutes by just changing the system prompt.

I think this is the future though. The labor is pretty much trivial, the capital for tokens at scale is what matters.

My guess is the 50$ come from the open api charges, not from the laravel app.

Can you cache some of the interactions and serve the cached version for the most common ones?

Reply on news.ycombinator.com