Hacker News Reader

Comment by leroman | Hacker News Reader

GPT-4 is already hugely useful. If they are able to lower the cost of GPT-4 further, say 5x or 10x that in itself would be useful and huge.

View on HN

IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep with implementation on some of these models.

If you could both speed up inference time and reduce cost of GPT to 3.5 levels, there are incredible amount of possibilities that open up that could actually help solve a lot of problems have had with trying to interface software to the real world (robotics).

4-Turbo is actually pretty amazing but its still a tad pricey for some tasks. Altman made a comment on Lex's podcast about compute and I think its true, essentially the world does not understand how much compute will be desired as the price of these things go down.

Agreed, the chat interface is quite limited compared to what's possible with the API. I recently wrote a script for my job at a publishing company that automatically writes social media promos for every new article published on our site by crawling the sitemap on a cron job. Still putting the finishing touches on it, but hoping to implement it soon.

But with only the chat interface, our social media manager would have to find every new article in every discrete vertical on our site, copy all the URLs to a spreadsheet, then prompt ChatGPT for every article, wait for it to respond, then copy that response into the spreadsheet...

> If you could both speed up inference time and reduce cost of GPT to 3.5 levels

Presumably this would be a major part of the big material improvement we'd need to see.

Tangentially, I lowered my personal cost by more than 3x, and was able to share GPT-4 with my friends and family.

I installed LibreChat on a Linode droplet, put $10 on my OpenAI account for API usage, and cancelled my ChatGPT subscription. Since neither I, nor my friends and family use ChatGPT a ton, I let them sign up on my server and the API costs are under $1 so far.

I actually think this is the way. ChatGPT honestly is not that great. I would be happy to pay per token for ChatGPT and be able to have a little more control over the system messages.

If you regularly use GPT4 for coding this isn't so viable, it gets pricey with the amount of context you need and the default GPT4 chatbot is decent with code. The GPT API is best hit with questions that don't have a lot of setup, can leverage its excellent general knowledge, and can be answered succinctly.

I find it pretty viable for myself. I use this kind of setup within the IDE using API keys. I will flip between 3.5 and 4. The cost of 4 is minimal compared to my time.

This sounds like a great idea and relevant to my current goals. I haven’t used LibreChat; how are you handling authentication?

I just let them sign up with an email and password, then I turned off the ability to sign up once everyone had done that. LibreChat does support other options [0].

I configured the yaml file to use my own OpenAI API key for all users.

[0] https://docs.librechat.ai/install/configuration/user_auth_sy...

edit: oh, the only other configuration change that I made was to set the default OpenAI model to gpt-4-0125-preview.

Got it, thanks! I’ve been wanting to do some API work anyway, and this seems like a great way to consolidate everything.

Reply on news.ycombinator.com