Hacker News Reader

Comment by somewhereoutth | Hacker News Reader

Given that GPT-4 was released a year ago now, and given that GPT-4 is in many ways 'not there yet', GPT-5 will need to show a big material improvement. Anything less will signal the end of the road for LLMs.

View on HN

What do you mean by "not there yet"?

I regularly use GPT-4 for things like "write exhaustive unit tests for the following function" and it gives fantastic results. They usually need fleshed out a little, sure, but in a few seconds I get something that would have take about $45 in billable human hours.

I personally cannot wait to see the productivity gains that come from GPT-5.

> They usually need fleshed out a little, sure,...

> I personally cannot wait...

My point exactly.

Would you be disappointed if someone came out with a magical new paintbrush that could paint any color in oil, acrylic or watercolor without actual paint because the paintbrush didn't levitate itself and do the painting for you?

You can get red in the face at AI the disembodied futurist concept all you want, but the tools are the real deal - people are using image generation and chatbots today to be 50%+ more productive and produce better work. These tools require skill to use, they're not just plug-n-play, but that skill pays huge dividends, and if you don't learn it you're going to be left so far behind.

> people are using image generation and chatbots today to be 50%+ more productive and produce better work.

I don't think we have evidence to support that.

https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id...

From the abstract, it is suggested that for some tasks AI was a benefit, but for others it actually made things worse.

So can we know which are which, and what are the relative value adds of these task groups? Where does an overall cost benefit analysis land, bearing in mind the high costs of AI.

Easy, AI is a tool like a hammer. Know when to use a hammer, vs when to use a saw or a drill? Models are a more abstract tool but if you spend time with them you'll build intuition quickly enough.

GPT-4 is already hugely useful. If they are able to lower the cost of GPT-4 further, say 5x or 10x that in itself would be useful and huge.

IMO that's the key there. I think a lot of people who make comments like the GP have either just been using the chat interfaces or perhaps have not gone deep with implementation on some of these models.

If you could both speed up inference time and reduce cost of GPT to 3.5 levels, there are incredible amount of possibilities that open up that could actually help solve a lot of problems have had with trying to interface software to the real world (robotics).

4-Turbo is actually pretty amazing but its still a tad pricey for some tasks. Altman made a comment on Lex's podcast about compute and I think its true, essentially the world does not understand how much compute will be desired as the price of these things go down.

Agreed, the chat interface is quite limited compared to what's possible with the API. I recently wrote a script for my job at a publishing company that automatically writes social media promos for every new article published on our site by crawling the sitemap on a cron job. Still putting the finishing touches on it, but hoping to implement it soon.

But with only the chat interface, our social media manager would have to find every new article in every discrete vertical on our site, copy all the URLs to a spreadsheet, then prompt ChatGPT for every article, wait for it to respond, then copy that response into the spreadsheet...

> If you could both speed up inference time and reduce cost of GPT to 3.5 levels

Presumably this would be a major part of the big material improvement we'd need to see.

Tangentially, I lowered my personal cost by more than 3x, and was able to share GPT-4 with my friends and family.

I installed LibreChat on a Linode droplet, put $10 on my OpenAI account for API usage, and cancelled my ChatGPT subscription. Since neither I, nor my friends and family use ChatGPT a ton, I let them sign up on my server and the API costs are under $1 so far.

I actually think this is the way. ChatGPT honestly is not that great. I would be happy to pay per token for ChatGPT and be able to have a little more control over the system messages.

If you regularly use GPT4 for coding this isn't so viable, it gets pricey with the amount of context you need and the default GPT4 chatbot is decent with code. The GPT API is best hit with questions that don't have a lot of setup, can leverage its excellent general knowledge, and can be answered succinctly.

I find it pretty viable for myself. I use this kind of setup within the IDE using API keys. I will flip between 3.5 and 4. The cost of 4 is minimal compared to my time.

This sounds like a great idea and relevant to my current goals. I haven’t used LibreChat; how are you handling authentication?

I just let them sign up with an email and password, then I turned off the ability to sign up once everyone had done that. LibreChat does support other options [0].

I configured the yaml file to use my own OpenAI API key for all users.

[0] https://docs.librechat.ai/install/configuration/user_auth_sy...

edit: oh, the only other configuration change that I made was to set the default OpenAI model to gpt-4-0125-preview.

Got it, thanks! I’ve been wanting to do some API work anyway, and this seems like a great way to consolidate everything.

Reply on news.ycombinator.com