OpenAI DevDay, Opening Keynote Livestream [video] | Hacker News Reader

OpenAI DevDay, Opening Keynote Livestream [video]

youtube.com

145 pointssurprisetalk116 comments

Open articleView on HN

Related ongoing thread:

New models and developer products - https://news.ycombinator.com/item?id=38166420

- Context length extended to 128k (~300 pages).

- Better memory retrieveal across a longer span of time

- 4 new APIs: DALLE-3, GPT-4-vision, TTS (speech synthesis), and Whisper V3 (speech recognition).

- GPT-4 Turbo, a more intelligent iteration, confirmed as superior to GPT-4.

- GPT-4 Turbo pricing significantly reduced, about 3 times less expensive than GPT-4. Input and output tokens are respectively 3× and 2× less expensive than GPT-4. It’s available now to all developers in preview.

- Improved JSON handling (via JSON mode) and function invocation for more sophisticated control.

- Doubled rate limits with the option to request increases in account settings.

- Built-in retrieval-augmented generation (RAG) and knowledge current as of April 2023.

- Whisper V3 to be open-sourced and added to the API suite.

- Copyright Shield initiative to cover legal fees for copyright-related issues.

- Ability to create your own, custom "GPTs".

- Assistants API and new tools (Retrieval, Code Interpreter)

This is much better than I would have thought, especially pricing. It may also be a sign that OpenAI expects competition in the near future, which would be another good news.

I just had a hope that the GPT 4 API was a little bit more cheaper

> It may also be a sign that OpenAI expects competition in the near future, which would be another good news.

Of course there is competition. OpenAI is using the Andrew Carnegie playbook of aggressively cutting costs so that the competitors have no chance to get up and running. They are cutting off the competition before they can start. I don't see evidence to believe that AI moves into a commodity business anytime soon.

So nutty. I didn't think they were going to drop such a huge bomb on the industry. We're firmly in "singularity" territory where the pace of acceleration is so fast, huge strides are being made every day / week, it feels like.

Well April 2023 is a key date here - that's presumably when they started final training a lot of these models.

We're probably able to conclude what we're seeing in public is about about 6 months behind what's happening in development. We know that in the past OpenAI relied on a lot more lag.

Point is innovation is certainly accelerating but also OpenAI's time-to-market is shrinking (/shrunk - it probably can't get much shorter).

For sure, I didn't mean to imply OpenAI's time-to-market was the main reason for my singularity comment.

Say what you want about OpenAI, and trust me I think there's a lot of material there, but you can't say they don't ship. Nobody else is even close.

The Whisper V3 model was released open source a few minutes ago: https://github.com/openai/whisper/commit/c5d42560760a05584c1...

Nice. So it's just a new checkpoint for the large model, no other changes? Should be easy to integrate, though it would be nice to have updates for the smaller models too.

Ooh, Whisper V3! I wonder how it will compare to Distil-Whisper just released by Hugging Face. I imagine it's better, although I noticed that ChatGPT's voice mode still has the classic Whisper hallucinations ("Thank you for watching!"), I wonder if it's been using Whisper v3 already.

3.5 Turbo 16k now cheaper than old 4k. 0.003c per 1k in / 0.004c per 1k out.

Fine tuned 16k: 0.003c / 0.004c

Assistants API with RAG and code interpreter (sandboxed jupyter).

> - Ability to create your own, custom "GPTs".

... and to publish them to the "GPT Store" that comes later this month. They are turning this into a new type of operating system, everyone can use the "GPT Builder" to publish apps/GPTs to that store.

OpenAI just pushed v1.0.0 of their openai Python package (with breaking changes) to PyPI, so some stuff is happening there: https://github.com/openai/openai-python/commit/08b8179a6b3e4...

Looks like the openai library is already on v1.1.1 now ?

https://pypi.org/project/openai/

Yes, v1.1.1 comes with API changes related to DevDay announcements: https://github.com/openai/openai-python/commit/baa9f07f5b503... and small improvements/fixes

It's amazing how quickly this company went from a "hacker startup" to "maximally corporate" within less than a year(!).

Just one year ago, it released a transcendental experimental product early and with no grand plan for monetization (chatGPT). Now, we have a polished keynote, with an "app store," a "developer platform," complete with your cheesy music and smiling company reps. How nice of them for sharing their revenue with us when we publish on their app store!

It's amazing and sad how quickly the hacker joy has been sucked out of this company.

> It's amazing and sad how quickly the hacker joy has been sucked out of this company.

I get the aversion to polished presentations like this, but I dunno, I work here and still feel like there's a lot of hacker joy still. You clearly aren't the audience for this presentation, but one audience doesn't exclude another.

Does anyone know if the AI granted credit to everyone at the very last minute or everyone except the five people who already had some? Because otherwise, that would be granting them twice within minutes.

It felt like an excellent example of a common sense test. A human would connect the two successive events, get that the first was planned as a fake-out or at least that the second augments the first, and know not to double-grant the lucky winners.

It granted them twice. You can see a Suraj S with the same apparent org id both times.

What are folks most excited about? I personally would love some of the code interpreter stuff as an API. But, currently trying to fix the breaking changes on their latest pypi package -- maybe before the livestream starts

robotics

All of the tooling they've done for ChatGPT folded into one tool instead of having to choose between evaluating code or browsing the web, would be a good one.

Tree history of chat messages would be great as well. Their data structures already seem to support it/made for it, so probably coming soon.

Longer context windows for their API models.

Delivered!

I'm hoping that this becomes available immediately rather than rolling out randomly. I am still waiting for the last update to hit my account.

Honestly? The JSON API they announced is the most exciting thing for me so far.

Not saying it's not useful, but you can achieve the same result consistently with just a few extra tokens in the prompt.

That's what I do today and I still occasionally see failures that are extremely annoying because there's nothing I can do but resend the request to the API.

It seems like you'd need finetuning to make it truly consistent, which is what I'm assuming they did.

Yes, so that is the outcome that I would imagine as well - basically it will never fail.

However, I have a virtually 100% success rate with the JSON feature in Langchain.

Leaked new API info, including 128K token context for GPT-4 and new batteries included "Assistant" API:

https://www.reddit.com/r/ChatGPT/comments/17p0tot/leaked_ope...

Post is deleted and as far as I know, none of the view removed Reddit post sites exist anymore like Reveddit.

had some trouble with the reddit link so here’s the google cache of it http://webcache.googleusercontent.com/search?q=cache%3Ahttps...

This was really spot on. The 128k context made me think it was fake.

Maybe I’m missing something, but their agents don’t seem to be very… agentic. It’s more like a GPT app builder plus plugins.

I wonder if they will release improvements to voice in ChatGPT. I have been working on a local AI with voice and it responds much, much faster than ChatGPT[1]. OpenAI's voices sound amazing and GPT-4 is still the best chat model, but despite that it's a lot more fun to have a conversation with reasonable latency. I expect it's only a matter of time before OpenAI does something better, I'm actually surprised they released it in such a slow state.

[1] For anyone who has a gaming PC with 12GB+ Nvidia GPU and wants to try: https://www.microsoft.com/store/apps/9NC624PBFGB7

I've been extremely happy with their generated voice!

I had GPT read something to me in the car. My wife was nonplussed with my "audiobook" until I told her that you can talk to it like Siri. Then she called it "scary" haha

The voices sound great! But the latency is too high, and it's clunky to use with voice alone, it only listens to you at specific times and you can't interrupt it with your voice. I wanted something that felt more like having a casual conversation with a real person, instead of "Siri but smarter". And I know it's possible because I built something closer to what I want.

I’m really curious how much of the speech to text is happening on device, I’m fairly sure the answer is “none”. That would have a fairly immediate performance boost. Right now it is cool as hell, just way too slow to be truly useful.

Yeah I expect the speech recognition is happening in the cloud. But that doesn't mean it necessarily has to be slow. You can stream audio with very low latency on most connections. The problem is what they do with the audio once it gets to the server. I suspect it would be far too expensive to dedicate a GPU to each customer so they need to run everything in batch mode which increases latency.

Was this just ChatGPT or a custom implementation with access to some data?

Normal ChatGPT. I asked it a question about the Soviet Union, and the response was lengthy

I'd really like to be able to use voice with the web version of ChatGPT

Yeah it's weird they haven't added that yet.

Oh nice a new open source Whisper version: https://github.com/openai/whisper/commit/c5d42560760a05584c1...

The keynote was well executed in parts and shallow in others. The video on "how ChatGPT is changing lives" is a good idea but didn't quite work for me. But they are sure heading towards Apple style marketing, with enough money and experience under the belt in future.

----

One thing that they probably didn't see/say but was a good story one could tell ...

Satya Nadella referred to initial days of OpenAI where they got some Azure credits to build their product.

A few minutes later all developers attending the DevDay got $500 in OpenAI credits.

This could have been highlighted as a similar moment in turn ... go build your own OpenAI (startup).

That would have been a powerful message I think.

Found this new official Twitter account as well, it says "coming soon"

https://twitter.com/OpenAIDevs

People that saw PARC back in the day: do you think OpenAI eventually suffers the same fate? As in, will all of these improvements end up 'stolen' by another company?

Why it could happen: natural language is a UI. A UI is only as good as what it can connect to. Browsers are OpenAI's current limit.

Why it could not happen: experts and foundation models are actually very hard to get working like presented. This is a moat.

Thoughts?

Fundamental difference is PARC was a side project for Xerox and they had no clue what they have in hand. OpenAI is AI only company and they know what they have built and how it may change the world, so today you can see all the new moats they are adding and going up the food chain.

Will they succeed or end up being MS division with "paltry" 10-20 billion payout, let's see.

Yeah you're right. Their rate limit page shows they know what they have, the business model is not an afterthought: https://platform.openai.com/account/limits

> As in, will all of these improvements end up 'stolen' by another company?

Isn't all of the technology that Open AI has popularized originally from Google?

A few thoughts: cheaper GPT-4 will only extend their lead. The biggest thing holding us back right now is not being able to get open-ai hosted in AWS with the rest of our stack. And I cant wait to see more tools built on top of these LLM tools that help us iterate on prompts and protect against regressions while pushing forward at the same time.

Increased competiton is likely more the cause of cheaper GPT-4 than infrastructure improvements.

What competition? Not being a jerk. While I see some companies slowly making progress, nobody is punching through the walls as quick as OpenAI.

You will wait long for this. MS didn't pay a billion for having others selling it too.

But click yourself an azure account, build an API and just use it.

"A billion". That's more like 10 billion MS has invested.

Nobody is asking whether Sam actually got the message?

Asking both because of his phone's black locked screen with no notifications showing, but also because doing this as an actual live demo looks like a dangerous thing to try...

I'm curious how folks approach risk on such a high-stakes live demo

He did show his screen. It was too small to read the message, but something grayish was there, popping up.

I believe that you could see someone else controlling the bottom of the screen typing the same message 10 seconds earlier, which made me think there was some demo-proofing happening.

I feel like sending a text message isn’t such a bold demo risk —– a bit like Jobs calling to order 1,000 coffees in a very public prank call with the then-new iPhone.

What seemed a tad too personal was the French engineer sharing his flight ticket and possibly the address of his AirBnB… I’m assuming those were fictional.

Did they mention anything about Azure OpenAI? The OpenAI pricing page already changed, Azure did not.

As an engineer, would you work at openai if they gave you an offer? Any reason not to?

No remote.

are there any ai startups that are remote friendly?

https://builtin.com/companies/hiring/remote/type/artificial-...

How much would they have to pay to make living in SF worthwhile for you

I'd be willing to move to SF but I would still want to be remote.

Family can't work in the US…

not gonna move to SF

I am confused - why would they need engineers since ai can do all the work, as per their claims?

No joke, I heard from someone who interviewed there that they told him he wouldn't need to know how to code for the role because chatgpt would do it for him.

That does sound like a joke but from them.

And they need him for what?

i guess someone needs to be ultimately responsible for decision making and so forth. chatgpt can do the coding but a human will confirm all business decisions

I am even more confused now. They claim ai can do everything. So that includes business decisions. Weren't there claims that someone even built a business following chatgpt advice? Someone even claimed it cured their dog.

Yes. But my understanding is that you have to be exceptional already - probably have a pHD and notoriety in some way so you won't be surprised when you get the offer anyways and already would have a strong opinion on it. Most of us don't have to worry about that decision.

Absolutly. Was already thinking about trying to get closer to the action but i'm only doing platform engineering based on k8s and live in germany.

there are plenty of people able to do what im able to do. They will not just move me over to SF just to have some MLOps/Platform expert.

The crypto stuff Sam A. is doing is garbage but he is rich enough, it has less of a 'get rich scheme' i believe. That would be otherwise a no go

Having OpenAI on your resume will definitely open a lot of doors. It's also a spectacular feeling to be at the forefront of technological shifts.

Note, ElevenLabs is $0.24 per 1K characters while OpenAI TTS is $0.03 per 1K characters. That's a pretty big difference. Elevenlabs still has voice copying but for many use-cases it's no longer competitive.

GPT-4 Turbo vision is much cheaper than I expected. A 768*768 px image costs just $0.00765 to input. That's practical to replace more specialized computer vision models for many use-cases.

It's also fake news because no one has access to it yet even though the docs say it's available

Goodbye all the startups trying to build LLMs/fine tuning/agents. OpenAI is so dominant all of these startups are solving sub problems of the generalized algorithms of OpenAI. Good for them for crushing the competition, finally a competent company broke the monopolies of Google/FB. All those highly paid researchers at Google/FB cannot release models that are remotely close. The emperor has no clothes

How does this actually kill fine tuning? I don't think OpenAI can kill open models. People are not going to to stop wanting to keep their data in-house or at least more under their own control. And there are massive companies behind the open models.

they just announced custom models for enterprise

That will put a dent in it for sure. But still I feel like fine tuning is the most resilient aspect of it for open models.

OK but their secret sauce is out there. There are other foundational models. Curious to see if they can create lock-in or network effects with their APIs and such

isnt this the basis of the conspiracy theory like ideas around asking for AI regulation from the goverment

Realistically nothing performs as well as GPT-4 at the moment. We've seen others make advances but it's still best in class.

at the same time it's very hard to monetize GPT4 because the cost are so high

my uneducated guess is they may aim to get major share of income via selling ChatGPT Enterprise to Fortune 500 list, where cost of running probably is not much of a concern

The irony of this all is that ai has successfully replaced ai workers, and will continue doing so.

There's thousands of niches where you have to really know the domain, which OpenAI will never bother doing. If startup is smart and has a moat, it's going to be OK.

I wonder how tokens will be applied to images...

IIRC there is a token > image fixed calculation.

There is a Vision pricing calculator now on https://openai.com/pricing

starts in ~1 hour at 10AM Pacific

300 pages of context. Way ahead.

Support for streaming in assistant responses is not there? Is there a workaround?

> We’re gathering developers from around the world

Where exactly were they "gathering"? I'm squarely in AI research at a major co and I never received anything about it.

It's in San Francisco. It was announced Sept. 6th on their blog: https://openai.com/blog/announcing-openai-devday

They didn't do a good job of "gathering" then. Real AI researchers don't have time to keep reading blogs. They should have announced it first and foremost on arXiv or inside the GPT API documentation if they wanted real researchers.

Are they even a real researcher, if an AI agent hasn't scavenged the Internet for relevant events and planned the trip. Check, maybe your virtual AI persona is attending it.

Well this is mostly for developers and not researchers. Its about using their system.

Lame the comments are turned off

Also not a bit surprised

Chat GPT turbo looks promising.

Underwhelming, in the sense that it s mostly predictable stuff that others are making, too.

they are making an app platform (GPTs) that might make some money. But more likely apple and google will make their own platforms.

I don't believe "building with language" is possible, unless you are already a programmer , which defeats the purpose. Most developers can attest to this, people don't actually know what they want, nor how to express it clearly.

I personally was hoping for a 129k context window

I thought the Zapier demo was pretty cool.

(GPT store and Assistant API too)

Not many people announcing a state-of-the art LLM for a third of the price these days

i m only trying to push them harder

Apple has to first get Siri working at least.

OpenAI is going to show us how closed they are and who’s startup that they are going to kill.

First it was Jasper, then the Chat with PDF apps and now it will be the custom chatbot services like Chatbase.

The second biggest lie by OpenAI after its now contradictory namesake, is that it competes against its own partners after denying that they don’t. [0]

[0] https://www.theinformation.com/articles/the-best-little-unic...

Obviously you're gonna get eaten if you just add some fluff on top of their API and it ends up popular. People are smart enough to realise this right?

> People are smart enough to realise this right?

Hardly the hundreds of so-called 'AI startups' crying for VC money over a wrapper AI product aren't smart enough to realize how OpenAI is directly competing against them.

they haven't even started tbh, i dont think people can pretend to build a company just because they found a cool prompt. Anyway, let's see ...

OpenAI has every right to "kill" any startup that is just a wrapper around their API. You aren't obligated to not compete with people using your own product to make money.

The IT worlds is basically integration and / or very niche custom software development.

There are plenty of countries around the world, special use cases, etc. and enough work for the normal it world to go around.

For my startup, im pretty sure the finetuning is a huge win even if we 'just use' openai because openai doesn't have the information we have and i'm pretty sure they also don't care for our niche (right now, and AGI in 20 years, i will probably no longer be needed one way or the other ;P)

Reply on news.ycombinator.com