The OpenAI Keynote | Hacker News Reader

The OpenAI Keynote

stratechery.com

120 pointstodsacerdoti75 comments

GPTs simultaneously feel very useful and a dead-end. Personally I can immediately see how it can make me more productive with specialized agents. However since its so easy to make there will be thousands of them and discoverability will be very hard. Probably will be like Chrome extensions.

This has alot of killer features for me as a low level developer. I gave it all the intel programming documentation for x64, and it's able to correctly reference and solve problems GPT-4 cannot or hesitated to.

I imagine anyone else with a case like this where the docs are extremely verbose and hard to read will benefit.

I wonder if it would be good at building libraries and emulators based on datasheets.

When I've played around with microcontrollers, dealing with external components requires deep studying of the datasheet, even for the simplest of components like a temperature sensor - if there's a library I would always use that, but of course there isn't for everything. Maybe GPT can help here...

Did you do anything else special in getting it to correctly reference the documentation?

Now that you brought this up, I could really use this ability with some 600 pages of documentation I have.

This is the prompt I use. I've been refining it a little to make sure it quotes the manual. But it's a good chunk of the time.

"You are an aimed at helping Systems Programmers, Low Level developers, Security Researchers and Kernel programmers. You will be an expert in the Intel64, x86 family of processors. You will use your knowledge to answer questions. You will also use the given manual and always quote the relevant sections and materials. You also also an expert in hypervisors and hypervisor programming from your knowledge and the files provided. Quote sections and relevant information from files provided."

But just like Chrome extensions, there are the 20 or so that everyone knows about, and then all the rest. There will be a few good ones trained on some very unique and useful data and everyone will want to use them.

Why would there be thousands of them? Wasn't the last GPT where the research actually was shared, GPT3? Hard to imagine that OpenAI suddenly will start releasing models more often than what they have been doing so far.

Edit: That's what I get for reading the comments before the article... "GPTs" is actually a new thing they introduced, where you can create your own "GPT" somehow, not just pluralization of GPT as we've know it.

That's some really messed up naming if anything. But regardless, parents comment makes a lot more sense now.

I get the discoverability point; even without this new announcement from openAI, there have been so many posts to HN regarding new AI tools. TikTok is full of videos of people showcasing the "five best new productivity websites" or "four awesome design tools", or "AI hacks", or whatever. It seems there's a new website or tool offering a wrapper around either GPT3/4 or an open source model every day.

Even better, GPT writing the "best of" website, after writing the GPT variant it praises! Over and over and over!

congrats for reading the article;)

I agree the naming is a bit unfortunate and one needs to differentiate between custom GPT assistants and custom trained GPT models only available for enterprise. then there's also also API based fine tuning and assistants which are a yet completely different thing. also it seems that there are no longer any plug-ins, So custom GPT with custom actions/functions filled that role.

( I'm sure Microsoft with their brilliant naming had a good influence :-)/sarcasm )

The problem with GPTs-as-products is that differentiation is almost impossibly opaque. How many “amazing prompts” have you tried from Twitter or Reddit only to find they cause the model to behave in slightly undesirable ways in certain scenarios. Or perhaps the output is basically the same as you would get from your own custom “do this thing” instruction? Beyond giving access to the latest documentation or meaningful performance leaps (like chain of thought prompting), it’s simply too hard to informally A/B test them against each other and decide if one is actually better than another. I know for me I’ve almost given up on custom instructions because the code that comes out of my most intricate system prompt is only superficially better.

The genius will be in delivering options, that's what I'd use it for. Say you run an etsy shop and you want it to go out, get the top bestselling listings for x keyword, you want it to examine each image using vision, every keyword in title, description, tags, etc -- and then give you ten titles, and then after you pick a title it'll give you options to build a description.

If you need something exact and on target, well that's going to be a problem - if you just need very targeted highly filtered and maximized choices to solve a problem - that's where this shines.

There's a "Proffesor Synapse" prompt that has been floating around for awhile, people share it and seem to find it useful. So there's some market proof that people appreciate and will gravitate towards popular prompts.

Professor Synapse is mind-blowing to watch in action. Link to source: https://github.com/ProfSynapse/Synapse_CoR

Okay, it may be my age showing, but that README there is entirely incomprehensible to me. The prompt.txt isn't any better, it feels like there's a software suite (or a GPT call layer) missing here?

Paste it into ChatGPT and try it out. I didn't quite understand it until I did that and it's pretty cool.

I tried this and it seemed to break ChatGPT, it blurted out something which made no sense and then offered to regenerate it. How is it supposed to work?

Maybe it only works with gpt-4? Is there a sample transcript of what it can do anywhere, I still don't quite understand what the end-goal is supposed to be. It seems to simulate mixture of experts technique, but I thought gpt-4 already did that.

type /start

Thank you, this worked! Quite an interesting prompt indeed...

> However since its so easy to make there will be thousands of them and discoverability will be very hard.

What you described really is a dead end for the majority of their authors, but not for the owner of the app store (OpenAI).

Hopefully someone will make a GPT discovery GPT.

I was already looking into doing it but I couldn't find any webpage where custom GPTs are in anyway listed or iteratable. this problem will also be solved by openai likely before next year, I assume

I think it will be God GPTs that will do anything and everything by automatically finding specialized agents and "employing" them on your behalf.

also known as copilot on windows and M365

I'd expect that the current iteration doesn't gain much traction. Just like the much vaunted plugins.

It seems like OpenAI is just trying to get the idea of agents out there, and hoping that at some point in the future they figure out how to make GPTs actually agentic.

It is agentic allright, just that its autonomy rate is close to zero. Usually an agent will get stuck and need human intervention.

How are they specialized? For anyone gpts I can get the same output from ChatGPT

Assuming GPTs are equivalent to the Assistant API (seems descriptively like two views of the same coin, but the info doesn't quite seem complete on that), they can rely on exclusive data resources and tools provided by the implementor that are not public ChatGPT plugins, and therefore can have distinct, and to the extent those data and tools are or leverage proprietary resources, not otherwise reproducible behavior.

I've started using a tool similar to "GPTs" recently, and the quality is definitely higher.

I believe this has to do with generalization — you're encouraged to make your GPTs as specific as possible: only give them access to what they need, and iterate on how they can do it better, both through giving feedback and improving your prompt. This is how you can refine them to be great at a certain task.

The more generalized they are, the worse they'll be at the specific task — this will likely be true until we create AGI, and this is how all these businesses with refined purpose-built models exist and create value.

Just had fun summarising the keynote using one of the new GPT-4 models (the 128K token model).

It produced a pretty impressive blog post - not the kind of thing I'd normally post, but certainly good enough for most "tech" publications... https://www.atomic14.com/2023/11/07/open-ai-dev-day.html

> not the kind of thing I'd normally post, but certainly good enough ...

What would you change to make it better than "good enough"?

Well, the summary contains a lot of flowery, say-nothing language like "promising to reshape our world with unprecedented possibilities" and "groundbreaking developments that are set to further empower developers and users alike" where a human writing a summary would use less words.

It retains video-only terms like "Join us as we explore" which would usually be omitted in a text summary.

It defines the well-known acronym 'AI' then doesn't bother to define 'GPT'

It has a section entitled "How GPTs Work" which doesn't say anything about how GPTs work.

And it consistently calls the "Assistants API" the "Assistance API"

Aha super meta. How did you transcribe the video? Whisper?

The article says it used the transcript (since it's on YouTube I assume they mean the YouTube transcript).

What prompt did you use? (Besides the transcript itself)

I often wonder if the future of OpenAI is using GPT as a meta interface for the rest of their models. Interpret and extrapolate human intent and then select a more appropriate model to execute requests and return the asks in a human understandable format.

It seems that's indeed the direction. And it makes sense. When saying that their overarching goal is building _an AGI_ there's an implication that there is really only one model / interface / entry point, and that how it does its magic (one neural network, several similar neural networks, an ensemble of various neural networks of different architecture and training + some glue code to tie them together) is an implementation detail and not something we should concern ourselves with as consumers.

Isn't that sort of how GPT-4 already works? It's a Mixture of Experts architecture that bundles together a bunch of models and selects one for each query.

do you have a source for that info? (not accusing, i've heard this before, i just want to see it from openai) i'm curious because i believed that myself but i'm not sure how some of the developments from this release would work in that architecture

Building specialized agents is a great way to achieve what humans do through teams and organizations. For individual developers the benefits are dubious, but for ChatGPT, it’s obvious that the end game is letting developers/people build and humans curate (through downloads, ratings, usage) the most interesting agents and then setting ChatGPT up to be grand central to use that knowledge to hire the best agent/teams for a job (or putting them against each other and presenting pros/cons).

To that end, the ease of building is key: lots of folks would be happy serving their little niche with no monetary gain (see open source software) which, for non-developers has never really been possible outside of social media posts/influencers.

Maybe it pays off, maybe it doesn’t but generally the strategy feels sound.

You have to pay to make these custom agent's. In a way it feels like people are paying to work.

You have to pay to make them public? Can you expand? I haven't seen this mentioned anywhere yet the and I'm not yet able to make my own.

The interesting point/question in this is whether Open AI will invest in hardware to build a ChatGPT interface in the future that allows for less friction of use than just “being an app on an IPhone”. I think it’s amazing and would be so cool to see a new device people use that has function that hasn’t been totally defined yet. The IPhone is so mature it’s time for the next thing!

Even just the free iPhone app with free-tier GPT3 is better than Apple offers with Siri (low bar, but still) in many regards. Imagine that built out, and given system-level access to your phone tools, emails, calendar, and your smart-home tech (as Siri has) - there's so much potential there.

(I'm not sure I like the idea, as at this point I have no reason to trust OpenAI with my personal data, but it's undeniable that the potential is there.)

Have you seen the tab being worked upon by Avi Schiffman. Seems like a interesting concept.

https://twitter.com/AviSchiffmann/status/1708439854005321954

Seems like an open-source hardware concept device to address privacy concerns would also be quite successful. Could be kickstarted.

Didn't they hire Jony Ive? I suspect something is in the pipeline...

https://www.ft.com/content/4c64ffc1-f57b-4e22-a4a5-f9f90a741...

OpenAI and Jony Ive in talks to raise $1bn from SoftBank for AI device venture

ChatGPT creator in early discussions to create the ‘iPhone of artificial intelligence’

Sam Altman, OpenAI’s chief, has tapped Ive’s company LoveFrom, which the designer founded when he left Apple in 2019, to develop the ChatGPT creator’s first consumer device, according to three people familiar with the plan.

I haven't though this through, but this seems to be a good opportunity for MS to get back in consumer devices game.

They can expand on Surface/XBox brands to add Echo like ecosystem. Till now all smart devices/assistants aren't smart enough to be really useful beyond playing music and setting timer. GPT can change that, now the device can actually be useful. They have the momentum right now to build whole ecosystem around it. With latest increase in context length, complete user history can kept in sync across devices. Not sure if it is economically feasible.

If the hardware is just a microphone and speaker that calls an API, why does the world need a new device? Every phone, tablet, laptop, game console, smart watch, smart speaker, smart doorbell, earbud and TV can do this already.

To justify adding yet another new piece of hardware, it would need to be something different. What would that be, exactly?

The obvious one would be a dedicated device that just observes the world around you (sound, video, etc) and provides some kind of ambient intelligence to support your day in context. This cannot be done on a smartphone and none of the other devices move around with you. The big question is whether there is a killer use case for something like this.

because absurdly the hardware is locked behind iron software gates.

i'm not aware of a device with a high-quality microphone where one can set their own custom Siri

The world is ready for smart jewelry worn around the neck, similar to Tab

Perhaps Microsoft could get back into cell phone space with a phone built from the ground up with an OpenAI OS of sorts. A phone designed with OpenAI in mind first.

I think the play is around intelligent meeting devices. A single device that connects to your internal data and becomes the ultimate meeting sidekick.

I’m thoroughly surprised, MS hasn’t built an Alexa like device but with chatgpt.

I know current events are a struggle but that’s the easy part of smart speakers.

Not exciting enough for a new consumer device.

OpenAI needs to leapfrog into robotics, that’s the only hardware that makes sense.

Was thinking the same yesterday.

Microsoft will make it cheaper but I'd also be very interested in a generic echo like smart home device that can be plugged into any LLM.

I have a few echos, and more google home devices, not a single one is plugged in anymore because I got tired of them not being capable enough or google doing stuff like adding "oh and by the way, if you ever want to do x you can ask me y".

No way to turn that shit off.

one of the issue is latency because users don’t wait for anything beyond near real-time.

The Universal Interface currently has a huge problem in the form of "prompt injection" where currently we have no principled way to separate instructions and data, e.g. https://embracethered.com/blog/posts/2023/google-bard-data-e...

Not to say this is the only problem, but this one is at least moderately well known and a clear and visible blocker.

I assume we're unlikely to truly mitigate anything with LLMs we can't mitigate in standard english.

This would be easy enough to fix by having a modifier added to each token embedding vector to say "this is an instruction token" or "this is a data token".

It seems intuitively that that should work to mitigate some prompt injection, but even if it would, it’s quite possible that a good chunk of the power of modern models comes from not making that distinction, and the model in effect determining what kind of weight to give data by context. We might make much safer but also far less useful models by doing that.

It also isn’t likely to eliminate all bad data –> undesirable actions within the scope of what is enabled by the tools the model has access to problems. Because as long as the intent is for behavior to depend on data, you can manipulate behavior by manipulating data in unexpected ways.

LLM complexity depends on its use case - if it’s a chat window, this kind of dichotomy wouldnt work (Unless I am missing something in this thread)

The model will be treating all text as input, whether it’s tagged as instruct or data. You could find a way to abuse either entry point.

I agree this is a natural thing to try. Given I have not seen anyone report success with this, I lean towards thinking this doesn't actually work reliably (or the entire community is asleep at the wheel).

Recent and related. Others?

New models and developer products - https://news.ycombinator.com/item?id=38166420 - Nov 2023 (533 comments)

GPTs: Custom versions of ChatGPT - https://news.ycombinator.com/item?id=38166431 - Nov 2023 (307 comments)

OpenAI releases Whisper v3, new generation open source ASR model - https://news.ycombinator.com/item?id=38166965 - Nov 2023 (49 comments)

OpenAI DevDay, Opening Keynote Livestream [video] - https://news.ycombinator.com/item?id=38165090 - Nov 2023 (115 comments)

Hmmm did he confuse general pretrained transformers with generative pretrained transformers?

It would be great if there was some kind of open decentralized protocol for agents as an alternative to being locked in to a few particular platforms.

It's basically messaging app bots. As far as I know, users can't communicate across messenger platforms.

Maybe we will end up with messenger convergence/bundling products merging with agent services.

> It would be great if there was some kind of open decentralized protocol for agents as an alternative to being locked in to a few particular platforms.

If you're expecting a company with "Open" in it's name to want to make anything close to "Open", you've lost.

The new trend is to make the company name as different as possible from what you want. OpenAI is all about vendor lock-in, which should be clear from the name, so everything they do will try to lock you further into their ecosystem.

There is going to be an immense amount of experimentation before someone figures out the right way of building the experience.

Enter: everyone copies the UX

Result: Brand, marketing, distribution and becoming top of mind will determine the winner.

All of this, if OpenAI does not kill you in the meantime

I feel like we are very close to soft AGI.. gpt as language understanding and able to create and execute arbitrary programs, and maybe just a small portion of magic to keep some state and purposes.

I imagine a world where these kinds of models are a stepping stone to personalized autonomous assistants. Where every person is able to have an agent that learns how to help the user and is embedded into their personal technology. The assistant can be delegated to act on behalf of the user, dealing with common tasks in the user’s personal and work life. I imagine sending my AI assistant to work meetings on my behalf to interact with other AI assistants, and making decisions for me based on its learned delegated behaviors via shadowing and training. In terms of managing teams I already explicitly delegate decisions to the people that report to me, and I trust them to inform me of explicit types of things they believe I need to know, otherwise I let them just get on with the job. I want the same from my AI assistant. Maybe this kind of assistant takes the form of a orchestrator of many GPT-esque models dedicated to specific tasks, but in the end, I want my own interactive AI autonomous agent that can be federated to act on my behalf when given permission to do so. This brings many challenges with regards to topics such as GDPR and corporate BYOD policies in the enterprise but it’s nothing we can’t solve in the long term.

Until I get that, I’d like a tool that evaluates everyone’s diary in an organization and optimally reorganizes everyone’s calendars to make maximum use of time. Microsoft are optimally positioned for such a tool.

OpenAI CEO Demos How Its GPTs Can Eliminate Mark Zuckerberg's K-12 CS Teaching Position

https://slashdot.org/submission/17304520/openai-demos-how-it...

> Via claude.ai: > What would you say are the 2-3 key insights

Here are the 2-3 key insights I would highlight from the summary:

1. OpenAI is evolving to be more of a product company than just an AI research lab, with iterative improvements to their API and tools to make them more usable and accessible. This includes pricing changes and ease-of-use focused updates like removing the ChatGPT model picker.

2. There is a tension between modular plugin models for integrating tools with ChatGPT versus pursuing further integration directly into a "Universal Interface". Consumers tend to prefer integrated solutions for convenience, so OpenAI may continue down that path.

3. To fully realize frictionless AI, new consumer hardware interfaces may emerge as alternatives to being "just an app" on existing devices. OpenAI or others could pursue this route despite the challenges of competing with Apple and Google.

Reply on news.ycombinator.com