Comment by qqtt | Hacker News Reader

Comment by qqtt | Hacker News Reader

Parent Full thread

ChatGPT is cool and novel, but FAANG's requirements for ML/AI go far beyond what ChatGPT provides as a product. ChatGPT is good at answering questions based on an older data set. FAANG typically requires up to date real time inference for huge rapidly changing data sets.

Working on the practical side of ML/AI at FAANG, you will probably be working with some combination of feature stores, training platforms, inference engines, and so on - all attempting to optimize inference and models for specific use cases - largely ranking - which ads to show which customers based on feature store attributes, which shows to show which customers - all these ranking problems exist orthogonal to ChatGPT, which is using relatively stale datasets to answer knowledge based questions.

The scaling problems for AI/ML for productionizing these ranking models from training to inference is a huge scaling problem. ChatGPT hasn't really come close to solving it in a general way (and also solves a different class of problems).

Agreed. For my job maintaining real-time models with high business value to be disrupted by a chatbot, an LLM would have to be able to plug into our entire data ecosystem and yield insights in realtime. The backend engineering work required to facilitate this will be immense, and if the answer to that is "an LLM will create a new backend data architecture required to support the front-end prompt systems", then... well, suffice to say I can't see that happening overnight. It will require several major iterative and unpredictable pivots to re-envisage what exactly engineers are doing at the company.

For the time being, I expect LLMs to start creeping their tendrils into various workflows where the underlying engineering work is light but the rate of this will be limited by the slow adaptability of the humans that are not yet completely disposable. The "low hanging fruit" is obvious, but EVPs who are asking "why can't we just replace our whole web experience with a chatbot interface?" may end up causing weird overcorrections among their subordinates.

Isn't this as straightforward as semantic search over an embedded corpus ? Unless i'm missing something, i don't think the backend engineering would take much

I think generating useful embeddings off of a lot of realtime data flows (eg. user clickstream data) is in fact fairly difficult. Furthermore, if you had such embeddings it's unclear if an LLM would add value to whatever inference you're trying to do. If the LLM is not only be used for inference but to actually retrieve data ("find and summarize the clickstream history of his user") then I would not expect this to be doable in realtime.

I can tell you that we have applied teams working on open problems, which can be solved out of the box with ChatGPT. Its a huge deal

What's an example of such a problem?

What kind of problems?

ChatGPT is human level intelligence, it’s not just novel and cool, it’s the thing. Remember, GPT-4 training was finished 6 months ago. Listen to people at OpenAI, their concern is: disruption to the world, UBI, getting people used to superintelligence as part of our world. I think they have quite a few things in the pipeline.

So yes ads optimisation/recommendations still need to be reliable for the time being, but for how long?

GPT-4 is not human level intelligence, nor is is above or below. It’s quite a different kind of intelligence not entirely comparable to humans. That’s probably why we’re moving the AGI goalpost; we visualize AGI as a robot human, but these machines may simply be founded on too different principles to ever receive that honor.

I think it’s mostly different because they crippled the public version for now: no internet access, everything is done in one pass. In our mind we get an idea, we inspect it, we try different variations, we simulate how it will be perceive(consciousness). In this way we iterate before putting the information out. This is not difficult and is getting added on to it externally.

Chat GPT is just to get us used to the idea, it’s the toy version.

This is pure speculation, in a very implausible direction.

I would be interested to know which part you feel is implausible, to me it seems inevitable

You have a language model produce an outline with steps and then recursively set agents to consume and iterate on a task until another language model finds the results satisfies the specification.

This includes interactions with the real world (via instructions executed over an API) and using the success of those interactions for reinforcement learning on the model.

Many are building such things.

But I think they are mostly pointless as OpenAI is so far ahead of everyone external it’s not even funny. Most externals things with the API will be obsolete in a few months.

They had GPT4 6 months ago or more! They have access to the full model without crippling. They (for sure) have larger, more powerful models that are not cost effective/safe to release to the public.

Now they have a new data flyweel with people asking millions of questions daily.

Put your speculation hat on and listen attentively to the interviews of Sam Altman and Ilya Sutskever.

You will see were their minds go: UBI, safety, world disruption, etc.

Reply on news.ycombinator.com