When Will the GenAI Bubble Burst? | Hacker News Reader

When Will the GenAI Bubble Burst?

garymarcus.substack.com

79 pointsisaacfrond142 comments

GPT is great and useful.

But it reminds me of young kids who are verbally strong. Some 5-year-olds are amazing talkers, giving you the impression they have a very sophisticated intelligence at first glance. But the more time you spend with them, you realize they don't understand but imitate.

Although the kids will grow up and understanding naturally develops, many people seem to assume LLMs will take the same path, which seems less likely.

Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that.

Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing.

GenAI will continue to have its purposes, some applications will become more useful with more specific models and more developed reinforcement models on top of it. So there is definitely a commercial use for those. But it will keep getting stuck on the accuracy part, the same place where AGI development is stuck.

That phenomenon goes a lot broader than young kids. Verbal fluency and rhetoric has almost become a sort of personal marketing technique. Sounds impressive at first, but doesn't necessarily correspond to anything of substance behind it. My tentative hypothesis is that it's a symptom of the "optimization" of educational content on the internet.

Various times I hear someone described as "an idiot's idea of a smart person".

I can vibe with that description of GenAI: it does all the things that, as a kid, I regarded as synonymous with intelligent. And yet, now we have it, it's at the level of an intern… an intern in a huge number of fields all at once, so very general and still useful, but an intern nonetheless.

There's a whole lot of people on forums like HN that like to write in this intellectual style with a bit more "Queens English" sounding idioms and some $10 words thrown in. They increasingly sound like they're Generative AI.

Maybe a useful side effect of GenAI might be that people will start speaking more bluntly so that they sound more human.

The AI is mimicking the humans, so if the models had been trained predominantly on nice efficient (blunt) German culture, it would be the other way around.

("Queen's English" and "$10 words", in the same list?)

The style reminds me of high school essays. More advanced writers deviate from that to make it entertaining.

The style of ChatGPT? I've not read many high school essays, but I can easily believe that. Maximum length of generated output is about the length of a 15-21 y.o. student essay.

Yeah I was thinking of ChatGPT 4 but they all seem to have quite a similar style.

> Which is ridiculous, it's just an incredible token predictor.

I'm not sure I'm not just a mediocre token predictor.

I've never met anyone who didn't just say the next word in a sequence. So far nobody's been able to produce a complete sentence in one go!

Stop.

I'm sorry, as a large language model, I can't do that Dave.

Now you have.

And I don't just mean the one-word example in the sibling comment: I think a complete sentence first, and then generate the words. The part of my mind I could describe as an "inner voice" does not need to turn this thought into words… but (if you will excuse me giving you an anthropomorphic personification of a homunculus) that inner voice gets annoyed if I don't let it "read out" the words of that sentence.

That's how you claim you came up with that sentence. Your answer basically reverts back to the Cartesian theater, you even bring up the homunculus yourself. There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. Ask chat gpt something equally unknowable and it will also earnestly confabulate a similarly magical-thinking answer. I wonder if you two are really so different!

Either way, what you claim cannot be an explanation, it simply pushes the problem one step back. However it is that you came up with the sentences, when it came to saying out loud the words, or typing them, you still typed one token after the other. Sure you had a buffer that kept getting filled up faster than you fingers could type or swipe, but ultimately you still produced the next letter/word/sentence. Entire sentences can be tokens. BPE isn't the only encoding.

> Entire sentences can be tokens

Sure, but that feels like you are shifting goalposts from "words". (But: What is a word?)

> There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us.

"There is a thought now" -> inner voice wants a go at it, regardless of if I speak or write what the inner voice says.

This part can be (please excuse the anthropomorphic personification) "told" to not bother in order to save time because I already know the entire thought, but doing so generates an emotion which the part of me which did the "telling" can also directly experience.

Some people report not having an inner voice. Mine sometimes gets an echo. (I wonder about comparison with aphantasia/hyperphantasia?)

> Your answer basically reverts back to the Cartesian theater

Indeed. Altered states of consciousness when tired suggest to me this is the organisational form within my mind: my sense of being has, at times, and for lack of a better description given no shared context for the words I'm using[0], "switched off" and I am left with a lack of a sense that I'm really there — not dissimilar to watching a TV show, with the input from my eyes playing the role of the a screen, and my ears the speakers.

Not all minds are organised the same (again, aphantasia etc.), so I don't expect this description of mine to be found in all humans, though I also don't expect to be unique.

[0] SSRI withdrawal has sensations described as "mini electric shocks". Having had both that withdrawal and mini electric shocks: it doesn't feel like that, but I can totally see why someone might describe it that way.

No shared context, prevents deliberate agreement on the meanings of words.

When you look at a computer screen, I expect you wouldn't imagine there really are little rectangles inside, even if it appears so. It's the same with the mind - just because something appears a certain way doesn't mean that's how it works. An explanation has to satisfy certain requirements - it has to be made of simpler parts, not a Matryoshka doll of ever shrinking homunculi with the exact same capabilities.

You might find Dennett's books like Consciousness explained or Sweet Dreams an interesting challenge. If your description of consciousness involves pushing back the veil to a point where exactly the same words/thoughts/actions are being created further inside, and then mediated to the outside via a clerk, then it's not very useful.

My description of my own mind is not meant to be a complete description — if I had that, I would win a prize in Philosophy for finally solving the hard problem of consciousness.

Even merely close would probably win a prize of some kind.

My description claims no such depth — it is merely the first-level of the interior of my mind.

Nevertheless, it remains a counter-example to your prior claim:

> So far nobody's been able to produce a complete sentence in one go!

But so too would be another statement of your own:

> Entire sentences can be tokens

Aren't human beings just token predictors with very specific rules though

The tokens in question didn't exist for most of human evolution, so I'm going to say a fairly confident no.

So if a mechanism was developed for the LLM to add more tokens over time, you would retract your objection?

Nope (afaik LLMs are already capable of generating novel tokens parsing novel tokens generated by [other] LLMs)

The point is that humans and their ancestors had been self-replicating, long-running biochemical processes for millions of years before language (and related abstractions) were even a thing. So I don't see the impressive performance of token-prediction algorithms at associating tokens representing human abstraction with each other in ways humans discern as meaningful as offering much insight into human behaviour, never mind giving a complete explanation for how it happens. The claim that the way we navigate the world boils down to "just" predicting the next token feels even more extraordinary in the context of prediction being something we're pretty bad at.

Tokens don't have to be units of language, they can be any arbitrary chunk of an input stream: https://arxiv.org/abs/2006.03677

(I'd be a bit surprised if our brains are transformers; from what I understand, we're much more sample-efficient than most of our AI so far).

But whilst tokens in the form of pixel values might actually map quite well to sensory input from rods and cones, why on earth would we assume that everything that goes on in there maps neatly to the concept of "predict future tokens"? And where are the analogues for stuff like emotions, evolutionarily useful chemical processes which actually make us worse at prediction?

> why on earth would we assume that everything that goes on in there maps neatly to the concept of "predict future tokens"?

I have no reason to assume anything, positively or negatively, about the underlying mechanisms in the moist fatty ball of electrochemistry that is a brain.

There is no reason at all, when reading such a reductive description, to expect the qualia of emotion to arise even within my own brain — that it does is known to me, but to all others the question "is Ben a P-zombie?" is outside the realm of the scientific method as we have not yet determined a way to test for the presence/absence of qualia.

Even without making assumptions about mechanisms, "predict future experience and generated signals to maximise what you regard as goodness" seems to be a fundamental task for all brains of all species to perform, and one can map the concept of a token to this — though it may be as wrong as the mapping of any of the other time-series prediction machine learning systems that came before Transformer models (these models are certainly not as sample-efficient as we are).

> And where are the analogues for stuff like emotions, evolutionarily useful chemical processes which actually make us worse at prediction?

Similarly, the value proposition of emotion can be easily simulated by an LLM even if any given model does not[0] possess the qualia. But this is a statement that we don't know, it definitely isn't a yes or a no — it isn't even as good as being an "it depends".

[0] my guess is that none have qualia; but as I say, I have no reason to be confident of this guess, as nobody knows how to test for it or why we ourselves may possess it.

> And where are the analogues for stuff like emotions,

My mental model for emotions is an arena of competing monomaniacal agents each trained over evolutionary time. Whether it's lust or jealousy or greed, each one at some point <insert just-so story just like we do for genes>. The prediction from each emotion is the next action that will result in more resources, more offsprings surviving to reproductive age, etc.

The tokens are just aligned signals to communicate between people.

So they appeared through bruteforce to enable communications between people.

For example initially there were species that spewed out tokens randomly, then some of those tokens got assigned to indicate danger, and other members of those species got aligned on that due to natural selection, evolution and neural networks. More complex things got built on top of that due to environmental and survival pressures.

We don't yet know what we are.

"If the human brain were so simple that we could understand it, we would be so simple that we couldn’t." - Emerson M. Pugh

One of the fundamental issues with token prediction is the weight of something is based upon consensus of your data set.

So "token truth" is build up based upon more data points. That works pretty well lot of daily tasks.

LLM reinforces existing knowledge, it doesn't expand on it.

So it's not very good at detailed questions where it's data is limited, or novel thoughts are required, let alone true innovation (Einstein, Newton etc). It also has a hard time deciding what is better choice between similar datasets.

Most adult humans perform those tasks a lot better, which is why they are using large amount of reinforcement models made by humans.

> Lot's of devs here, but als in the media, believe GPT is the precursor for AGI. Which is ridiculous, it's just an incredible token predictor.

Geoffrey Hinton and Ilya Sutskever have said it is impossible to predict the next token without solving reasoning and intelligence. I'm not entirely sure how they justify that. If I have to guess, they would say there's no hacky shortcut that the neural net can cheat with to solve this problem, actual intelligence is the only way, and so that's what we get.

My 2c: The thing that LLM critics miss is that LLMs have learned useful representations of complicated abstract concepts. Analogous to edge detectors -> object detectors in vision networks, LLMs have circuits that correspond to human-understandable concepts. I have no idea how we're going to solve hallucinations, but it's obvious to me that we've already figured out a key milestone on the way to AGI: How to encode abstract concepts into neural nets.

It's now just a matter of how to use those representations downstream in a way that doesn't hallucinate.

Ever thought about ... let's say you were God. You wanted to design intelligence, to control insects, animal bodies and eventually humans. What would it need to do?

Well, you'll quickly find that bodies require constant control to survive. Whilst an animal's body can survive for months with an animal's mind operating at 1% capacity, if the output of the mind truly stops, the body dies. The heart, respiration, the circulatory system (deciding where the blood goes), ... all need to be controlled with some amount of intelligence for them to function at all. Even just to remain alive, they need intelligence. They don't work on reflexes alone.

For mammals, total loss of control leads to fatal injury ... in seconds. Without control, the heart will either stop or (wrong order of muscle firing) be totally ineffective at building pressure in the circulatory system. This will kill oxygen transfer through the blood-brain barrier as soon as the pressure drop reaches the brain, which will be much less than a second, this leads to further loss of control in 2-3 seconds. Irreperable damage sets in before 20 seconds pass, death (permanent and total loss of control) occurs in humans after about 1 minute 30 seconds. The plus side of this seems to be that, when that control is there, mammal bodies are far more efficient than a "passive" robot can be (any robot you can turn off -safely- at any time is a "passive" robot, the big counterexample is a flying drone, which will incur damage and may injure others if turned off or it loses control)

So "predict the next token" is really THE most important function of the brain. Second, it is incredibly important to KEEP predicting the next token, to keep going, and keep going, and keep going, no matter how stupid the output, because literally any output will give a result superior to no output at all. No matter the damage, no matter how terribly wrong things are going, it must keep going further.

I like where you're going with this! I think a stronger argument is that animals can survive so well by ONLY predicting just a few seconds in to the future for the most part.

If you own a cat you'll realize most of them don't have a good understand of physical properties like levers or strings or one thing can knock into another. Of course you'll find on SM cats opening doors with uncanny ability. But I can see from my own cat who can open doors, that he has an incredibly rudimentary understanding of levers. He only knows the lever must be moved but not which direction or how. He can only mash at it with his body until it eventually opens. He doesn't understand why a child proof lock stops him from opening the door so he'll try over and over again. With other cats as well, they only form vague associations over either many repetitions or from an emotional experience. But they don't reason the way we can learn from one experience.

It seems to me you're conflating two different things - control systems and intelligence. We already have pretty good engineering tools for control systems and better ones are discovered now and again.

LLMs aren't meant to control individual servos, although it appears that transformers can do that too. The human brain doesn't use the same architecture for controlling the heart as it does for processing vision, for example.

Everyone's always comparing LLM and trasnformers to the human mind. Hence this is a valid comparison, as the human mind definitely is a control system. And it has the active robot limitation.

Everyone's always compared the human mind to the most advanced technology we have. First it was pumps, then transistors, then computers, and now the highest abstraction so far, the transformer.

They may or may not be sufficient to replicate the human mind, but they just haven't failed to scale up and show new abilities yet.

> I'm not entirely sure how they justify that.

You know that humans hallucinate too, right? All the time, whether intentionally or unintentionally. Our memories are sparse, and we fill in the gaps with things that sound reasonable. We do this constantly. We also lie. Also, constantly.

LLMs are definitely not intelligent from a reasoning and "shape rotator" perspective, but the competence of wordcels and the entire concept of "verbal IQ" has been absolutely obliterated in a just few years.

Once we integrate human reasoning and "shape rotation" models, it's game over.

I was going to say humans do something like hallucinate all the time - we come up with what ifs, fiction and so on. I think the thing is we then hopefully go back and check if it seems real or made up, which would not necessarily be hard to do in LLMs. Having come up with a fact it could google it to check for example - something I do myself.

It's interesting with the "shape rotator" or physics engine type stuff OpenAI's Sora video thing seems to have picked up a reasonable amount of that.

The current LLMs seem to miss out on a looping thinking aspect as in I could do option A - visualise it - not that wouldn't be good because of some problem - how about option B and so on that we humans call thinking about things. But it seems that could be programmed in.

You're correct, I have grown to be very suspicious of smooth talkers. eg UK PM Richie Sunak and London Mayor Sadiq Khan are both extremely eloquent and they are both utterly useless in their jobs with zero ideas or imagination as to solving the problems they face. Contrast Elon Musk not a very smooth talker but brilliant at solving real-world problems.

> Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor. Its language capabilities might be useful, but AGI won't be possible without a self-correcting model that can actually reason. Even Altman hinted at that.

I think part of this is that all three letters of the initialism mean different things to different people.

To me? The domain is broad rather than narrow, so it's got the "G" of "AGI".

Definitely agree it is not a complete solution, and thus that reasonable people will say "no Ben this isn't general what are you talking about".

I also definitely agree isn't superhuman, but I think superhuman ability across all of its domain is an unreasonable requirement for AGI and what I would instead call that ASI… but I'm also a linguistic descriptivist, so it doesn't matter if the consensus is unreasonable[0] just what the consensus actually is.

Also: my understanding from "Fine-Tuning Language Models from Human Preferences"[1] (in particular the summary version of the bug, "The True Story of How GPT-2 Became Maximally Lewd"[2]) is that these models are capable of self-correcting behaviour — we just don't do that live because that makes it much harder to research and to use (every time it even seems like the ChatGPT model changes, someone complains).

> But it will keep getting stuck on the accuracy part

I suspect otherwise. Some — not all, but some — domains have easily testable results. In these domains, the generated output can be quickly tested, and used to update the model.

This is already done with human preferences, but I'm thinking more along the lines of a coding assistant which gets feedback from a compiler, or a maths assistant which gets feedback from… maths isn't my field, I'm just going to say MATLAB or the Wolfram Language and hope these are both as highly regarded as Photoshop.

[0] this happens a lot: tautologies such as "ATM machine"; or redefinitions such that "PC" excludes MacOS and all phones and tablets; that Elon Musk can only be called "African-American" by pedants (including me, despite being a descriptivist); exactly how effective and how altruistic is "Effective Altruism" etc.

[1] https://arxiv.org/pdf/1909.08593.pdf

[2] https://forum.effectivealtruism.org/posts/5mADSy8tNwtsmT3KG/...

Being superhuman is the defacto definition of AGI leading to singularity, mainly self-correcting is main requirement, and that's why they fear it.

If you just change the definition of AGI to what GPT does currently then indeed by your definition it's AGI.

But that's not what OpenAI & musk mean with AGI.

"""Artificial general intelligence (AGI) is a type of artificial intelligence (AI) that can perform as well or better than humans on a wide range of cognitive tasks,[1] as opposed to narrow AI, which is designed for specific tasks.""" - the opening sentence of the Wikipedia page on this.

"[A]s well or better" allows "better" without requiring it.

I will also note that while OpenAI is indeed clear that their definition of AGI is superhuman[0], this is not a universal position[1].

[0] "(AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work" - https://openai.com/charter

[1] "The most popular definition of AGI is the ability to perform any intellectual task that a human can do. While highly desirable, this may not align with what’s essential for achieving the singularity." - natanael.wf, Dec 2023, https://community.openai.com/t/agi-is-what-we-want-but-not-w...

It's what we all mean without saying it, because singularity

Given all the people you're critisising for saying ChatGPT a precursor to AGI, can we at least agree with my opening line?:

> I think part of this is that all three letters of the initialism mean different things to different people.

People only look "ridiculous" to you because those who consider AGI synonymous with ASI are doing exactly what you (apparently) don't like me doing: "just chang[ing] the definition of AGI".

No, because if you read the other comments in this thread you will see lots of people think that human's are just Token generators as well. The might say it with more sophistication at times, but essentially a lot of technoligist are arguing that from an statistical anaylisis alone, conciousness arises if it's complex enough. Okay that's a different debate, but it all gets mingled up.

The mechanism of our minds vs. an AI, and the question of "consciousness", are utterly irrelevant.

The mechanism doesn't matter because the only thing that does matter is results ("do submarines swim?"). Even the fact that current AI require more examples compared to a human matters less than the magnitude of the difference — given the speed difference between transistors and organic synapses is on-par with "runner vs. continental drift", the limit of current GenAI models is running out of training data, which we humans don't live long enough to do.

"Consciousness" doesn't matter because that word has about 40 different definitions, many so bad that tape recorders pass some of the tests and humans fail to pass others.

Economically speaking, it doesn't matter if an AI (be it narrow or general or anywhere between, or on the other axis be it sub/medi/superhuman) is a P-Zombie or not.

… it doesn't matter if a transformer model feels something, only if it might take my job by writing code that works well enough that the market for software developers disappears. (This may be a higher or lower standard than "is it as good as a human?" due to it being unclear at what performance level the tool saturates demand: as a toy model, if it turns out 90% of apps are a CRUD database feeding a JSON API feeding a trivial UI, I can easily imagine an LLM (or even a future version of Figma) automating all that and forcing all the devs to compete for the remaining 10% of tasks and there's no guarantee whether or not the remaining 10% of tasks will expand to fit the supply of developers).

… it doesn't matter if a diffusion model has a love of art, is inspired by the beauty of nature, genuinely empathises with newlyweds who ask for their alter photo to be transformed into a renaissance oil painting; I imagine that artists only care if demand for their services has gone down (see all the artist saying both that it threatens their livelihood and that it sucks).

… it doesn't matter if a business agent has a monomaniacal focus on profits or if it has side-interests, hobbies, and a pet giraffe; the only thing that matters is if the business may harm my interests. (See any discussion along the lines "are corporations a type of AI? And are they aligned?" regardless of how you feel about terminology).

… it doesn't matter if the Google search engine experiences joy when I click on a result it provides, just that it gets me results.

… it doesn't matter if Deep Blue or AlphaZero enjoy winning, just that they did.

There are two ways "consciousness" (in the sense of qualia) matter to me, but they are not performance issues:

1) Brain uploads, where anyone who wants to use it to circumvent death wants consciousness to be present

2) Slavery: Will this economic value be an accidental re-invention of slavery, where we use exactly the same "they're not really people" arguments to avoid the cognitive dissonance of how we treat conscious beings that we own, create, and destroy on a whim.

These are ethical issues, which are not at all relevant to the question of "what do we need to get human-level capabilities?", any more than Eli Whitney needed to ask "what's the ethical aspect of this?" to be capable of actually inventing his cotton gin.

>Interestingly enough, OpenAI seems to go all in on GenAI, or at least spend a significant part of its resources on it. Generating videos seems like a cool feature, and perhaps commercially viable at some point, but it doesn't seem to bring them closer to AGI. Not sure if that's shiny object syndrome, pressure from Microsoft, surfing the wave, or a big masterplan I'm not seeing.

Building AGI will be extremely expensive. They need ways to stay alive and fund it. I would compare it to Tesla building the Roadster to fund S, and S revenue to fund the next car, etc.

I kind of feel like reasoning models will be less based upon the amount of parameters, but I could be completely wrong.

We aren't sure we know how to do it at all yet[0], so there's going to be a lot of research going from here to there.

Also, we don't know how to do it safely, which they also care about, and also takes a lot of research as quite a lot of us don't even agree on the question.

All that research is going to be expensive.

[0] It's long been possible to make an AI which can learn any function, but all the current ones need too much training data, and we don't know if we can reach economic usefulness in all human domains with the currently existing data even if we ignore that so many data holders responded to ChatGPT and Midjourney/Stable Diffusion by locking away all their data.

> Lots of developers here, but also in the media, believe GPT is the precursor for AGI. Which is ridiculous; it's just an incredible token predictor.

What test (or facet, for that matter) of intelligence/AGI can't be modelled as a token prediction exercise? Bearing in mind that quadriplegics are intelligent even though they have lost control of their arms and legs.

Gary Marcus predicted that deep learning was "hitting a wall" [1]... right before ChatGPT came out. At this point he's so committed to the bit that he has to keep predicting that this time, really, it'll stall, and hoping eventually a stall comes.

1: https://nautil.us/deep-learning-is-hitting-a-wall-238440/ Also note he doubts that scaling will work past GPT-3... Not uh, 20/20 foresight exactly.

When will the “When will the GenAI bubble burst? "bubble burst?

I've seen contrarian think pieces like this pop up on social media so much lately they pollute my feed just as much as the GenAI evangelist articles.

Maybe it just indicates a general AI buzzwords fatigue, regardless of actual progress in the field.

That's a signal that the tide IS turning. I follow some people who went all-in on AI (consultants mostly) and their tone has changed quite a bit in the last few months.

Cool new tech that was oversold in terms of what it can do for businesses. A correction is in order.

A correction is in order in the short term. We don't yet know what ChatGPT can do in the long term. In the short term, though, it can't do as much as it has been sold as being able to do.

Do you have statistics on his error rate, or just examples?

I'm of the school of thought one should aim to be about 70-80% right: score above that, and the problems you're tackling aren't hard enough; score below that, and you're not good enough to help with the problems.

His Twitter feed is nonstop, more-than-daily posting that a new AI winter is just around the corner, really, this time. I haven't bothered to count every prediction but he's got at least three 9s of being wrong about this topic and refusing to update his arguments when faced with new evidence.

Fair enough then.

(I don't have a twitter account, so I couldn't learn this myself).

This is a pretty inaccurate summary of the linked article.

I guess you can model Gary as a token predictor writing the most likely token based on his previous published work.

Here's one of many example use cases we found for GPT4 API:

Our sales people request invoices from a potential customer. On those invoices are our competitor's services and price. Invoices can come in PDF, png, jpeg, excel, csv, email formats. Content formatting can come in random forms. Pricing breakdowns are also non-standard across invoices. We have matching services and our own prices.

The goal is to find similar services where we charge less. In the past, our sales people would spend hours combing through those invoices. We wrote a prompt for GPT4, fed in our services and prices, and asked it to find services we could potentially replace as well as our profit margin. It took us a day to write this prompt. The results were outstanding and GPT4 gave accurate results. We even asked it to package it up in a PDF for us to send to the potential customers. On a Monday morning, we started on the prompt. By Tuesday morning, we got it working well enough that we were confident ship it to a few of our sales people to test.

This will save our company hundreds of thousands each year and we can get back to the potential customer much faster than before - increasing the likelihood of a sale.

If we had to program this like normal software, it'd probably take months to get it right with dedicated engineering resources to account for new invoice edge cases. Chances are, engineering would never even prioritize this feature for our sales people because there is simply no economical way to account for so many different invoice edge cases.

I believe we're just getting started. If we get GPT6 in two years and massive improvement in inference cost and context size, it's going to change everything we do. Heck, even GPT4 with 100x context size and 100x lower cost per inference would be transformative.

If this is a bubble, I'd like to live in it. I believe that many businesses have found use cases similar to the impact of ours. But they're just not broadcasting them to the internet in order to keep it a business advantage.

That's a great use-case! But what about when GPT hallucinates - gets it wrong. What happens when you have sent a (potentially legally-binding) quote to your customers, and you have to back out of it?

Humans make mistakes - whether directly, or indirectly through the software that we craft. But when the mistake happens, we can reason about it, understand why it happened, and fix the problem.

Wouldn't it be embarrassing to not be in a position to reason about / explain a mistake to your customers caused by this "thing" we use that's been trained on all the random information available on the internet? Would that not reduce the value and standing of your business (which is ultimately about the people and expertise contained within) in the eyes of customers?

As an engineer, technologist, and CTO I absolutely love what I've been able to craft using generative AI under direct supervision.

But I cannot imagine a world where we give our carefully thought-through, easily changeable, algorithms and processes for a black box that cannot be reasoned about, no matter how good the output _sometimes_ is.

I have to think many people feel this way.

Our sales people do a quick eye test before sending it out.

Usually, the list of recommendations are about ~8 services that we can replace. Our sales people know those services and prices by heart. A quick glance is enough for them. Our quotes are not legally binding. The contract that we sign is.

We found that it was highly accurate. GPT4 has been especially good at stuff like this after OpenAI changed it to write Python code for this kind of calculation.

This! GPT is not an "intelligence" by itself, but more an extension to the human brain, like an "exocortex" holding information that we cannot or dont want to keep in our neocortex. Google search was the first of these "exocortex" extensions to the brain that was popular among many people. So the same seems to happens now with LLMs. Maybe we will not end up with an AGI but more with a human/machine hybrid which has the "superhuman" capabilities.

just my 2 ct

Thanks for the example and that sounds really solid cost savings and definitely agree with the trend that it is here to stay.

For invoice parsing (various formats), are you just using GPT4V? When GPT4V initially came out, i benchmarked it against an out of the box invoice parser from Google Cloud (https://cloud.google.com/document-ai) on 16 documents and it was much better accuracy wise. For ex: i'd get results parsing 10,100 as 101100 (no comma).

Curious if you saw problems like this in your pipeline or if its gotten much better since?

There is one big difference compared to previous hype cycles:

While crypto or VR tech still hasn't arrived in our daily lives, most of my friends are already using tools like ChatGPT on a regular basis.

Generating images and videos is entertaining, but I believe that the true value of AI lies in automating tedious, repetitive work, both in the digital (e.g. data entry) and physical space (e.g. assembly or logistics work).

It's these "boring" use cases that will bring big efficiency boosts to our economy.

The bubble bust on the Internet too; the outcomes can be wildly different.

You sure that the bubble bursted ? Tech companies are among the wildest valuations for all-time, including post-burst.

If you were QQQ investor in 2000 you are now extensively rich.

Most investors are fairly sure the dot-com bubble burst; but it recovered well, and obviously there were still winners. The last two times the AI bubble burst also did not stop us getting here.

> There is one big difference compared to previous hype cycles

Not if you compare for example to the dot-com bubble.

And it's actually useful.

There is a big question to ask yourself when you see a new evolution: would I get sad if this disappears or be missing it ?

crypto ? wouldn't care VR ? wouldn't care google glass ? wouldn't care facebook ? wouldn't care internet ? would be a serious step back LLMs ? would be a serious step back etc

> There is a big question to ask yourself when you see a new evolution: would I get sad if this disappears or be missing it?

That’s the wrong question because it takes time to get used to that evolution. If I asked you in 1997 if you’d be sad if the Internet disappeared, it’s very likely that you would have said "no".

But GPT is already a big part of my workflows and a great productivity multiplier.

Very reductive. I recognize it's useful in some very specific use cases, yet I wouldn't miss it a second or care if it disappeared.

I would feel coding is much more tedious without Copilot.

When I'm using an editor that doesn't have Copilot, I keep waiting for it to auto complete, and get frustrated that now I have to write this block of code myself, and think what a waste.

I was able to code in the past and solve the same problems, it was just much more tedious and time consuming having to type all of it out that was in my head anyway.

Especially if working with multiple languages, which I used to despise in the past because of muscle memory ruining it - Hey python. I wished there was a single language syntax to rule them all.

Now copilot does it for me.

I use CodeWhisperer instead of Copilot, but sometimes it gets disconnected and it takes me days to realize "Oh I don't have this nice autocomplete anymore".

I agree that smart autocomplete it's a nice use-case, probably the most compelling I've seen so far.

But I could live without it.

How do you think about using it?

Because for me with Copilot, I think I can usually predict with 95% accuracy what it's going to output for me so I know that if I write "fun" (beginning of a function) in this context, it will immediately know based on all the context around it what will generate for me and that is what I'm looking for.

So my whole development is around it, that I can predict what it's going to complete, it just does it much faster with many lines at once, without the typos , index + 1 errors and other minor bugs I might do myself.

E.g. simple example, while usually things are more complicated, if I want a leftPad function, I would just write "function lef"... And mostly it is already able to know. While previously maybe I should've figured whether I should use an npm library where historically something went really wrong, etc. I'll just keep those helper fns in the codebase now.

But that's only a small example, it's able to handle non generic helper fns quite well too.

Like my muscle memory and expectations are around what Copilot can do or what I expect it to be able to do.

"This time will be different."

Now ask those friends how much they would pay for GenAI.

ChatGPT is free. A VR headset is hundreds of dollars for the most entry level, crypto is literally money, etc.

Although I probably spend up to $700 or so on AI tools monthly. I feel like I get multiples of value out of that.

Mainly it allows me to build side projects significantly faster. My salaried workplace output is accelerated so I have more time to work on side projects, I can do more side projects in similar amount of time, I can do consultation on the side, which allows me to bill for much more than $700.

Copilot gives enough energy to be able to code 12h+ a day.

Usually those little things that Copilot auto completes for you, really stack up in terms of being mentally exhausting.

But I can be in a "big picture" role, and think in terms of that, while Copilot does the coding for me, I will just check the code, rearrange something if required.

It is like I'm more of in a CTO or CEO role of my side projects and have this extra coder that I'm doing reviews for.

I probably bill around 60% of my salaried work freelancing on the side, and I wouldn't bother to do this freelancing without GPT, so for me it has directly contributed to 60% increase in income.

I think people who aren't willing to spend on that, either don't understand how to multiply their output from it or don't understand how to make multiplied output into effective income.

$700? What do you even pay for??? I don't think I'd be able to spend even $200 on GenAI even trying hard.

I'm interested to see which tools warrant this kind of money for an individual.

I imagine ChatGPT 4 and CoPilot but I can't think of anything else.

I have subscriptions to many different AI services, from Sona, Gemini, Perplexity to others, many I have just tried out, but of course they are not the main cost.

My largest costs are related to OpenAI API.

I use OpenAI API to go over my data, create embeddings of it, reorganise things, scrape data, standardize and analyse data, etc.

I have a personal journal/assistant type of thing that I use to chat with, or help me make decisions, sanity check myself - it has a prompt filled with various context it can search from embeddings. Like the people I know, the projects I'm working on, the goals I have in life, etc.

GPT goes over all the data and does summaries, organises, adds better search keywords for potential embedding callbacks.

Something being in a bubble and being useful is not mutually exclusive. Remember the dot-com bubble?

There’s also an argument that ChatGPT kind of transcends the bubble. The bubble is everything that calls their API and now suddenly gets a huge valuation because it’s “AI”, but the feature the AI enables is of dubious value.

We'll see. I don't think that would help them if and when it pops. The survivors/big players tend to suffer disproportionally with regard to the actual fundamentals (look at dot-com again for comparison).

I think it's quite clear that we are both in a bubble and GenAI has a lot of potential for productivity gains and maybe even further utility beyond that.

That seems about right. There are definitely some “Long Blockchain Iced Tea” aspects to the AI hype cycle. Crypto was an embarrassing bubble given that it created almost zero value.

But there’s real AI products with real use cases, and a lot of other things that have no business calling themselves AI. And people who say it’s “all hype” are just as wrong as those who say it’s “all legit”.

Being the cause of the bubble doesn't make you immune to the bubble: I think Stable Diffusion is also equally important to the GenAI topic, but I wouldn't want to be one of their investors; and much as anyone saying "moat" in this context my eyes roll[0], just like every other business in the history of forever there can be no certainty that competitors won't undercut OpenAI on pricing and/or quality.

And until the dust has settled on intellectual property issues (and I don't just mean lawsuits: regardless of the results of those, governments may change the rules), all of the current models and labs are at risk.

[0] it's a thought-terminating cliché spoken by organic stochastic parrots.

> most of my friends are already using tools like ChatGPT on a regular basis.

Are they?

As a counter anecdote, I don't know a single person who does. Even non techies are skipping personal assistants and such because they can run a google search themselves.

Gary Marcus is not a person writing serious things. We barely have any tooling for this stuff. Most people don’t know still how best to use these things in production workflows. Investors will continue to throw their piles of dry powder at this until a large large company comes out and says “we’ve failed to figure out how to use genai to improve things”. And no company ceo is going to come out soon and say something that bold. Nobody wants to look like a moron.

So the answer is, there is no bubble. This is an exploratory phase, and there is a lot more money left to chase opportunities.

IMO LLMs are basically the best thing since wikipedia for learning/autodidacts.

They're also pretty fantastic for discoverability if you do a bit of filtering to remove hallucinations. Only downside is you end up finding that project is abandoned for years or so.

Glad to see some scepticism. I agree that the development seems to have slowed for a bit. Claude 3 is a bit better than GPT-4 but if we continue with such tiny improvements every year its not enough to keep the hype going imo.

GPT-5 has to really slap.

This reminds me of the exciting time when I first got to make my own ray-traced 3D graphics... in 1991. I had to sit there and watch as the free POV-Ray tool generated the image pixel by agonising pixel, left-to-right, top-to-bottom.

Toy Story was four years later, and required 800,000 machine hours on a "render farm" to produce an output resolution of just 1080p.

Now? I get to play games on a single PC at 60 fps with ray tracing at 4K resolution!

Conceptually, the glacial pixel-by-pixel rendering in 1991 and the RTX games today are the same thing, but qualitatively there's no comparison.

We've been shown that machines can be prompted in English, think about what they've been asked, and produce useful output... at the same kind of rate I was getting from POV-ray in the 90s.

Now picture the same thing five years later (equivalent of Toy Story), ten years later (Grand Theft Auto III), twenty years later (Battlefield 3, Kung Fu Panda 2), or thirty years later (Cyberpunk 2077).

Some of us can extrapolate from the present and foresee that the pixels crawling one at a time across the screen is not how it's always going to be. That one day the same technology will make our dreams come alive.

Others look at nascent technologies like generative AI and think "Pfft... too slow." and write Substack posts.

You mentioned GTA III, now GTA VI that has LLM support... This is going to be the best game of the decade ("full life simulation")

I use GPT-4 to assist in some programming, and it's useful. But I've been struggling to figure out where LLMs make sense as something integrated into a larger product. As an end-user, where I can directly ask them for something specific ("write a function that...") that I can both inspect, double check, and further refine before accepting, they make sense to me. But as an integrated aspect of some system, I haven't been able to come up with any compelling, or even mildly useful usecases.

Also, is anyone using the local, non-OpenAI LLMs for anything useful? They've always been slower and measurably worse than OpenAI's GPTs for me.

All hype cycles involve bubbles and bursts. The point is on a longer time scale the growth happens anyway... look at the .com bubble followed by the internet eating the world. Things just take longer than you expect and initial leaders tend to fall behind. The big question here is where OpenAI and Nvidia will be in 10-15 years... my theory is that chips will be mass produced as we've seen in past waves bringing costs and valuations down for Nvidia... meanwhile OpenAI's death grip from Microsoft will strangle it slowly into an AOL like outcome

One thing we're desperate to do in our team is to somehow feed all of our Slack history into an LLM and ask it questions.

There are many decisions, reasons for those decisions, and business logic lost in Slack channels. Slack search isn't nearly good enough to surface them.

I can't count how many times I know the answer is in Slack somewhere 5 months ago but I can't find those messages anymore.

I see this as an inference problem that is solvable by more efficient LLM software and better/more AI hardware.

I have also thought of doing that, and that can be solved using RAG.

Agree that Slack search is terrible.

Even just having vector embeddings for search would already be huge.

Are there any companies doing this for Slack now? Or are we all just waiting for Slack to add this and charge an extra $20/month per user?

You can kinda do this with Notion AI. It's not quite the same as slack ofcourse because you need to manually build up the knowledge base first but if it is well maintained you can use it to answer questions easily.

Probably there is some tools out there to create notion pages from slack history? Would be a useful tool at least.

I think GenAI is definitely in a hype phase right now, but even so this take seems out:

> $50B in, $3B out. That’s obviously not sustainable

There's been a lot of building out of models and tools, presumably the bet is that it'll pay off over the next few years.

For what its worth, I'm not convinced it will, but calling it now based on those revenue figures is like saying a ship is a failure after its first voyage because it cost more to build than it made in that single trip.

Meta spent $100b on VR.

$50b is nothing.

The release of ChatGPT 3, 4 made us astounded because it was doing things a computer never could have done since they exists. That made us believe that they could replace humans in a lot of fields, where, in fact, it only improves the work of an already existing human. The hallucinations, specially, can be very costly. The money saved by doing 95% of a taks with an AI can be swallowed easily by an human having to work to fix the 5%.

For developers ChatGPT is like having a team of good juniors on your fingertips. I think it can increase the productivity of a senior engineer probably three fold. But you must do code review and understand the code.

Yes, I believe it is specifically good when you have a lot of experience so you can immediately tell what is correct, what is wrong. I think this is an excellent time for experienced engs in the market, because GPT might in certain terms hinder low experience engs, but scale exp engs output many times.

I think the main issue is that they look for revenue and income directly generated by AI/ML while this technology (at least currently) is mainly aimed at cost reduction. There might be already tens of billions saved in operating costs for many companies achieved by using "AI" solutions.

It won't burst, it'll just eke out over a few years as people realise it's harder to implement than it is to talk about. Some things will be successful, a lot won't. People will move onto the next thing, marketers will market.

See: digital twins, cryptocurrencies, semantic web...

>$50B in, $3B out. That’s obviously not sustainable.

Though there will probably be massive value, greater than $50bn in AI in the future. I'm not sure if this plays out like the dot com crash where there was obvious long term value but capital got misallocated to a bunch of pets.coms (https://en.wikipedia.org/wiki/Pets.com) which got wiped out in the crash after 1999 or not. This time most of the $50bn seems to be going to large companies that will probably stick around.

My org at work is organizing a hackathon and the theme is AI-based solutions to help with software development execution. For the past year they've also tried to shoehorn LLMs anywhere they can.

Of all the ideas proposed so far, not a single one of them needs LLMs or AI to be done and could be implemented with 30 years old tech.

I won't say LLMs aren't more useful than Blockchain, they are, but this is exactly the same situation of all companies trying to find a problem to the "solution" that GenAI is, like they tried to find a problem that Blockchain solves.

Except much bigger companies have invested much more money in GenAI. The bubble burst will be spectacular.

> At least one company must..

> Find some huge problem(s) that they can charge a lot for.

> Ship a solution to that problem at a reasonable price.

> Whatever they sell must give customers enough confidence to merit adoption.

You know which company is this?

This is any of the million of companies, who build products based upon customer need and sells with reasonable or great profits - whether it is in space of consumer or enterprise.

Pureplay GenAI should only strive to serve these compamnies (via APIs). GenAI companies would have very hard time understanding end users and designing product as per their needs or domain specific workflows. Chat interfaces does not count as product building.

When will railroad bubble burst?

> Thus far, the closest thing to a killer app is probably coding assistace

Oof, if that is actually the case, then it's in serious trouble, because it isn't actually very good at that.

What if it isn't a bubble? I mean yeah it's overhyped, it's a bit _too_ easy to find capital as long as you slap the AI label on something, but the damn thing _works_ already, you can easily verify this by using it. Current LLMs may be ultimately a dead end and not the road to GenAI, but they are useful already today as they are, and likely will still improve for a while.

Being useful in some areas is different from tech companies trying to cram "AI" into every product, no matter if it makes any lick of sense or not. That irrational behaviour is what I would call the real AI bubble, and I can't wait for it to burst ;)

Agree on that. The need of an AI rice cooker, an AI printer, an AI Juicero, is definitely questionable. Hopefully, eventually, this stuff will run out of money and we'll be left with what makes sense.

But where is the real use? Chatbots and stock images seem very niche and hallucinations have not been solved either.

Hallucinations don't matter if you check the output before using it, at least as long as it still saves you plenty of time.

Can it find 500 issues in a codebase, 470 of which are real? Good. Can it summarize a conversation and find who proposed a certain course of action? Can it go through 170 years of legal precedent and quote relevant cases (better than the existing, conventional search engines, which you would then later use to verify the quoted cases)?

> 50B in, $3B out. That’s obviously not sustainable.

Ouch. GenAI seems to promise no shortage of solutions but offers no reliable revenue stream.

Plateau of productivity is still out there waiting, it is just not in 1 year, as some investors and valuations assume.

In 10 years the impact will be so profound LLMs will be everywhere, in 15 years we will not even recognise industry. Mobile boom will be humbled by LLM boom. Of current giants, maybe Microsoft will come out on top, they have everything they need.

> Plateau of productivity is still out there waiting

Well, actually for me it seems to be there right now. It's not great, it's mediocre, I understand its limits, but I regularly recourse to ChatGPT for simple tasks. I don't trust it, I need to verify each time, but its all factored in. I got to manage around its limitations somehow and it's basically boring tech for me. I guess it's similar for many others.

This is what is going to happen. Recall that most people didn't believe in the internet revolution nor the print revolution when they just started, it feels the same

Probably not until speculators find a new bubble to up.

My hunch: GenAI will be very useful embedded in many applications, but mostly in the background (like relational databases and unlike "blockchain technology"). It won't become ubiquitous and ever present, like the internet or mobile phones.

Within a year of launch, nearly all my friends use ChatGPT on a weekly basis. Granted, I'm a university student, so I'm sure I'm in quite a specific bubble, but still - it already feels quite ubiquitous.

100M weekly users too. I don't see how it won't become ever present.

> Probably not until speculators find a new bubble to up.

I mean in that case it's probably ~dead, or at least on life support, within a year or so; speculators are quite good at finding new bubbles.

My pet theory is that OpenAI releases are on hiatus pending the US elections. It would be a huge disaster to release SORA in the next few months and have it some way sway voting. What do you think?

Sora is so ridiculously expensive, they doubt it has a market.

Not that I doubt it, but curious what's the source for costs?

I don't know of any official source. But reading between the lines: Murati also told the Journal that Sora is “much more expensive”. And just how few sample videos they have output. The point of AI generated content is you _never_ need to reuse it, you can just generate more.

You can’t halt this stuff. There isn’t one player and there is considerable market incentive.

Not a chance. Aren't they gearing up for GPT-5 in the next month or two?

Tangential, is the concept of AI Winter still relevant?

Yeah, nuclear winter and that grey snow which is falling is burnt dollar bills and startups.

Whilst we are certainly in a hype bubble (AI's third) and there are some indications it may lead to the third AI winter, the two possible outcomes are this bubble doesn't burst, or a future bubble doesn't burst. The long term trend has always been to stronger AI and higher highs.

Of course. Right now we're in an AI Spring, but winter will come again.

AI Winter was related to neural networks existing but computers not being fast enough to handle the amount of data needed to train them + massive datasets themselves not existing.

So far, I'd say no? New models are still being developed and getting better in very short time frames vs what folks did historically.

The finance bubble thesis is more related to: "are companies going to generate revenues to justify the massive investments being done?"

Once AI has been used to launch a successful cyber attack on any vital resource and it gets into the press

Just like crypto, AI has recently seen a seismic grift shift in startups and courses all around.

Even the VCs know this as the AI industry spent 17x more on nvidia chips than it brought in in revenue. [0]

So we will likely see a massive fallout when most of these AI startups are quickly commoditised either by OpenAI, Anthropic etc (better & cheaper AI models), Apple (local AI models) and open source (replicating closed source models that is 10X better)

It is an industry that is the race to the cheapest and ultimately the bottom.

0 https://www.wsj.com/tech/ai/a-peter-thiel-backed-ai-startup-...

Unlike crypto, AI already has very strong real world use cases and has been wholly embraced by the public.

It’s an unfortunate reality that one of crypto’s long term side effects is instilling a strong tech-cynicism in a large amount of people. This is a consequence of the deluge of bad actors that capitalized on crypto’s inherent ability to enable fraud and speculative “investing” at unprecedented scales.

Well said.

It's a lazy comparison between crypto and AI.

Crypto is a bunch of scammers selling securities in an unregulated market. It exists simply because politicians move too slow.

GenAI is more like the invention of electricity or internet. People will figure out ways to build on top of the invention.

I'm sure you can go back to the 70s/80s and find that many people have said the bubble would burst or that the technology would never be useful.

They are both the same.

Both have breathless hype.

Both are extremely overvalued and are a solution looking for a problem.

Both aren't really useful unless you're (trying to count generating cat memes and broken hallucinating code a use case, it isn't).

And last and the most important of all:

Both of these waste a ton of energy either by training or running the software and are unlikely to make startups money due to it's rapid commoditization.

https://arxiv.org/pdf/2311.16863.pdf

https://www.theverge.com/24066646/ai-electricity-energy-watt...

I use ChatGPT and Copilot embedded my IDE every day. I can review my commits from the past year objectively say that these tools have increased my code output and quality. So reading that AI is useless seems like a stark denial of plain facts. People are already getting plenty of real world use and productively increases from AI

Using AI to objectively on average churn out broken and unreadable code to increase technical debt on your team isn't a use case or at all useful.

https://visualstudiomagazine.com/articles/2024/01/25/copilot...

I also missed out another close similarity AI has with crypto.

Both of them aren't regulated.

Both of them unfortunately cannot be stopped.

>Just like crypto, AI has recently seen a seismic grift shift

Also seems to miss the fact that crypto was grifty from near the start. I don't think it's quite as bad now as it was in the 2017 ICO boom.

There is a distinction between "the technology would never be useful" and "we are in a bubble that will burst." See https://en.wikipedia.org/wiki/AI_winter for many examples of the latter with regard to various incarnations of "AI."

We need that laughing Leonardo Di Caprio meme with Jensen Huang's head pasted on.

Reply on news.ycombinator.com