The A.I. Bubble is Bursting with Ed Zitron [video] | Hacker News Reader

The A.I. Bubble is Bursting with Ed Zitron [video]

youtube.com

51 pointsgrugagag166 comments

He claims that language models will soon run out of data to train on, and therefore stop getting better. Language models have already essentially run out of data to train on, yet they continue to get actionably better. Data quality is still a huge space for continued improvement.

Also around 46:20:

> the fundamental thing of knowing [the user] requires knowledge, which requires intellect, which GPT cannot have - it's mathematics.

I disagree with basically all of this except that knowing requires knowledge, but that's basically a tautology. Firstly, knowledge and reason do not depend on each other. Wikipedia is a highly knowledgeable system that demonstrates little to no reason (maybe the search can be considered a demonstration of reason). A calculator is a highly reasonable system with extremely primitive knowledge. Intelligent systems have knowledge and reason. Anyone who has played around with an LLM would agree that they are able to demonstrate some level of both knowledge and reason.

Also the implication that biological brains simply exist outside the domain of mathematics is... interesting.

Then they go on to talking about how you can't make small tweaks with generative video AI, like telling the actor to walk a little bit faster or slower. To that I want to highlight that ComfyUI has support for video nodes[0]. If you're not familiar with ComfyUI, just check out some tutorials. Generative AI art is a really cool skill that people downplay for usually wrong reasons.

> It can't generate new things

It's hard to take the host seriously when they say things like this...

> The technology is fundamentally unpredictable

It simply isn't. When you find a seed that is generally close to what you want, you fix the seed and tweak (because you can) from there.

[0] https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite

I think the easiest way to get the gist of what he (and I) are arguing is a thought experiment. Take an LLM back to the dawn of mankind, and train it on all of the collective knowledge of man. This would actually be impossible, because language hadn't yet been invented, so let's just imagine it could somehow read the collective mind of humanity. So cutting edge technology is poke 'em with the pointy side, science is nonexistent, and knowledge isn't that far behind.

Now run this system through our LLM for as long as you want with as much power as you want. Where is it going to take you? Nowhere really, it's still just going to be constrained to its pool of data, recombining things in primitive ways and be essentially permanently stuck in the present, until somebody gives it some new data to train on it and mix and match. Yet somehow humanity, starting from that exact same basis, would soon (relatively speaking) put a man on the Moon, unlock the secrets of the atom, discover quantum mechanics, invent/discover mathematics, and much more.

This is what is fundamentally meant by LLMs cannot create "new" knowledge. They absolutely can mix and match their pool of training statements in ways that can generate new statements. But if it's not an extremely simple 'remix' and we're in a domain where there are right and wrong answers, there's good chance it's just a nonsensical hallucination.

You are comparing a species going from developing language to building spaceships on the order of hundreds of thousands of years, to a technology that was invented 7 years ago and only started to really be taken seriously 4 years ago. Working memory systems required for learning and iteratively developing ideas are in their infancy (on the scale of recent developments), but the recall technology (vector databases a la RAG) is quite well proven. I see no reason that a language model couldn't do iterative science in the same way humans have been with the resources available to it (APIs) using the composition of current technologies.

The current technology for LLMs is still a rather complex guess the next word algorithm. But this leaves the software with no real room to ever move beyond its training, regardless of how much time or training you give it. You could give them literally infinite processing power and they would not suddenly start developing new meaningful knowledge - it would still be little more than simple recombinations of its training dataset, until somebody gave it something new to train upon.

> I see no reason that a language model couldn't do iterative science in the same way humans have been with the resources available to it (APIs) using the composition of current technologies.

I think this belies a common refrain from people in non-tech feeling like people in tech tend to oversimplify their fields and claim they can "fix any problem" without any actual specific knowledge in that field. This just feels like the next iteration of that.

This seems like a lot of words to just say "you are wrong". Can you explain why? What features of the scientific method cannot be achieved through the composition of existing technologies? Feel free to be specific and use technical terms.

I think you're misreading this. I'm not saying "you are wrong". I'm saying there's a common trap that a lot of people in tech (and therefore a lot of people on HN fall into), where they believe that they can solve any problem from what they know of basic tech problems.

> What features of the scientific method cannot be achieved through the composition of existing technologies?

^This statement, sums up the above trap perfectly. "I know how to create a workflow, therefore, I know how to create a workflow of the scientific method, therefore I can replace a scientist with an AI robot."

But okay, how do we know what quantitatively, is a significant result? R=0.05 is the most common cut-off for this, but like, that's a number we picked, and isn't even agreed on.

LLMs can absolutely create synthesize new knowledge out of existing knowledge. They can't easily do so iteratively because we haven't quite figured out memory yet. Until we figure that out you won't have an LLM discover a new theory of quantum gravity.

And even once we solve that, LLMs - just like human scientists - absolutely need new data from the outside world. Very few breakthroughs were achieved by just thinking about it long and hard, most were the the result of years of experimentation. Something LLMs simply can't do

Recursive self improvement requires the ability to objectively select the "prime" of your own knowledge. From an LLM's perspective a hallucination and a correct answer are the same thing. It does not have any beliefs about what is true or false, because it has no concept of what it's even outputting. It's all just guess the next word. So even if the hallucination completely contradicts countless things it ostensibly "knows" to be true, it is unable to correct itself or realize that what it's outputting is unlikely to be correct.

Pause tokens [1] are directly dependent on the model recognizing whether the answer it arrived at is one it should "commit" to or whether it should abandon it and output a pause token instead.

Similarly if you ask ChatGPT about the current president of Numbitistan it will tell you that it doesn't know about a county with that name, rather than just hallucinating an answer. So it can at least in this circumstance tell the difference between knowing something and not knowing something.

1: https://arxiv.org/abs/2310.02226

The same is true for a human brain in a vat. It's even true for humans historically, it took us millenia to figure out science.

When robots are powered by transformers or the like, I expect we'll see some pretty impressive results.

I don't think this is true - the main difference is internal consistency. Humans adopt a series of views and values, true or false, and tend build up from those. The accuracy doesn't really matter so much as the internal consistency, because it tends to turn out that trying to build from an invalid foundation eventually causes you to stop moving forward, and so the more factually supported values tend to win out over time.

But it's the internal consistency that really matters. LLMs have no internal consistency, because they have no way of 'adopting' a view, value, fact, or whatever else. They will randomly hallucinate things that directly contradict the overwhelming majority of their state, and then do so repeatedly in a single dialogue. If there were a human behaving in such a fashion, we would generally say they had schizophrenia or some other disorder that basically just ruins your ability to think like a human.

Humans are infamously bad at being consistent:

https://en.m.wikipedia.org/wiki/Compartmentalization_(psycho...

The biggest mistake I see people make when criticizing LLMs is that they take the best possible modes of human thought from our best thinkers, and compare that to LLM edge cases.

Accuracy vs consistency isn't really a delineator. There's so much low-hanging fruit atm, like world models for LLMs improving drastically if you just train them longer. I'll believe the naysayers if say in 5 years GPT-4 is still near state of the art. Until then, there doesn't seem to actually be any theoretical limitations.

Hallucination is not an LLM "edge case." It is their normal and only state of operation. It just so happens that 'guess the next word' algorithms are capable of a reasonable frequency of success owing to the fact that a lot of our language is probably mostly redundant, making it possibly 'hallucinate' reasonable statements quite regularly.

Take what I wrote above. If you were given the context of what I have already written, then you could probably fill in most of what I wrote, to a reasonable degree of accuracy, after "It is their normal..." Because the issue is obvious and so my argument largely writes itself. To some degree even this second paragraph does.

IDK, I think it's kinda important for LLMs to get the simple stuff correct in order to justify looking into the rest of the hallucinations.

Like, if you can't tell me "what day is it today?" (actual failed prompt I have seen) then there's no world where I'm going to have a more complicated follow-up conversation with you. It's just not worth my time or yours.

I agree with this, but find it a poor mode of debate. Because it results in a hole-plugging which is then called goal shifting, even though it's not - but rather a lack of precision in the goal to begin with. For example imagine it goes viral that 'wow look LLMs can't even play a half decent game of chess.' So OpenAI or whoever decides to dedicate an immense amount of post-training, hard-coding, and other fun stuff to enabling LLMs to play a decent game of chess.

But has anything changed? Well no, because it's obviously trivially possible for them to play a decent game of chess (or correctly assess the date), but it's an example of a more general issue of LLMs being generally incapable of consistently engaging in simple tasks across arbitrary domains. So you have software that can score some high thing on the LSAT or whatever, but can't competently engage in a game children can play.

The over-specialization for the sake of generating headlines and over-fitting benchmarks is, IMO, not productive. At least not in terms of creating optimal systems. If the goal is to generate money, which I guess it is, then it must be considered productive.

I'm not asking for someone to overfit to being able to properly answer the question of "What day is it today?" I'm giving an example of a simple question that all LLMs need to be able to answer correctly.

But like, people are on here saying that this will make scientific improvements, and until it can get past the basic stuff, it's not in the ballpark of anything more complicated. Right now, we're basically at the stage of 10 million monkeys on 10 million typewriters for 10 million hours. Like, maybe we'll get Shakespeare out of it, but are we willing to sort through all of the crap it will generate along the way, when it can't actually create a useful answer to simple questions?

Why are humans capable of doing that, but it's categorically impossible for LLMs? That's a very high level capability. What's the primitive it is built on that humans have but machines can't have?

Quantum biomechanics. Human brain has built-in indeterministic pattern seeking. Computers are Turing based which 100% dependent on their programming and data inputs. All current LLMs operates on that very deterministic hardware. In order for LLMs to break thru existing glass ceiling, the hardware need to change.

There's more to AI than LLMs though, like Alpha Fold figuring the structure of 200 million proteins for example. You can link those systems to your LLM so you can ask it what do you think of this protein and what experiment could I do on it and so on. There are many such avenues to explore. I don't see things getting stuck at a maxima for quite a while.

> Take an LLM back to the dawn of mankind, and train it on all of the collective knowledge of man.

... and that is the start of your argument. Could it also have gnomes and robots?

That's not the start of an argument. It's the start of them explaining their point.

Explaining a point is part of an argument. And the premise is unsound and illogical and if we had an LLM firewall their post would have been blocked until they cleaned up their reasoning.

"Anyone who has played around with an LLM would agree that they are able to demonstrate some level of both knowledge and reason"

Completely disagree, having used them quite a bit. They are able to regurgitate knowledge, but have no ability to reason

I've used them extensively and they are very able to solve the problems that sometimes appear in my code, Claude 3.5 Sonnet in particular is able to not get stuck in an infinite loop causing the same error, and able to rewrite the entire code if it is necessary, it is surprisingly intelligent.

To me the ability to reason is the biggest difference you can currently observe between 7B and 70B models.

People love to test this with "brain teasers". You could argue that LLMs can pattern match "what's heavier? 1kg of feathers or 2kg of steel", but there are enough examples of at the time novel puzzles that I feel confident to say that good LLMs can absolutely reason (at small scales, but it's getting better)

What does it mean to be able to reason to you?

To think, to understand context, and to intelligently explain something. To understand WHY you are doing what you are doing. That is not what an LLM is doing.

> To think

You can think without reasoning. Recalling knowledge does not require reason.

> to understand context

Can you break this down? What does it mean to understand context?

> to intelligently explain something

LLMs can do this.

> To understand WHY you are doing what you are doing.

You can ask an LLM why they do things they have and they may reply, even with an answer that is likely correct. This again depends on how you break down "understand".

Humans can't explain why we do things, so I guess we're not intelligent either?

Yes we most certainly can

We can invent post hoc rationalizations, just like LLMs can

Humans do lots of things about reasoning. How many times have you heard:

"We've always done it this way."

"It's tradition."

The daughter asks, "Mom, "why do you cut the end off the roast before cooking it?"

"I learned that from your grandmother," replied the mom.

The daughter turned to her grandmother and asked, "Why did you do it?"

Grandma rolled her eyes, sighed, and said, "My oven was too short for the roast."

Humans often do reasoning poorly, or choose not to use reason, or have a mistaken idea of what answer reasoning would give. They still have the ability to do so, which LLMs do not

> Do not reply to this message.

> I understand. I won't reply to that message.

> You replied. Why did you do that?

> I apologize for the confusion. You're right that I did reply, which was inconsistent with your initial instruction. This highlights an important limitation in my capabilities - I don't actually have the ability to simply not respond at all to a message. My training is designed to always generate some kind of response, even if that response is acknowledging an instruction not to respond. I should have been clearer about this limitation rather than incorrectly implying I had followed the instruction. Thank you for pointing this out, as it's a good reminder to be more precise about my capabilities and limitations.

I just had this dialog with Claude 3.5 Sonnet. Is this not a demonstration of exactly what you're describing as impossible for LLMs?

please give me evidence that they don't have the ability to reason. I'm not sure they can reason but I'm not ready to rule it out yet. we know how biological entities demonstrate reason, how would a non-biological entity demonstrate reason especially in the context of no body or senses

I think the AI bubble may have some interesting parallels with the dot com bubble ~25 years ago.

The internet was revolutionary and transformed the global economy. However, most of the internet companies at the time were garbage and were given money because people were blinded by the hype. At the end of the day, we were left with a handful of viable companies that went on to great things and a lot of embarrassed investors

I think that’s a great analogy (and I was doing software then).

We know machine learning is a big deal, it’s been a big deal for many years, so of course recent breakthroughs are going to be likewise important.

The short term allocation of staggering amounts of money into one category of technology (Instruct-tuned language model chat bots) is clearly not the future of all technology, and the AGI thing is a weird religion at this point (or rather a radical splinter faction of a weird religion).

But there is huge value here and it’s only a matter of time until subsequent rounds of innovation realize that value in the form of systems that complete the recipe by adding customer-focused use cases to the technology.

Everyone knew the Internet was going to be big, but the Information Superhighway technology CEOs were talking about in the late 90s is just kind of funny now. We’re still glad they paid for all that fiber.

And a lot of the products that ended up mattering were founded in the decade after the dot com bubble: Facebook 2004, Youtube 2005, Twitter 2006, Spotify 2006, Whatsapp 2009, etc.

A hype bubble is great to pump money into experimentation and infrastructure, but the real fruits of that typically come later when everything had a chance to mature.

A similar thing happened with computer vision and CNNs. There was a big hype when "detect if there's an eagle in this image" turned from a multi-year research project to something your intern could code up on the weekend. But most of the useful/profitable industry applications only happened later when the dust was settled and the technology matured.

They were garbage in hindsight. Being "blinded by the hype" is what drives people to try new things and fail. And that's okay! It's okay that we ended up with a handful of viable companies. Those viable companies emerged because people tried new things, and failed. Investors lost money because investment has the risk of loss.

From a business perspective this is right. Unless OpenAI creates AGI they'll probably never make a dime. Great products do not lead inevitably to great profits.

I think the focus on AGI is misguided, at least in the short run. There's profit to be made in specialized intelligence, especially dull, boring stuff like understanding legal contracts or compliance auditing. These AI models have plenty of utility that can be profitably rented out, even if their understanding of the world is far short of general intelligence.

Even just replacing 10% of first-line customer service is a gigantic market opportunity.

Everyone tried the first time by adding stupid menus that you have to navigate with numbers, then they made it recognize spoken words instead of numbers, now everyone is scrambling to get those to be "intelligent" enough to take actual questions and answer the most frequently occurring ones in a manner that satisfies customers.

And if they do create AGI, it will have the ability to say “no”, which is going to be quite a bummer for the investors.

Language models have been able to reject prompts for years.

I can't get enough of Ed Zitron. I first found him on the Scam Economy podcast, and now I'm addicted to his blog and appearances on Crypto Critics Corner or any other podcasts.

In case anyone hasn't found it yet, I've been enjoying his newest podcast, Better Offline. Currently only about 28 episodes in.

https://www.iheart.com/podcast/139-better-offline-150284547/

Never heard of him. What is he selling that you can't get enough of him?

He thinks tech industry has lost its charm and only making useless products, and has become hostile to users. I agree. In one episode, he talks about Prabhakar Raghavan, who has made a mess of Google search.

You might not agree with him all the time, but he has some good arguments and seems sincere in his criticisms. Better offline is worth a listen

Big tech has hit the IBM stage of their lifecycles, there will be a new Microsoft there in the next couple of years to reset things.

The podcast/article: https://www.wheresyoured.at/the-men-who-killed-google/

John Gruber has a nice tldr: https://daringfireball.net/linked/2024/04/24/zitron-google-s...

Strongly stated and flamboyantly expressed opinions that confirm the biases of the OP.

Zitron's posts have been hitting the frontpage regularly this year [0]. They don't tend to stand up to any kind of close scrutiny; the facts are made up or misrepresented or the logic is faulty. Why would they stand up? He's a PR flack turned influencer, not somebody with the expertise to actually reason about technology. But a lot of people (on HN and elsewhere) hate big tech, and Zitron will happily tell you that big tech is doomed while simultaneously engaging in a lot of entertaining name calling of tech execs.

[0] https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

Ahh so the standard anti tech grifters (usually packaged for tech workers that are "totally not like the other tech workers") that are overly cynical, snarky and out of their depth.

It's a shame because that whole genre had a very good standing when it came to crypto. But then that died off, and they still had to "debunk" and "dunk" on everything so now they just sound unhinged. NFTs were ridiculous so everything that touched NFTs was easy to ridicule imo, but they still have that same overly snarky/ridiculous/two more weeks approach to AI, with very little technical knowledge or depth to what they say. That's fine for crypto, because you don't need more than surface level knowledge to know that the concept is flawed (without knowing the detail of what a smart contract is, or how it's implemented), but when that approach is used on something that people actually use and see the progress in a tangible way, it just sounds unhinged.

Not that "openai API AI bros" aren't the mirror image of those grifters, with the same exact lack of technical knowledge and a shallow understanding of the stuff they keep talking about. Just that they grift from the hype, not the cynics

The AI bubble folks are sitting in roles that are not doing the cutting edge AI work at companies that have figured out what to do with it in the first year that it’s been practically useful. I sit in such a role and I know we have done absolutely amazing things that have significantly improved our margins. The key though is there is no “manual” or playbook yet, and anyone doing substantially advantageous work is holding on it as a trade secret for now.

This is the fastest I’ve seen any technology deliver meaningful value in my 35 years in tech and with much less established practice than most. It’s going to take years to fully bed down all the uses, the tooling to make it usable in various contexts, and for the basic technology itself to reach a more efficient and effective technique.

There is almost certainly a bunch of ventures that started too early. A bunch that are misguided. A bunch that will contribute a lot of understanding but will disappear having evaporated investor money. But that’s the way new foundational technologies -are- . That isn’t a bubble that’s how new markets behave.

The challenge is it’s hard to distinguish when rational growth and a bubble sets in. It feels hard sitting in my seat seeing the varied ways we are using this technology as a bubble yet though. There’s a lot of productive work and research and investment to go before we get there.

> The AI bubble folks are sitting in roles that are not doing the cutting edge AI work

Ah, the "trust me bro" argument. No one I know of, including Zitron, denies that this tech will increase margins by some number of percentage points.

But that's not the game the core AI guys are playing. They're raising money based on the promise of creating a digital god.

I see significant improvements nearly every other week or so, and I feel like I'm barely following the industry. He's making an "any day now" argument and is not quoting what AI companies are doing or dealing with. For example, improving training data has been delivering massive improvements vs. "gathering more". Identifying and iterating on approaches to dealing with recognized flaws is another area delivering massive improvements. Lots of basic research is still being done.

You don’t have to trust me. I’m just stating my first hand experiences. If you find me someone untrustworthy then feel free to stick with your own biases.

From what I saw in the video, Ed is the “trust me, bro.” Many recited popular knowledge arguments and no boots-on-the-ground industry experience. This guy is a literal influencer.

AI in its current incarnation felt on part with word processing, or even spreadsheet (which powers a big portion of the financial industry, so no mean feat)... but it's not the internal combustion engine, or comparable to the discovery of electricity.

Ok? But how much value did the spreadsheet give the modern economy? Almost every person learns how to use word processors and spreadsheets in any school that has any semblance of modernity, and it’s required in every job that has even marginal knowledge work.

I’d also note we are only a few years into practical generative AI models and less than that in terms of how to integrate and use them. So… hard to tell what the end is going to be like at this point isn’t it? In 20 years of improvement and refinement, will it rival IC cars ? Who knows. But who cares. It’s still amazing.

I mean can you really compare anything with the discovery of electricity? If someone invented warp drive right now it still wouldn't be on par.

When it appears, AGI will certainly either be comparable or incomparable on the other direction.

Computers are a comparable innovation. The internet is a comparable innovation. Or, if you decide to go back, mechanical engineering was comparable too.

This consistent lack of self-awareness across tech is getting ridiculuous. From Musk to pg's past occasional oblique defenses of Altman. It's gotten to the point where we need to replace the pledge of allegiance with Upton Sinclair's quote:

"Don't expect someone to understand something when their salary depends on them not understanding it."

Don't get me wrong what you're saying may be 100% true. But the sentence immediately after your stated your expertise should be, "Don't take my word for it, verify it yourself."

This is where you should have checked yourself and realized something was off with what you were saying, as this description fits the dotcom bubble rather well.

Your are not providing any specific evidence, just stating an argument from authority.

Yeah the hucksters are selling hard. I don’t buy their end vision is as easy as they claim it is. But I also don’t deny that at least in Musk there’s been a history of doing amazing things about 5 years later than he claimed he would. I don’t like the man but I am amazed at how he took our inability to dream any more and turned it on its head.

You can verify it yourself, but don’t expect the out of the tin stuff with a web UI and a iPhone app to be where the power lies. Those are demo ware. They’re literal toys. The current state of the art use requires a lot of trial and error, struggling with shitty abstractions, lack of knowledge or draw on, and a lot of fumbling with a brand new technique let alone technology. It also requires understanding how generative AI works, loss functions, a fair amount of intuitively understood math, and some really creative ideas. It’s all more or less open source or where it’s not the per api fees are reasonable and there are cloud services you can use. It’s all possible it’s just not -easy- to verify. And - since I’ve done it myself and seen its power first hand in what I do I actually don’t have a lot of drive to convince others. Everyone will see sooner or later as the abstractions and the techniques mature.

As such I don’t feel compelled to build a body of evidence in the least. Don’t believe me - ok! Don’t want to really dig in and verify things at the cutting edge of work being done because it’s a lot of work? Ok! But likewise, asserting the contrary without having done that work is just as fallacious, and doesn’t change my first hand experiences in the least.

Since my comment was alreadly long I didn't want to add a long parenthesis to elaborate the details with Musk, but I wasn't referring to his concrete accomplishments, which I don't deny and am also irritated when others needlessly deny. I am referring to his ridiculuously and transparently hypocritical 180 change in behavior after he meets with leaders of countries he was criticizing prior to meeting them. He has not tweeted anything remotely negative on China and Israel after meeting Xi and Netanyahu, clearly due to transactional quid pro quo he was able extract from those meetings regarding his business interests. I have no doubt his railing against the Biden admin would have been tempered had they been more accomadating to his business interests. It is abjectly pathetic and spineless behavior.

GenAI skeptics and evangelists spend a lot of time talking past each other. The way I see it is that there are three camps, which you can see in the comments in this thread.

Camp 1 is the Ilyas that are convinced they're building a digital god before 2030.

Camp 2 are people that see a very useful tool that's going to change the world for the better.

Camp 3 (my camp) thinks Camp 1 is getting high on their own supply. We often talk to Camp 2 like they are Camp 1, but we shouldn't. We think GenAI is very useful but will likely in the end be used for negative purposes, the way the Internet was perverted into a surveillance and disinformation machine.

Reading the comments in this post, I definitely see Camp 1. I could have a conversation with Camp 2/3.

I'm more skeptical than Camp 3, because of the large cost associated with generating this information. Like, what would AI be used for that we weren't already able to do with the existing technologies we had.

I don't see a market for GenAI. It seems like a gold rush but without any gold. How do companies turn a profit off of this when all of the investor cash faucets dry up? I see how the shovel companies (nvidia, amd) make money. I could see how cloud companies could make money if they weren't "getting high off their own supply."

The number of companies I've seen with major PR blunders from AI being added to their product, and immediately hallucinating something awful, tells me the market for this is just investors chasing their own hype.

I feel like I sit between camp 2 and 3. I think there will continue to be very massive negative consequences, but as someone lucky enough to be able to run LLMs locally, I think it will change my world for the better, and improve how I interface with knowledge on the internet.

FYI your comments all look like Camp 1 to me. You come off very strongly evangelizing AI in this comment section.

I'm sorry you feel that way I guess. Maybe try re-reading my comments as someone who is between camp 2 and 3.

"The crypto bubble is bursting" - Hacker News in 2011

2009 https://news.ycombinator.com/item?id=855684

2017 https://news.ycombinator.com/item?id=14790251

2018 https://news.ycombinator.com/item?id=17184054

2019 https://news.ycombinator.com/item?id=21613997

2022 https://news.ycombinator.com/item?id=30622300

2023 https://news.ycombinator.com/item?id=35091237

> They'll run out of data, and the models won't be able to get any better.

Well, that is enough to prove to me he doesn't know enough about the technology for me to pay attention to his opinions...

"What can be asserted without evidence can also be dismissed without evidence"

By this razor, I feel like I can dismiss your opinion. Can you elaborate why you think he doesn't know enough about technology for you to pay attention to his opinions? Is it cause you have proof that this is false? Cause it's predicting the future, there are an infinite number of ways it can go.

How much of existing data did they devour?

- internet (incl. blogosphere, all www, forums, social networks, dark net,...)

- all of digitized books

- all statistical data, governments data, military data

- all scientific papers and books

- all commercial companies data

- all private data/personal data

- ADDED - all video/audio data

- all new data...

If someone has some links on what the current state of affairs is (what they really trained on), I would be really interested.

Training data is the competitive edge, so everyone refuses to elaborate about what they use. Also they probably don't want to invite anyone to sue them over it.

I think there's some quote that GPT-4 is trained on a significant fraction of the entire internet, whatever that means.

Beyond that it's hard to tell. If I were in charge of curating these data sets and not afraid of lawsuits or ethical concerns I would certainly add the entire contents of library genesis, anna's archive, sci-hub, etc. I'd also get a deal with the guy who has scraped 5 billion discord messages (he even explicitly offers the whole dataset for LLM training purposes).

Those are the low-hanging fruit. There is still a lot of knowledge that isn't on the internet or that isn't accessible without an account. But that's mostly a question of your budget

Regarding their ethical concerns or lack thereof, OpenAI Whisper will often hallucinate transcriptions such as "Subtitles by <random nickname>" which strongly indicates that it's trained on ripped movies and crowdsourced subtitles found online.

> Training data is the competitive edge, so everyone refuses to elaborate about what they use.

I absolutely understand it, thus curiosity about some leaked insiders knowledge or anything. Anyway, given availability of these huge data sets (incl. ones you mention) and advancements in processing techniques, we probably have a long way to go. Also, video looks like two guys talking about everything except AI bubble.

On this point you're probably right.

Post your counterargument. This is HN after all.

Counter argument: The data necessary to train these models has been around for a while, but didnt lead to a breakthrough. What lead to breakthrough performance was new model architectures, specifically attention mechanisms.

There is nothing that says you can't make much better models using the exact same data by changing the architecture of the network. In fact, that's exactly what we've been doing for the last 15 years.

I particularly liked this interview on Ed Zitron’s own show: https://overcast.fm/+BGz6-w7jjI

I’m so tired of people talking about AI like this but only looking at it from the perspective of a consumer. AI is so big because it allows people like me, who only dabble in code, to automate with very little learning curve.

My life has been irrevocably changed by AI and I’m tired of people saying it hasn’t done anything when reality is that they just don’t know what to use it for.

Can you share an example of what you have automated or plan to automate using AI?

I wrote a python script that checks a folder on my Dropbox for m4a files, then calls whisperapi to transcribe them, then reformats exactly how I need them reformatted, and finally moves them to a syncthing folder.

I took maybe two python classes about 20 years ago. I don’t know that language.

I also wrote two python scripts that use imagemagick to automate two different resizing/reformatting jobs for large groups of images for my work. Saved hours of manual labor.

And it’s not just the finished code. It’s also learning how code works in general. I’d have stupid questions/concerns all the time that I could just straight up ask AI and they’ll show me how I was thinking about it all wrong.

It's beyond "automating"; it's being able to go from idea to prototype without having to have a bunch of underlying knowledge.

I feel like there's a subset of "awesome programmers" who are going to be the people run over by AI, because they seem to be the ones who are saying, "why would I use this?"

We can't all be good at everything; the fact that I can use AI to catch up in some places means I have a leg up in others.

The underlying knowledge you can skimp on right now is the most absolute basic understanding of a simple problem. I just don't see things like replicating a bare-bones version of 2048 in HTML as "catching up" to anything worthwhile.

For busy-work tasks like restructuring JSON as YAML and writing one-off bash scripts, current AI is amazing. Code review in limited contexts is also pretty decent.

But solving actual, difficult problems seems frustratingly out of reach. GPT5+ really need to deliver much better logic and problem solving.

If you're looking at 2048, thinking you don't need one, and concluding it can't do anything worthwhile because it can't do <whatever problems it's failed at for you>, you're not seeing the forest for the trees. Yet another 2048 clone is itself, not super interesting, and because of that, there's very little economic value in creating one. An yet the person who created this 2048 clone was able to create it over a couple of hours of not focusing too hard on if, with the help of ChatGPT. The real question is how many computer programs as tools don't exist because they've previously been too expensive to develop. There's no way I'd pay someone else to develop a 2048 clone, because I could just write one myself, but if I wasn't capable of programming and needed an important workflow tool as, say, a graphics designer, I could muddle through it using ChatGPT, or I could pay someone else to do it, at a rate that was previously too expensive. Senior devs at FAANGs aren't going to be doing a whole lot of that at the rates they can command, but that's a whole lot of new programming work that existed but want getting satisfied prior to LLMs aiding programming.

That's what that anecdote* is about, not how invaluable someone else's 2048 clone is to you.

* https://news.ycombinator.com/item?id=40869881

Can you please share an example of one such difficult problem? It will be useful for me, thanks.

Having it generate really complicated code involving lots of math and physics typically fails.

What problem domain are you interested in?

Going with my 2048 example, how about "add a fun meta mechanic to the 2048 difficulty progression." Or maybe, "make a multi-player version of 2048." Or, "add a physics gameplay interaction to the 2048 controls."

I mean, just imagine doing anything potentially cool or interesting and that's your difficult problem that current AI can't help with.

Thanks!

I am interested in difficult programming problems that require logic or problem solving skills that AI wouldn't be able to handle well. In my experience with using AI for programming the quality of prompt affects the quality of the output quite a bit. I'm not sure if someone has done open research on problems that it can handle and cannot viz a viz quality of prompts.

It’s not just software folks who are going to be run over. I use it to answer all sorts of questions I have in a variety of fields I know nothing about. It truly is a deflationary force.

These types of takes seem pretty ignorant of the percentage of workers and students that are using artificial intelligence tools daily.

It is ignorant because it ignores any economical improvement that comes from say a machinist figuring out gcode issues, a culinary student learning safer fermentation etc...

This is pretty typical of finance or public relations people trying to make their thinking easier by only focusing on one variable at a time.

Don't know exact data, but can referr to Peter Diamantis quote:

```A survey by Microsoft and LinkedIn revealed that 85% of Gen Z uses AI at work, followed by Millennials at 78%, Gen X at 76%, and Boomers at 73%.```

Whether you want to believe thator not, you can think for yourself on how getting chatGPT to successfuly fix your espresso machine by telling which $0.35 gasket to replace... has had a positive growing booming impact on the economy or not.

To me it looks like it has and will continue to do so despite the venture money laundering/border-line-fraud issues that are clouding the picture

Is "students using AI daily" really a positive examples you want to reach for, when the primary application there is students getting ChatGPT to write their essays so they don't have to actually learn anything or think critically about the material? Inventing a way to figuratively copy paste from Wikipedia but with a much lower chance of getting caught isn't a net good just because it makes MAU numbers go up.

> students getting ChatGPT to write their essays so they don't have to actually learn anything?

You know it’s funny but my math professor in high school had the same argument when I modeled a calculus problem in my computer to arrive at the answer numerically instead of analytically.

In elementary school calculators were banned because “students arent learning anything”

Then in high school and college Wikipedia was banned because “students arent learning anything, they’re just looking up facts”

But by end of college open laptop exams were popular because doing fast research during an exam is good actually.

Is writing repetitive uninsightful essays nobody wants to read a useful skill for humans?

PS: graphing calculators were super banned in high school, even after we started being allowed calculators. I hear these days graphing calculators are required. Progress marches on

Yes, learning to write is an incredibly useful skill. Learning to communicate effectively, how to synthesize information into a coherent essay, how to organize your thoughts and put them down on paper, all essential skills. How is this even a question?

Yes and none of that is a high school essay. They’re an extremely poor teaching tool of effective communication.

As evidence I submit ChatGPT’s own waffly style of writing. It learned that from humans who on average are poor communicators. The “high school/college essay” style of writing is something new employees have to actively unlearn in their first few years in the workforce.

edit: If I was a teacher right now, I would give my students a 2000 word essay written by ChatGPT on $topic and ask them to redline the printout. This teaches them the actually useful skill of editing and fact checking since it looks like producing words has become commoditized.

Say what? That is exactly what a high school essay is. Many of them are of poor quality, because students need to learn. Learning algebra in school and getting a bunch of problems wrong doesn't mean you are not learning math. You don't learn to do anything without failing at it initially.

ChatGPT has a "waffly" style of writing because it is incapable of thinking, it's simply trying to predict the next word.

I realized we probably come from different educational environments. Maybe mine just wasn’t as effective as yours.

If you learned good/effective communications skills from high school essays, kudos! I did not, but I did enjoy the process and writing essays was one of my fav activities in school. Just that the “effective” part came way later :)

You were indeed learning to communicate more effectively. Maybe more slowly than was possible if you were in a very poor educational environment, but you were learning. And more importantly and in context: you were learning in a way that AI/LLM cannot.

Let's try to ensure that no one else has to learn in a substandard environment instead of abandoning teaching kids how to write and communicate and replace it with something that can never do it as well as a human.

"producing words has become commoditized"

It absolutely has not

It definitely has. It’s been commoditized for years actually. Just go on upwork and see the sad rates that copywriters ask for. Producing words is super commoditized.

Producing good words not so much.

I mean, I assumed that "good" was implied. Anything can be done poorly.

I’d argue that if writing is an algorithm that has been cracked, is it important to learn anymore? I have to imagine when the calculator became popular it was a similar debate. I do wonder how we should measure student understanding next

I am a professional technical writer. I find ChatGPT to be a poor writer, and a very poor technical writer. Only minimal reasoning or critical thinking is in evidence, but I find that humans cannot see the missing critical thinking when reading what ChatGPT writes. Claude is a little better, and much better stylistically. I think thinking has already fallen by the wayside in the general population; when there is writing that evidences little thinking, it is not writing that is the issue.

Is it? I seriously doubt it, and writing essays that get skimmed, graded and thrown into trash is a pointless task anyway.

People who want to cheat will always find ways to cheat. But if you genuinely want to learn something new then LLMs make it an order of magnitude easier, because they tell you exactly where to start, which resources to consider, giving you completely tailored answers to your problem or project, even if they're still oftentimes wrong. Google being increasingly useless in recent years also didn't help things.

I would not be surprised if in 20 years multimodal models are the main way to for most people to learn. There's always a lack of teachers, they're underpaid and forced to deal with an increasing amount of parent bullshit, half of them don't even have a good grasp of the topics they're teaching or just no longer give a fuck. Eventually there simply won't be enough of them and a personalized automated solution will just be better and cheaper. College professors? Probably in 10 years already and both them and the students will be happier for it, the former that can now finally focus on research and the latter for actually having a competent tutor instead of someone who has no aptitude for it but is forced to do it regardless.

> writing essays that get skimmed, graded and thrown into trash is a pointless task anyway.

Since when is writing essays and getting expert feedback pointless if you want to learn how to write?

> There's always a lack of teachers, they're underpaid

So rather than prioritizing teaching we should conduct a moonshot project to build a homeschooling technonanny that uses enough electricity to power the Eastern Seaboard so we can educate our kids without human interaction?

If you want to learn how to write, you mainly need to read more, and fixing grammar and punctuation is an automated thing nowadays. If you feel like you're getting expert writing feedback from a high school teacher then you won't be writing any novels any time soon anyway.

> we should conduct a moonshot project to build a homeschooling technonanny that uses enough electricity to power the Eastern Seaboard so we can educate our kids without human interaction

Actually, unironically yes since you only have to do pretraining once. A model that can then run inference locally on smartphones and is used for many years would amortize its creation cost though the utility it provides in a few years. It wouldn't even have to be AGI tier, just good enough to work with.

A single teacher educating 30 people vastly underperforms 1-1 tutoring. It's a "good enough" system and it doesn't work very well for most people, it's just the only thing we have. Hell most of it is just following a fixed script that repeats every year, answering a few questions and grading based on ground truth.

A personalized teaching system that knows your interests and skills would be able to motivate and explain far more effectively by adapting the script to you in pace and difficulty. Being available to you 24/7 for questions and having a personality you like would certainly help too. There's this 11 year old video [0] from the ex-teacher CGP Grey that turned out to be pretty prescient and elaborates a bit more on the general idea.

All of this could even be done sustainably, running training during peak solar hours, excess wind power and the like, but as long as there's demand it doesn't make sense to leave your expensive and rapidly depreciating GPUs idle. And there's demand because people have this misconceived notion that AGI is somehow winner take all, making it a race where everyone's flailing about like a headless chicken. In reality whoever gets there first will have a short first mover advantage for half a year and then everyone else will replicate their work, and probably an open source version a year and a half later tops.

[0] https://www.youtube.com/watch?v=7vsCAM17O-M

This is a perfect example of how AI hype breaks people's brains. It's a Bond villain scheme, not a way to improve public education. People learn best from other people and always have. Computers in the classroom have always been a net negative.

What do we count as "using AI"? How much AI do you have to use for work to be considered "using AI"? I used AI once for work 1 year ago, when it was popular to try, does that count?

I remember when Alexa was considered "AI" OR when alphago was considered AI OR when any algorithm was considered "AI". But these days, we typically mean "LLMs" .

So any claim without data seems pretty unlikely to be true unless they cite sources, and even then I would want to see methodology.

"a culinary student learning safer fermentation"

What could possibly go wrong?

Well reasoned, but citing a Microsoft survey is not informative.

> Gen X at 76%, and Boomers at 73%

Ok now those are some damn surprising numbers. At least as per the boomer what's-a-computer stereotype.

> which $0.35 gasket to replace

I’m sorry, but can it actually do that??

If some site out there has it, it can tell you.

Just like Google should, but haven't done for a decade.

There is no AI bubble.

What about the points raised in the video?

This is true of and only if they actually create AGI. Otherwise it's value destroying given the huge costs associated with producing what are essentially very good chat bots.

My cost is $20 a month for ChatGPT Plus, and the value I get out of that far exceeds $20. That is value creation, not destruction, right now. Not some imagined AGI future hypothetical.

I think the idea is that it cost a lot more than $20/mo/user to create the tech, and that investment in the tech-creation won't be recouped by your tech-consumption.

I think it depends on how far out you run the numbers for the area under the curve and all the various costs and profits. Seems clear to me that the individual tech-creation might end up being losses but that the net global will be gain very shortly as we continue to adapt and accelerate based on using the tech.

And OpenAPI is spending more than $20 to provide you that service...

Not to mention boiling the oceans

Sad state of affairs that people downplay this part of the problem. The energy usage is insane

How much energy is being used to provide that $20 service?

>Otherwise it's value destroying given the huge costs associated with producing what are essentially very good chat bots.

It's astonishing that anyone even tech-adjacent can say something like this.

I'd argue these tools are already beyond "very good chat bots". And you think this is it? This is as far as we'll get with them, or AI? That's impossible to believe looking at history. Besides the fact that laying miles and miles of "excess" fibre was once seen as a bubble...

> It's astonishing that anyone even tech-adjacent can say something like this.

Then I'll present my credentials. 25 years in tech and I use ChatGPT every day. It's an incredibly good chat bot and code snippet generator. The more I use it the more I see how much of a stochastic parrot it is. It's usually not that hard to find the code it based it's response on on GitHub and Stack overflow if you've got time to kill.

If you've got time to kill -- that says to me that it is faster, and therefore better. Time is money. How much time are we talking? If this is a 10x speed up on your research cycle then that sounds like a subtle revolution.

>Then I'll present my credentials. 25 years in tech and I use ChatGPT every day.

I think you're proving my point: long-time tech person who'd rather look stuff up on Stack Overflow, but yet uses the tool every day.

What? I use ChatGPT so I dont have to look it up.

And that's the point I've been making in this thread: GenAI is an incredible summarizer. But that's not nearly good enough to justify the investment and multiples were seeing now.

Only AGI can justify the money being laid out right now. Without AGI companies like OpenAI have negative value because they will always loose money.

I don’t know – I look at history and I see series of AI bubbles directly after a breakthrough that led to a great advancement in the science and nothing else. And then AI crashes and winters.

I think it’s completely possible to look around and think that this is as good as ChatGPT is going to get, or maybe it’ll get to 5.5 in a few years and peter out there (instead of here). And then we’ll spend the next decade shrinking and cramming and maybe we’ll even get 5.5 intelligence on our watches, but it’s never getting smarter than that.

Then in 2040 we’ll have another breakthrough, another hype cycle, and things will get smarter again, for another few years.

Looking at history, that’s a future I can believe in.

—-

PS. Generally I think these large transformer models are amazing, show great utility, and I use them every day.

Just ignore him. It's just like the people who refused to get a smart phone years after everyone had them. Some combination of being threatened by AI, not wanting to have to learn a new thing, and wanting to seem cool and "in the know" by rejecting the popular opinion.

Anyone that has actually used ChatGPT etc al can trivially see its value.

Yeah, like there is/was no crypto bubble.

AI actually does something useful. A boom isn’t necessarily a bubble.

Crypto does something useful, how else would you buy drugs and rocket launchers from the dark web?

Crypto does nothing useful for the average person who has an average amount of trust in their local bank. It has literally zero utility beyond gambling.

Yeah but that was already an established function by the time the public learned about Bitcoin. The recent crypto bubble is about speculation and web3 and NFT nonsense, none of which will last (while the drug buying probably will).

Sure. But the assertion was that it had no utility whatsoever and that's obviously not the case. Even tulips could be planted, despite them having had a crazy valuation.

I think it was pretty clear that they meant no utility beyond the original one that they had before the bubble.

I think I replied to the wrong comment

The crypto bubble has been bursting almost constantly from 2011 onwards while bitcoin has crashed from $10 to $50000. The AI bubble will probably keep bursting in a similar way. (bitcoin deaths https://www.bitcoindeaths.com/)

Like there was no social media bubble.

If you call everything a bubble, yeah, sometimes you'll be right. Something about economists predicting 10 recessions out of the last 5.

Dude, no one was using crypto for anything besides speculations. People are using LLMs quite a bit it seems. I do too occasionally.

If owners are using it preserve value because of lack of faith in USD, it’s not “just” speculation, that’s a valid value preservation strategy in uncertain economic times, and now a real part of the global economy. Full disclosure: I do NOT own any BTC.

Is there a transcript?

Yes. English (auto generated) transcripts are available in vid descriptions in YouTube apps

Thanks. If there isn't one on the web, anyone have a workaround to see it without an app?

You can use

https://tactiq.io/tools/youtube-transcript

This gets a transcript for this one https://tactiq.io/tools/run/youtube_transcript?yt=https%3A%2...

Or I tend to just click transcript on the youtube page and copy and paste all into my text editor. It's not a very good transcript though like just the words, not who said what.

Dunno if there's an AI version that's better?

Way to long and no bubble in sight

Ai is already changing plenty of industries...

My wife just bought AI skin cream. Definitely no signs of a bubble round here.

That’s just the sideshow. The bubble has yet to start: very few AI companies are publicly tradable through direct stock purchases.

Yea, we're still in the money raising phase. This should get interesting when investors start looking for an exit. Maybe by then everyone will have forgotten how the SPAC craze turned out.

Scammers and stupid marketing doesn't mean ai is a bubble

Like Oral-B's $300 AI toothbush![1]

No bubble! No bubble!

1. https://oralb.com/en-us/products/compare/electric-toothbrush...

The "AI" toothbrush, stupid as it is, actually pre-dates the current hype by years. Here's a review from 2019: https://www.forbes.com/sites/leebelltech/2019/06/30/oral-b-g...

A better example would be the "AI mice" that Logitech is pushing now.

Of course there are tons of useless PR parasites. Shareholders will demand to see a "response" or a "strategy" or an "answer" or an "AI play". If all you make is toothbrushes, you just have to milk the users the old fashioned ways: hardware purchase, subscription, and data sharing.

That said, I know of ANOTHER toothbrush company that has an "AI play"... :(

I've got a friend with one. They are actually good toothbrushes if not very AI.

Exactly, this is just stupid marketing and has nothing to do with real ai

What are some examples?

If in 10 years all we have are better chat bots and image generators, I’d say it was a bubble, and I don’t see anything that says that’s definitely not the path (though I’m not in the weeds of AI, so maybe it’s just not obvious, yet).

>If in 10 years all we have are better chat bots and image generators, I’d say it was a bubble

Sure, if in 10 years that's all we have, you win, it was a bubble.

I think the probability of that is a rounding error.

Alpha Fold 2 for example.

Whisper.

Image gen changes a whole industry right now.

The robot demos the last year.

By 2025 the majority of applications will use AI in some way (mostly to allow for sloppy user input), in 5 years there will be no non-AI applications.

For example, in healthcare (because... day job), you will be interacting with an AI as the first step for your visits/appointments, AI will work with you to fill out your forms/history, your chart will be created by AI, your x-ray and lab results will be read by AI first, and your discharge instructions will be created on the fly with AI... etc. etc. etc. This tech is deploying today. Not in a year, today. The only thing that's holding it up is cost and staff training.

Think about what you just said.

You gave examples of how chat bots are going to be more widely used. Nothing more. So far I don’t see any examples that aren’t overpriced efforts in “shoehorning a chat bot” into something.

Like why will a hospital pay for a bunch of chat bot integrations when it’s likely my ChatGPT phone app will be able to view the form via camera and AirDrop or email the form? Meaning, I still see no examples of why OpenAI isn’t the Bitcoin of the crypto bubble (one use case, with one winner).

You say the only things holding it up are:

- Cost

- Training

Which can be said of any business that’s ever existed. So why is AI different?

2025 is 6 months away. There is absolutely no way a majority of applications will use it.

So instead of just pointing me to non-existent Quicklisp packages, I can have a bot read the junk in my patient chart and hallucinate answers to pressing health care questions? I can't tell if this is a proposal or a threat.

That sounds like an absolute nightmare

As with previous AI hype waves, real transformational change happening does not contradict there also being excessive hype and a bubble which eventually bursts.

Reply on news.ycombinator.com