YC's Winter 2024 Demo Day confirms that we are indeed in an AI bubble | Hacker News Reader

YC's Winter 2024 Demo Day confirms that we are indeed in an AI bubble

techcrunch.com

71 pointsjshchnz65 comments

If you look at total software spend in the US a year it’s 500B, total spend on payroll is 5T. If companies become 10% more efficient because of GenAI you’re effectively doubling the size of current software market.

Actual value capture would obviously be lower, but 10% efficiency gains is a low estimate based on the studies coming out.

There’s a ton of terrible startups right now, but some of them will become whales. You can’t predict which ones ahead of time, so you invest in everything.

I haven't seen LLMs do anything more useful than generating a wedding speech in the style of H.P. Lovecraft and there are people out there earnestly predicting this technology will increase efficiency 10% across entire economies. I genuinely feel like I'm living in a different world to you all.

I have yet to see an example of Copilot do something that doesn't seem like basic autocomplete/snippets that editors and IDEs have been doing for decades or code so basic that it betrays the lack of competence by the user.

I find the Copilot completions to be slightly better than IDE auto complete, but the latency is too high for it to be useful IMO. It's to big a break in concentration to wait for it to type almost what you were about to.

IMO chat-based question-asking when encountering unfamiliar language constructs or framework idioms is where LLMs shine in coding assistance.

> I find the Copilot completions to be slightly better than IDE auto complete, but the latency is too high for it to be useful IMO. It's to big a break in concentration to wait for it to type almost what you were about to.

This matches my experience perfectly. It gets decent/usable results like 40% of the time, but latency kills it.

> IMO chat-based question-asking when encountering unfamiliar language constructs or framework idioms is where LLMs shine in coding assistance.

I have never found this to be productive. It gets details wrong frequently enough that I can't trust the output which effectively doubles (or more) the cognitive effort because you have to validate that it's correct against a more reliable source, or find out the hard way that it's wrong when code/systems break.

Interesting, what systems are you asking questions to? I find GPT-4 is pretty much always correct about simple questions I ask. I just avoid using it for anything really deep and refer to documentation instead.

I've mostly used GPT-4 and Gemini Pro. For simple stuff, it can be correct, but it highly depends on the topic. I've found it very effective as a dictionary or thesaurus but for technical questions, even simple ones like small help with a Scheme macro, it's near useless. I imagine if I was asking it about, say, Python/Django APIs or basic HTML/webdev stuff, it would do better.

When you say technical questions do you mean writing code, explaining code, or something else? If it's writing code I agree, I don't often ask it to explain code unless I want to know about some obscure syntax, but it works pretty well for that. I would not trust it to explain complex logic in a function.

I've tried all of the above. I've asked it to summarize information about what languages in a class had strong symmetric multiprocessing characteristics and it made up over half of the details. I've asked it to generate code for a simple linked list and it created garbage. I've asked it to summarize what code does and it gets it blatantly wrong.

These are just a few examples. I've found it to be nearly useless for anything where a semblance of correctness matters. Every single thing it says must be validated against external sources and it can easily lead you down rabbit holes.

GPT-4 was a little better than 3(.5), but I've yet to see it be good at anything enough to warrant the price tag or the time.

The auto complete is much more effective than what IDE has...

Even if you type at 140wpm it is effective.

Do you not find the latency disruptive? I find that the auto complete results taking even a second and a half breaks my flow, so I disabled Copilot and I don't miss it

There are some UX concerns I wish could be improved, but I think I have been able to fit decently with the rhythm of it and know when it is efficient to use for me. I can't write multiple lines in a second. In some cases it will give me the exact 10 lines of code I wanted to write myself with few seconds. That is significant.

It should be better at auto importing and status indication I think.

Also it never makes typoes or rarely index+1 errors, things like that.

It is much less mental fatigue compared to if I have to make sure I did those things correctly myself.

I think the delay I kind of in my head use like a guessing game of what it will exactly give me or I will plan other things ahead.

Because it is also always interesting to see how accurate my guess for what it will give me is. Guess it is kind of fun.

It can write code. I can assure you that. Still need a developer but it can make a developer incredibly more efficient much much more than 10%.

He was saying that he expects to see a 10% increase in efficiency globally, not amongst programmers.

Have you used GPT4 as a drop in replacement for Google, especially while coding?

I don't have to constantly Google things while coding.

I didn't imply constant Googling. IDK about you but I frequently work with frameworks and tools I have not encountered before and I find GPT-4 to do a far better job of giving me quick and straightforward answers about how these tools work than Google. Not just one-off "how do I do X", or "explain this syntax", but also things like "how does framework X handle Y", etc.

I even use it to figure out how to find settings that are buried in cluttered UIs.

Even nicer

Nice

Would be curious how “software spend” is defined here. When people subscribe to Netflix does that go under “software spend” or “entertainment spend”?

Markets are just so tricky to define. The labor pool includes Netflix, even if they’re not competing for B2B SaaS spending.

To be fair, a lot of software spending in Enterprise is what I would like to call Bullshit SaaS, a la David Graeber's analogy. Stuff that was subscribed to by some PM once upon a time, but then later sat unused, even as the company gets billed on it annually or monthly. Not to mention a lot of SaaS simply buying and using each other's products, fully funded by VC capital (which is the YC model).

Take for instance any of the Talk to your PDF products out there. They were popular when ChatGPT came out, but nothing close to the billions of dollars claimed to be in that market. Chatbase made a few millions in revenue, before selling, but now that OpenAI has native functionality, I'm certain that product has been zombified.

Of course, this is different from the large enterprise-level products being built by, say, companies like Harvey AI, but the utility of those products within an org over steady state (after all the hype has died down) remains to be seen.

I think people will lose their jobs with all types of cost-cutting but the people left will be left doing more of the work. I think like Web 2.0 a lot of AI software will be created to manage other AI software.

Buster - Software that links databases and large language models

Vectorview - Custom LLM evaluation

Nuanced - Helps detect deep fakes and misinformation

I'd love to see the studies mentioning 10% efficiency gains if you have a chance. also, do these studies balance against cost of running the models? remember, that's why stability.ai has collapsed.

I am aware of this paper from National Bureau of Economic Research.

https://www.nber.org/papers/w31161

> There’s a ton of terrible startups right now, but some of them will become whales. You can’t predict which ones ahead of time, so you invest in everything.

For what value of “you”?

So is everything being called "AI" these days just LLMs now or is it more?

I know a company that claims to do AI. Their models didn't work so they ended hiring humans to manually do the job AI was supposed to do. Obviously that won't scale, but they still call themselves an AI company.

I've read somewhere that AGI = A Guy in India.

Its just 'human in the loop' AI as long as you don't reveal the humans are the whole loop.

That can still be a legit description as long as there's some learning capacity in the loop, and that can slowly be automated.

Legal informatics is one such domain, there can be use-cases that are very domain-specific, and some high-value events occur rarely, hence can be a small-data problem, with big errorbars (think Apple-Samsung litigation).

Human is the loop.

The official term is "Artificial Artificial Intelligence"

Official term should be AI 2.0 double the artificial, zero intelligence

A2I

  Artificial to Intelligence

Is Monte Carlo AI?

No because AI is just a marketing term for unproven technology to generate hype. Once something is proven and use cases/limitations understood its no longer AI. Spellcheck was once considered AI

> Spellcheck was once considered AI

No it wasn't, let's not start spreading misinformation.

Spellcheck was around since the 1990s. Originally it was simple heuristic-based against a dictionary (localized, maybe also allowing some personalization). Noone called it AI, pre-2010 AFAIK. Can you show any pre-2010 citations for any actual mainstream consumer-available spellchecking using AI?)

I only see a small number of references to the use of AI in spellchecking, mostly post-2018, and they're still the minority of mentions of spellchecking, dictionary heuristics is still a simpler much more basic technique, even if ads don't say that. (There are now (post-2018, GPT-era) AI-based spellchecking/writing-coach/style-type startups, and obviously there are also many companies trying to infuse their collateral with claims about AI, or pivot/bolt on AI to a non-AI concept. Let's not muddle all that together.)

> let's not start spreading misinformation.

The first non-research-project spell-checker was in 1971 and came from the Stanford Artificial Intelligence Laboratory (SAIL), where earlier research on spell checking had been done.

> Spellcheck was around since the 1990s.

And the moon was settled by whalers sometime in the 1750's! Seriously, you're approaching that level of wrong.

> No it wasn't, let's not start spreading misinformation.

This website ("hacker" news) should really be renamed "confidently wrong"; just like smcin's laughable assertion. HN is a propaganda site run by billionaires who editorialize and moderate it to promote their own interests. It's infested with this type of confidently wrong know-it-all. You know what's worse than a pedant? An incorrect pedant. And this site is stuffed with them. It's a Dunning-Kruger hall of mirrors.

Mostly concur, but I'd downplay the D-K.

https://www.psychologytoday.com/us/blog/how-do-you-know/2020...

So is linear regression

What about slapping a wrapper around OpenAI's APIs and calling it a AI company?

..bUt ThEy fInE TuNEd iT...

What a weird article. Title mentions bubble, body only talks about favorite AI companies.

Just a clickbait, like most news articles online.

HN frontpage seems to have much less AI than only a few months ago. Maybe things are slowing down ?

The amount of AI news/HN discussion is mostly a decaying function of time since last major GPT release. Right now people have gotten accustomed to GPT 4, so people quietly muse that AI has plateaued and will never match human ingenuity.

When GPT 5 comes out we’ll see another burst of people saying that strong AI is now just around the corner (and should either be stopped — or businesses should be started up to capitalize on it).

Progress is continuous but hype is cyclical.

> Right now people have gotten accustomed to GPT 4

Anecdotally, Lots of my friends and family was using it heavily when it came out but they seem to rarely use it anymore.

I use it all the time while coding and cooking but it is not something I couldn't live without. I can easily do the same thing even without chatgpt .

The HN frontpage is pretty heavily moderated. You may have noticed no news related to Elon Musk/Twitter, for example, because that is usually instantly removed or downranked. The article about Israel's bombing in Gaza on the front page right now was initially removed but came back after too many people complained, and now has 800+ comments and upvotes. So what you see here isn't necessarily indicative of general public interest or even interest of the community.

Can you give examples of AI or Musk/Twitter news that would be of interest to the community and should have stayed on the frontpage?

What's the objective measure of "interest to the community"? The voting score is obviously the closest one, and links I am describing constantly get upvoted to the top and are then removed.

That doesn't necessarily mean there was any manual moderation. HN isn't just based on upvotes; it also has a flagging mechanism, as well as anti-flamewar mechanisms. They help tilt the scales in favor of technical topics and against lowest-common-denominator content that attracts very poor quality commenters, such as Musk/Twitter drama and most other politics.

Example?

If anything we are in the earliest stages of the AI bubble. There's a long way to go..

Not too late to buy NVIDIA stock

I wonder what happened to all the YC crypto companies in the last 'bull run', it seems they have fell off.

In the same vein as crypto, the grift continues in AI.

Most of web3 companies that I spoke to / know of pivoted to AI circa mid 2023

I've read some CEOs leaving AI to build decentralized AI. A crossover between web3 and AI.

What on earth does that even mean?

(Besides "a foolish VC and his money are soon parted".)

Well they claim that currently the data is centralized and owned by a small group of companies, and they want to decentralize the data so that "users own their data". At least that's what they claim

Decentralization is a myth.

Nah, it’s just that.

As pg says, it’s the people not the product that matters.

This is getting down voted, but I hope people realize that what he's saying is that the Venn diagram of crypto grifters and AI grifters is, in fact, a circle

A lot of them "pivoted" to AI.

Reply on news.ycombinator.com