Hacker News Reader

Comment by rsp1984 | Hacker News Reader

rsp1984

AI models are in a race to the bottom and everybody inside Anthropic knows it. Besides OpenAI, with billions to spend plus a partnership with MS, there's also Google, Apple, Meta and Amazon who can afford to run losses on AI for years without blinking an eye.

And if that wasn't enough the Open Source world is releasing new models almost weekly now, for free.

Anthropic is putting on a big show to convince gullible investors that there's money to be made with foundational models. There's not. I expect a big chunk of the raised money to go out the door in secondary sales and inflated compensations. Great if you're working at Anthropic. Not great for investors.

View on HN

Truth is, and this applies to all companies regardless of size, is that you don't have to be first, best, biggest, fastest, or most well-known in order to win market share that out-paces your investment. The AI pie is going to be very, very big. To estimate this size, let's take McKinseys rough estimates of job displacement (~30% of ~60% of jobs, ~20% of work) and use that to estimate the actualized [US, apologies] GDP that can at some point be attributed to AI: it is in the 4-5 trillion range using today's figures.

To say a market that large will be owned by only 4-5 companies doesn't make sense. Let's take the PC market for example: there are roughly 6 companies that make up ~80% of the market, sure. However, let's look at a tiny participant compared to the total market (~65B): iBuyPower at rank #77 had sales of 40MM or 0.06% (small, expected) of the market with a much smaller capital investment. If look at this percent compared to 5T, we would be at 3B. While the 5B investment stated in the headline could result in a lower ranking and smaller share, the point stands that there is still a lot of money to be made on the long tail. Even if Anthropic fails, there will be other companies with similar infusions that succeed.

It is not clear if (i) a lot of the surplus will be captured by the AI providers and (ii) that the impact will be anywhere as big as people now guess/want it to be. Making a bet on the future is fine, of course.

My question would also be what kind of insight McKinsey can provide here. What, if anything, do they know about AI that we don't know?

If anything McKinsey has a lot to gain from exaggerating the numbers so more companies come to them for AI solutions or whatever their next consulting product is.

Likely not much and assuredly wrong, I just wanted to ground my argument with numbers that came from people who presumably did more research than I was willing to do for an HN post.

We’ve got a lot of data scientist talent but I wouldn’t put a lot of stock in this particular estimate. If McK is gonna produce a novel insight it’s usually derived from having the input of many businesses across an industry and experience looking at their problems. It’s hard to imagine this one isn’t more or less made up due to the number of assumptions required.

You don't need to just take one source. OpenAI authored their own paper [1] on the economic impacts of just LLMs: "Our findings reveal that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted."

Goldman Sachs Research just pushlished their own analysis as well. [2] Their conclusions are "As tools using advances in natural language processing work their way into businesses and society, they could drive a 7% (or almost $7 trillion) increase in global GDP and lift productivity growth by 1.5 percentage points over a 10-year period." and "Analyzing databases detailing the task content of over 900 occupations, our economists estimate that roughly two-thirds of U.S. occupations are exposed to some degree of automation by AI. They further estimate that, of those occupations that are exposed, roughly a quarter to as much as half of their workload could be replaced."

[1] https://arxiv.org/pdf/2303.10130.pdf

[2] https://www.goldmansachs.com/insights/pages/generative-ai-co...

From [1]: "In our study, we employ annotators who are familiar with LLM capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding LLMs’ reliability and effectiveness in performing tasks within unfamiliar occupations."

From [2]: "Analyzing databases detailing the task content of over 900 occupations, our economists estimate that roughly two-thirds of U.S. occupations are exposed to some degree of automation by AI."

These are people who do not understand the jobs they are claiming AI will do. Ultimately, I think they are not doing much better than guessing.

The AI (LLM) market as a whole is very immature, trying to guess today what it will look like in a decade based on the investments/behaviour of the first couple movers is pretty foolish. Even predicting for a specific submarket (ie, consumer LLM products like ChatGPT) is hard enough. Who knows what other categories could develop and be dominated by companies who narrow in on them and once the R&D progress starts flatlining like it always does.

The best way to predict the future is to invent it.

- Alan Kay

Although a large total addressable market (TAM) is very alluring, know that most markets are dominated by a few players. For example, sugary beverages (Coca Cola), office software (Microsoft), or luxury sports cars (Ferrari). Exceptions are markets where companies cannot find a moat such as air travel or farming. In those markets, profit margins are tin.

At this point in time, it’s hard to tell whether moats will arise around large language models. Peter Tiels thinks so or he wouldn’t have invested (see his Competition is For Losers presentation).

What is unlikely is that semi-good companies will thrive. Maybe for a few years but at some point the smaller players will be pushed out of the market or need to find a specific niche. Just look at cars to see this. Around 1900 there were hundreds of car brands.

PCs are hardware which have a minimum cost to be produced. Now do the same calculation for search engine or computing clouds.

Can’t find this study, have a link?

> let's take McKinseys rough estimates of job displacement (~30% of ~60% of jobs, ~20% of work)

These studies seem to be largely focused on job displacement. There is is a reasonable likelihood that AI grows the overall economy.

I think we forget that our perspective of AI now is comparative, probably to that of a preindustrial worker worried about machines. Displacement, sure but complete replacement seems a non nuanced view of how it may all turn out.

The counter argument is that it's a growing market where any early entrants will be lifted with the tide and can probably yield enough profit from spillover hype for investors to make their investments back.

> can probably yield enough profit from spillover hype for investors to make their investments back.

The correct term for this is “pyramid scheme”.

Nope

No, this is more "everyone is selling X, let's get in the business of X". On the other hand, yes, some will miss the boat and lose money.

I interpreted “spillover hype” as meaning “more investors coming in in future rounds” (ie pyramid scheme), but it’s possible that’s not what the commenter intended.

But if early investors only profit due to late investors pouring money in, that’s by definition a pyramid scheme.

I can already run gpt4xalpaca on my PC, a model that is not-bad-at-all and is completely uncensored (i.e. does things that chatGPT can't do). I think it's true that LLMs are racing to the bottom and will be even more once they can fit as a peripheral to every computer. whoever is investing in this to monopolize has not thought it through

This is a repeat of the early GPU era.

It's not the software or hardware that will "win" the race, it's who delivers the packaged end user capability (or centralizes and grabs most of the value along the chain).

And end user capability is comprised of hardware + software + connectivity + standardized APIs for building software on top + integration into existing systems.

If I were Nvidia, I'd be smiling. They've been here before.

Nvidia: just as the sun starts setting on crypto mining, the foundation model boom begins. And in the background of it all, gaming grows without end.

If you've got a choice, sail your ship on a rising tide! And if you can spread the risk over multiple rising tides, so much the better!

My dad told me a quip once: "It's amazing how much luckier well prepared people are."

Imagine you're a tech company that pays software engineers $200K/year. There is a free open-source coding model that can double their productivity, but a commercial solution yields a 2.1x productivity improvement for $5000 annually per developer. Which do you pick?

Not sure if parent had a certain answer in mind, but my answer is OSS because (1) I can try it out whenever I want, and (2) I don't have the vexing experience of convincing the employer to purchase it.

That’s the endless «build vs buy” argument. And countless businesses are buying.

I don't this it's the same thing, at least for me.

In the GP's scenario, I wouldn't be building either piece of software.

I find your argument persuasive, companies should spend extra for the significant productivity gain. But then again from experience most companies don’t give you the best tools the market hast to offer..

The existence of the models is making programmers cheaper rather than the reverse.

But i think it is underestimated how important it is for the model to be uncensored. ChatGPT is currently not very useful beyond making fluffy posts. As a public model, they won't be able to sell it for e.g. medical applications because it will have to be perfect to pass regulators. It cannot give finance advice. Censorship for once is proving to be a liability for a tech company.

In-house models OTOH can already do that, and they can be retrained with additional corpus or whatever. And it's not even like they require very expensive hardware.

It’s astonishing to me that people seem to believe the llama models are “just as good” as the large models these companies are building, and most people are only using the 7B model, because that’s all their hardware can support.

…I mean, “not-bad-at-all” depends on your context. For doing mean real work (ie. not porn or spam) these tiny models suck.

Yup, even the refined ones with the “good training data”. They’re toys. Llama is a toy. The 7B model, specifically.

…and even if it weren’t, these companies can just take any open source model and host it on their APIs. You’ll notice that isn’t happening. That’s because most of the open models are orders of magnitude less useful than the closed source ones.

So, what do want, as an investor?

To be part of some gimp-like open source AI? Or spend millions and bet you can sell it B2B for crazy license fees?

…because, I’m telling you right now; these open source models, do not cut it for B2B use cases, even if you ignore the license issues.

gpt4xalpaca is 13B

7? 13? Who cares? It’s an order of magnitude smaller than the GPT models. It’s a toy.

You know what I believe is also a toy model? chatGPT Turbo, you can tell by the speed of generation. And it works quite well, so small size is not an impediment. I expect there will be an open model on the level of chatGPT by the end of the year because suddenly there are lots of interested parties and investors.

Eventually there will be a good enough model for most personal uses, our personal AI OS. When that happens there is a big chance advertising is going to be in a rough spot - personal agents can filter out anything from ads to spam and malware. Google better find another revenue source soon.

But OpenAI and other high-end LLM providers have a problem - the better these open source models become, the more market they cut underneath them. Everything open source models can do becomes "free". The best example is Dall-E vs Stable Diffusion. By the next year they will only be able to sell GPT4 and 5. AI will become a commodity soon, OpenAI won't be able to gate-keep for too long. Prices will hit rock bottom.

> I expect there will be an open model on the level of chatGPT by the end of the year because suddenly there are lots of interested parties and investors.

I really don't think you understand just how absurdly high the cost is to train models of this size (which we still don't know for sure anyways). I struggle to see what entity could afford to do this and release it as no cost. That doesn't even touch on the fact that even with unlimited money, OpenAI is still quite far ahead.

Still cheaper than a plane, a ship or a power plant, and there are thousands of those.

And how many are given away for free?

I think you're conflating speed of inference/generation with optimization. gpt-3.5-turbo does not fit on a single GPU unlike the "toy" models.

I think that Alpaca 30 billion is pretty competitive with ChatGPT except on coding tasks. What benchmarks are you using to make your determination about suitability for B2B?

> I can already run gpt4xalpaca on my PC

You can also run your stack on a single VPS instead of cloud, gimp instead of photoshop, open street maps instead of Google maps, etc.

There will always be companies who can benefit from a technology, but want it as a service. In addition, there will be a lot fine-tuning of LLMs for the the specific use case. It looks like OpenAI is focusing a lot on incorporating feedback into their product. That’s something you won’t get with open-source models.

Yeah but with very simple tasks with the 2k tokens limit. Let alone the fact that it can't access the internet, or have more powerful extensions (say Wolfram).

Alpaca is the Napster or LLMs

That’s an argument, but I don’t buy it. Models are a commodity. You don’t get VC valuations and returns from raising $5B for a grain startup.

The application of AI to business problems will be lucrative, but the models are just a tool and the money will come from the domain-specific data (i.e. user and business data), which Microsoft, Google, and even Meta are positioned for. Having a slightly better model but no customer data or domain expertise doesn’t seem like a great recipe.

Then again it’s AI, so there’s more uncertainty than the commodity market. Maybe Anthropic will surprise and I’ll be as wrong about this as I was about OS/2 being the future. But I’m very skeptical.

I don't think the grain market is growing as fast as the AI market

Don’t confuse the ai market with the foundational llm model market.

Think of LLMs as the understanding component in the brain, once you can understand instructions and what actions need to happen from those instruction you’re done.

The rest is integrations, the arms legs and eyes of langchain. Then memory and knowledge from semantic search, vector databases and input token limits.

This is the real answer.

The LLM is but the core of the entire ecosystem. Just like how MLOps is 99% of the work, choosing an LLM is 1% of the effort in the final product.

What do they have besides an admittedly very cool name?

Anthropic is currently the only company that can compete with OpenAI (because they have comparable expertise). The rest (Google, Meta, Microsoft, etc) are still pretty far behind.

This approach didn’t work for Docker.

Plus first-mover advantage has consistently shown to not be a legitimate strategy as there are a ton of cases where the first winner gets taken over by a new entrant once the market matures (Friendster being the classic example). Often the later companies learn from the mistakes of the first play.

R&D heavy markets might have some different characteristics but it's still way too early to say with AI.

> convince gullible investors that there's money to be made with foundational models. There's not.

This is a ridiculously myopic statement. Foundation models are an extremely powerful technological advancement and they will shake the global economy as very few things did in human history. It's hard to imagine how this is not obvious to everyone right now, specially here in this forum.

Plus you're also investing in getting the talent together in the same building. Even if the foundational models aren't the money maker there's still a ton of opportunity having the best experts at building those models working together and figuring out which branches that LLMs spawns can turn into real markets.

It's a high risk investment at this stage but the money is being thrown at the people as much as the current business plan.

No one here is disputing that.

The question is whether whomever builds them can make a profit doing so, or will they just end up being the suckers that everyone who actually makes money piggybacks off. It's really not clear at the moment.

None of which means the money will go to the people making the models.

The game theory logic doesn't care about the labels "OpenAI" or "Anthropic" or any of the others, it's the same if you switch it around arbitrarily, but this is easier to write about if I focus on one of them:

At some point, someone will reproduce GPT-3.5 and ChatGPT, given how much is known about them. When that happens, OpenAI can't make any significant profit from it. GPT-4 might remain sufficiently secret to avoid that, but the history of tech leaks and hacks suggests it too will become public, but even if it does itself remain behind closed doors, there is a further example in that DALL•E 2 is now the boring 3rd horse in the race between Stable Diffusion and Midjourney, and the same may happen with the GPT-series of LLMs.

The models leaking or being superseded by others implies profit going to increased productivity in the general economy without investors getting a share.

DALLE2 is boring because pretty much everyone at OpenAI has been busy developing the next GPT model. It was simply not a priority for them. And when GPT4 leaks (or is reproduced) they will most likely have GPT5. In this race it’s far more important to be the closest to AGI than to make money now.

It's a bit early to tell, isn't it?

If we get more unexpected emergent abilities by scaling the model further, things could get very interesting indeed.

There’s needs to be some difficult barriers to entry beyond having the money to spend on training FLOPS in order for a startup to compete.

I have no idea if there are or there aren’t, but that’s the big question.

I mean, this is just the beginning. Just wait till we get actual scifi robots in the next year or so.

FWIW, I do find that Claude (Anthropic's GPT) is often better than GPT4 -- and very fast. Entrants can compete on price, safety, quality, etc.

And a big moat is going to be safety... and specifically configuration of safety.

Wouldn't be surprised at all if the major API-based vendors start leaning in on making their safety config proprietary.

If a business has already sunk XXXX hours into ensuring a model meets their safety criteria for public-facing use, they'd rather upgrade to a newer model from the same vendor that guarantees portability of that, versus having to reinvest and recertify.

Ergo, the AI PaaS that dominate at the beginning will likely continue to dominate.

Excellent point.

Fine tuning is at a low point now, but i expect this to create a moat for the same reasons.

Is it possible to test it somewhere?

https://poe.com/Claude-instant

It also provides a ChatGPT interface, and a number of other models.

I find that Claude is more conversational (better fine tuning), but not as smart as even ChatGPT.

Prompt:

  The original titles and release years in the Harry Potter series are:

  Philosopher's Stone (1997)
  Chamber of Secrets (1998)
  Prisoner of Azkaban (1999)
  Goblet of Fire (2000)
  Order of the Phoenix (2003)
  Half-Blood Prince (2005)
  Deathly Hallows (2007)
  
  Given this, generate a new Harry Potter title, using only the words found in the existing titles. Avoid orderings in the original titles. You may add or remove plurals and possessives.

Results:

ChatGPT: Blood Chamber of the Phoenix's Prisoner

Claude-instant: Chamber Prince Half-Blood Phoenix

The question for me is whether they understand complex concepts and can apply them in new areas.

So when I’m doing quantum computing work, I go back and forth between Claude and GPT4 and both complement the other very well.

I find the opposite, claude-instant seems to generally give me better results for my use case. FWIW gpt-3.5-turbo is good too, just not quite as good.

ChatGPT is more comparable to what Quora/Poe calls Claude+ - slower/more expensive/smarter. Claude-instant is closer to GPT-turbo in that tradeoff space.

Both bots are free on poe.com, so one is not more expensive than the other.

Would you rather? invest in super intelligent AGI or NOT invest in super intelligent AGI. Especially if one of those emergent abilities is deciding your either with me or against me..lol

That would be the AI version of Pascal's Wager [0]

[0] https://en.wikipedia.org/wiki/Pascal%27s_wager

It still remains to be seen if LLMs will lead to AGI.

The fact is nobody can risk not owning a piece of the foundational models. There is waaaay too much upside risk that they will tots dominate the market.

I mean, maybe they won't like you say, but what if they do? Then you're probably screwed. Better to gamble a few billion, imho.

This sounds like saying "internet search engines are a race to the bottom" 20 years ago without realizing that someone may end up as Google and obtain market dominance for a decade or so.

It also sounds like you believe you have defined the bounds for what AI will be, and figure we'll just iterate on that until it's a commodity. I don't think AI will be that static. We're all focused on stable diffusion and LLMs right now but the next thing will be something else, and something else after that. As each new technique comes out(assuming they are all published), we'll see quick progress to incorporate the new ideas into various implementations, but then we'll hit another wall, and suddenly big budgets and research teams may matter again.

tldr is that it is way too early to make the cynical claim you are making.

It all depends on if they're in the business of producing the whitepapers that drive ML advancements in the first place. AI is far from a solved problem and whoever gets to it first wins. We have GPT because of a billion dollars worth of data, not algorithms.

The smart move could be an open-core approach. Release the models, but have the best engineering stack to run the APIs as a service.

But the models are the expensive part to train. Running the models is relatively easy.

The best models will always be closely guarded and have the best outputs, it’s the watered down models that are fighting for scraps.

Who says that the AI model is the business?

Will fine tuned models be lucrative then?

Ahh the old Lyft / Avis strat.

Correct. Stability.ai (Stable Diffusion), Apple (Won't be surprising to see them announce on-device LLMs with Apple Silicon support), Meta (LLaMa), etc are already at the bottom and at the finish line with their AI models given for free.

O̶p̶e̶n̶AI.com will eventually have to raise their prices which is bad news for businesses not making enough money and still are sitting on their APIs as O̶p̶e̶n̶AI.com themselves are running up huge costs for their AI models in the cloud for inferencing.

Anthropic is just waiting to be acquired by big tech and the consolidation games will start again.

Aside: I love the O̶p̶e̶n̶AI.com thing. I got caught off guard at least twice!

Wow I didn't know ai.com redirects to chat.openai.com. How long has it been doing that?

It's fairly recent, I think I saw an article here on their purchase of the domain for a few million

This sort of hand wavy generalizations about such a broad and ill defined market seems very naive/closed minded.

If you're quabbling over how much OpenAI charges for an API today that barely just launched and from which we have barely scraped the surface for applications... I don't know that seems like a failure to think broadly and assumes the market today is what it will look like in 5yrs.

There could be a ton of lucrative businesses which subsidize those operating costs. It doesn't have to be a mega-company like Google that floats it indefinitely off their ad business, or whatever other scheme. We have no idea what the value of those APIs are or if the API is the real business they (and others) are going to be relying on in the long term.

>O̶p̶e̶n̶AI.com will eventually have to raise their prices which is bad news for businesses not making enough money and still are sitting on their APIs as O̶p̶e̶n̶AI.com themselves are running up huge costs for their AI models in the cloud for inferencing.

Do you have data supporting this or is it just speculation? Given we don't even know how many parameters GPT-3.5 and GPT-4 have, yet alone how efficiently they are implemented, I don't see how we can go about coming up with an accurate estimate for the cost per token.

Reply on news.ycombinator.com