Hacker News Reader

Comment by mrkramer | Hacker News Reader

This is like dot-com bubble all over again...."we have this fancy and powerful new technology, you must invest in us." Good luck competing with Microsoft, Facebook and Google, because they are super-focused now on LLMs and generative AI and they are not going to stop.

View on HN

> Good luck competing with Microsoft, Facebook and Google

Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

Microsoft invested more than $10bn in OpenAI. Without Microsoft they wouldn't be able to thrive. Unless Elon and other investors were willing to invest billions.

So? People are investing in Mistral too (which this submission is about), albeit smaller sums, so far.

You said OpenAI is eating FAANG's lunch, which is only technically true because Microsoft isn't in that acronym.

OpenAI is part of the battle between big tech companies.

I strongly suspect that people are figuratively standing in line outside OpenAI headquarters with giant bags of money.

OpenAI is probably turning down multi-billion dollar investment offers on a weekly basis.

> Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

Precisely how is OpenAI eating anyone's lunch? This is a serious question. Is OpenAI fantastically profitable? Does it have a proprietary moat that changes its margins in a way that no other competitor can overcome? Does it benefit from network effects that lead to a natural monopoly?

From my mole's eye view I see a raft of "use AI on task X" applications that are incremental improvements on the underlying products, not game-changing products themselves. I also see insane equity valuations, which look more like Gilded Age railroad speculation than actual proof it's different this time. [0]

Don't get me wrong. LLMs have some pretty amazing capabilities. The fact that they can perform what appears to be reasoning on huge pools of data is fascinating. But it seems premature to say that anyone has won here.

[0] https://www.smithsonianmag.com/history/robber-baron-gamble-r...

> Precisely how is OpenAI eating anyone's lunch? This is a serious question.

Because they made, maintain and offer GPT-4 which is currently the top-of-the-line LLM available. GPT-3 was also top-of-the-line when it launched, and they've proven they can iterate on the model and launch better ones.

Currently, both open and non-models are just about barely competitive with GPT-3.5, but none is close to GPT-4.

Even if they aren't profitable today, it's hard to imagine they won't be able to monetize things better, since their model is way ahead of any other model.

That is, they manage to get that far before the company collapses from the inside, which seems more and more likely everyday.

Isn't that just a win-on-execution model? Given that OpenAI does not exactly have stellar management and the fact we're not sure where the big economic wins for LLMs will occur it seems premature to declare them the winner.

Funny, considering people said the same thing about OpenAI and look who is eating FAANGs lunch now when it comes to AI?

I tried Bard yesterday (Im late like that) and I'm not switching back to a competitor anytime soon. The UX was smoother I found and i was blown away by the multimodal thing. It generated some python and the picture of a graph the python is supposed to draw, it was awesome experience. I told myself I trusted Google more and it integrates with their other apps so I could search my emails.

I mean, they are clearly not f-ing around. They're going Iron-Mike on it rather. Which is the point of parent comment.

The latest Bard is good, but at this time (Dec 2023), Bard is still not better than paid ChatGPT 4 in my experience.

I use both daily (I use ChatGPT on my phone and at home, Bard at work because work doesn't allow ChatGPT). ChatGPT 4 still provides the highest quality answers to the range of questions I'm interested in. Bard's getting better with Gemini but it's still not good.

Claude is great at summarizing PDFs and has one of the longest context windows, but the quality of answers it has to general questions is still lacking. Outside of summarization, I have little confidence in Claude's Q&A abilities.

I've also been using Perplexity.ai and I would say it hits a sweet spot and is impressive. It's essentially a RAG-based search engine that's fast and snappy and it works much better than Bing or ChatGPT Bing. Because it's primarily RAG driven, the content is primarily external and you get links to original sources so you can verify the answer. This means the level of hallucination is lower than foundational models. It's become a practical tool for me. It's CoPilot feature, which asks clarifying questions about your query, has been helpful for finding new products. For instance, I was looking for luggage packing cubes that convert to hanging cubes. It's a known product class but I didn't know the lingo, but over a few clarifying iterations, it was able to help me find the right product.

>I've also been using Perplexity.ai and I would say it hits a sweet spot and is impressive. It's essentially a RAG-based search engine that's fast and snappy and it works much better than Bing or ChatGPT Bing. Because it's primarily RAG driven, the content is primarily external and you get links to original sources so you can verify the answer. This means the level of hallucination is lower than foundational models.

That is what worries me about the Google Bard; it rarely gives references to its answers so I'm not sure if I can trust it or not but you can double check the answer by Google Bard searching it on Google Search and then giving you the source and its textual reference.

Where do you find the references from Bard?

There is the G button that you click after the answer appears which double-checks the result, but it only underlines parts of the answer that show it was or wasn't able to corroborate with search results. It doesn't seem to provide links to the relevant results themselves.

Whereas wiht Perplexity, the links are part of the answer.

G Double check button only gives one reference(at least in my examples) but it seems like Bard does not give references by design[1][2]. Websites are kinda getting ripped off, because Google is not referencing them and giving them exposure and traffic. But Bard is still in the experimental phase so I hope that will change.

[1] https://searchengineland.com/google-explains-why-bard-rarely...

[2] https://www.seroundtable.com/google-bard-wont-link-to-source...

We were the first investors in for-profit OpenAI.

OpenAI is a completely different story and setup than Mistral.

> Good luck competing with Microsoft, Facebook and Google

Well, that's what the funding is for. These guys already proved they have the necessary expertise - Mistral models are top notch. Now they need the money (and time) required to train on large datasets.

Mistral's models outperform all of theirs at much smaller sizes and can be self-hosted.

Right now, but how long does it take for OpenAI, Facebook and Google's Bard/Gemini to catch up?

Considering Google been lying in their marketing videos about their AI capabilities, OpenAI seems to be falling apart from the inside and Facebook refuses to release open (not commercially-locked "open") models, there seems to be space for at least one honest and open AI company.

Google is trying and just can't? They are pretending they aren't behind despite setting a pile of cash on fire by spending exorbitant amounts on exaggerating their capabilities in basically fake videos.

Meanwhile you see Mistral casually dropping magnet links to weights with barely any instructions on how to use them on a Friday afternoon and you see reports of some very happy people finetuning on a civilian 4090 and getting GPT-4-quality performance in blind tests on Saturday morning.

Google is trying to catch up with OpenAI, not with open source 7B models.

Open Source 7B models have already caught up with OpenAI.

The current top of the open model 7B leaderboard beats GPT-3.5 and Bard while running on a laptop, smartphone, raspberry pi, or 10 year old graphics card.

Tim Dettmers just released code to get Mixtral 8x7b running in 4GB of RAM, the same amount required by Mistral 7b.

We are quite clearly a matter of weeks from Open Source being on par with GPT-4, likely before Google's Gemini Ultra is even released.

This is all to say nothing of the multi-model LLama-3 120B coming in 2 months according to insider leaks.

I’m very skeptical of LLM benchmarks because I’ve been testing models through Ollama and none of the open source 7b models are at even close to GPT4 (or Open AI’s GPT 3.5 implementation) in daily usage, despite what benchmarks say. LLM results are actually really hard to measure objectively because it depends on what kinds of questions you ask and what answers you expect.

I’ve just been finding lots of little errors and reasoning mistakes, and even in terms of producing useful results, they fall short of GPT4.

> Mistral's models outperform all of theirs at much smaller sizes and can be self-hosted.

This does not track with my experience. Mistral's models have bang for buck in that they are impressive for being so small, but they do not beat GPT4 at all.

Sure, but you can't host GPT-4.

No, but I look forward to the day I can trust Mistral results. Right now I have low confidence in them for any use that require at least a reasonable level of correctness. GPT4 on the other hand is correct enough for many use cases and much faster.

I’m also excited to see how new non transformer architectures like Mamba and state space models turn out.

> Good luck competing with Microsoft, Facebook and Google, because they are super-focused now on LLMs and generative AI and they are not going to stop.

Remember when Google was gonna kick Facebook's ass in social networking? All four times?

The localization of these broader services to trade and market demands is how they scale faster and deeper. Unless you think MS and Google and Amazon have Europe locked down now?

Reply on news.ycombinator.com