Hacker News Reader

Comment by ramesh31 | Hacker News Reader

Where's the moat for OpenAI? How does the company not end up as just a feature stitched into Azure somewhere?

I'm surprised to see this comment. I think OpenAI has a huge moat. Companies like Google who are pouring insane resources into beating OpenAI are struggling to. I think they have the potential to be worth $1T. I think the moat will continue to grow.

I haven't tried Gemini 1.5 yet, but every single model I have tried haven't come close to GPT-4. I don't care what benchmarks say - they really don't come close.

If Gemini 1.5 is competitive, that was a full year to catch up. I'm sure OpenAI has other things cooking.

I've read several reports Gemini 1.5 being excellent, I have on reason to doubt those reports because we can slag Google off all we like but there is nothing magic about what OpenAI is doing and it's only a short matter of time till others can do it too.

Again, you might not like Google but I mean, they have a pretty kick as AI team and all the money and hardware in the world to catchup for lord's sake.

People talk and there is zero chance anything OpenAI does will be kept secret forever.

I hope the tone of my comment did not convey I thought poorly of anyone here. Google has done incredible work in AI.

Which I think makes my point that it took a year to catch up a damn convincing argument that OpenAI has something special going on.

And what are you talking about nothing magic about what OpenAI is doing? LLMs are about as damn near close to magic as we have. It's incredible it works at all. They are so damn cool.

And Sora is crazy impressive too.

3 years ago, I wouldn't have guessed we would be anywhere near where we already are, and I was in the thick of it, doing ML research and engineering at that time.

Eventually sure, but things are moving so fast!

It's all impressive, in 2 years you'll be bored with it and new things will be happening, new companies will exists, new ideas will emerge, and on it goes.

Once you've generated your 50th short movie with Sora 5 you'll be sick of that too.

I'm not so sure. I'm not bored with my OS, or IDE, core parts of my day to day life. I'm not bored with YouTube.

If they are tools that provide value, it's not about entertainment. If I can learn better or do things better with their aid, I'll keep using them.

I'm much more interested in generating a 3blue1brown quality educational video than season two of firefly.

I mean, you don't use those things and sit there in awe of them everyday.

I mean.. is there anything magic about what Nvidia's doing?

No, and what's your point?

Training & individual user preference data - specifically the data being generated by people interacting with ChatGPT and DALLE. Maybe Google can out data them with all the data they currently hold, but they'd better hurry before the network effects get too strong in favour of OpenAI.

In an exponential game where the prize is ASI, being just a couple of steps ahead may turn out to be enough.

Try training and running really really big models.

The old industry question is whether it's complex and protected enough (or the separation wanted by the consumer, but that's an old guy's dream) that the owners of the production equipment can't do it on their own.

Azure, AWS, Google are easy candidates, Nvidia might even be proof.

You mean now. Commodity soon.

It's going the other way. The number of levers to pull around architecture, data amount/type/quality, hardware optimizations, decoding optimizations, evals, system architecture (distinct from model architecture), alignment tricks and various other things means the complexity is just getting started.

Because of scaling advantages, it likely never will be. Even if you could run a GPT4 quality model on your laptop (you can't), by the time you can, Open Ai will be running a model that uses orders of magnitude more compute.

This could easily end up like the PC revolution though. ChatGPT being the mainframe of AI, and when the hardware catches up and performance is “good enough” we’re all running our own models all over the place.

I dont think compute capacity will be the determining factor for moat. It will most likely be access to various types of data and I believe on that front Meta and google are way ahwad

Stupid question, but why aren't they doing it already? What's their bottleneck right now?

Try shoving a couple trill (estimate) parameters in memory. Or, shuttling it back and forth from disk as required to generate a single token. And then shuttling that into the chip. Assuming they are using FP16 it's 4 TB give or take. They aren't kidding when they call them "Large".

Reply on news.ycombinator.com