Nvidia releases its own brand of world models | Hacker News Reader

Nvidia releases its own brand of world models

techcrunch.com

70 pointsbariscan70 comments

> Nvidia wouldn’t say where this training data came from, but at least one report — and lawsuit — alleges that the company trained on copyrighted YouTube videos without permission.

Under the EU's AI act[1] there is now a legal obligation to disclose the source of the training data. Is this correct then that either the models cannot be used in the EU, or we'll get to know where the training data came from?

[1]: https://oeil.secure.europarl.europa.eu/oeil/en/procedure-fil... - "General-purpose AI systems, and the GPAI models such as ChatGPT they are based on, must meet certain transparency requirements including compliance with EU copyright law and publishing detailed summaries of the content used for training."

Unfortunately the solution is quite simple - don’t release in EU.

People keep saying this, but I've yet to see a company follow through.

I am from EU and we have been getting a lot of things much later and having to use VPNs to get past this. Sometimes even a VPN is not enough since we need US phone number etc. Couldn't get access to Claude and OpenAI advanced voice mode for a long time. It feels very frustrating since others can already start building with cutting edge things, giving them a huge advantage.

Have yet to see a case where they didn’t also include the UK and other decidedly none EU states, which makes me think this has little to do with regulation and more trying to manage load during the initial release. Regulation is of course an amazing scape goat, especially if one intended to convince people against any legislation not written by and for the current front runners of the industry.

Yes, unfortunately in this world not being "ethically constrained" can give you a big advantage. Still doesn't mean one should not strive to do the right thing.

If your ethics is innately tied to the nuance of fringe intellectual property law in a brand new domain, it may be time to relax some things

That is to say, a lax perspective of intellectual property law can be less or more ethical than one which supports the undeterred corporate collation of powers behind those laws.

Those in control do not have a monopoly on ethical goodness.

In this case the potentially infringed parties would be YouTube creators, who in most cases are not corporations or backed by them.

(I myself personally happen to think it's fine to train a commercial AI on content from the internet like this, but the framing of your argument just feels misleading or even manipulative. "Copyright" -> "IP law" -> "big business" -> "bad vibes", when for cases like this the affected people are almost all small individuals and the responsible entities are almost all big corporations.)

It is subjective.

It looks like opportunity.

Meta and Apple both declined to release AI tools in the EU due to regulations:

https://www.cnet.com/tech/services-and-software/meta-follows...

“Delayed” seems more accurate. At least Apple already released some AI features and plans to release more later this year.

> Mac users in the EU can access Apple Intelligence in U.S. English with macOS Sequoia 15.1. This April, Apple Intelligence features will start to roll out to iPhone and iPad users in the EU.

https://www.apple.com/uk/newsroom/2024/10/apple-intelligence...

I live in the EU and pretty much every major AI tool gets either delayed or not released here, it is terrible.

Another anecdote, I don’t mind. I’m happy to let the rest of the world beta test AI, I’ll wait and enjoy more transparency (and perhaps a more polished experience).

Then again I’m not a pro AI user and I was never in a situation when I wanted to use an AI product and it wasn’t available.

Or just lie

Short term: fine; long term: fines.

No problem. If you are in EU, you upload your input to a server located in US, that way EU law can watch you do that.

That would not work. It's the company who's responsible for compliance with EU laws and regulations in this case, not the user. So, if the company allows EU users they are de facto operating in the EU and thus out of compliance with the training data transparency law cited above.

> This Regulation applies to:

> (a) providers placing on the market or putting into service AI systems or placing on the market general-purpose AI models in the Union, irrespective of whether those providers are established or located within the Union or in a third country;

Why go to techcrunch and not directly to their only source of information on this? There are also some actual technical details there.

https://www.nvidia.com/en-us/ai/cosmos/

Techcrunch didn't omit the lawsuit information which alleges Nvidia farmed YouTube to train this, I found this more useful than pure marketing spin on nvidia.com.

What stops Nvidia from cutting out the middlemen? They have the chips.

I don't think we can know about the long term, but I would assume that in the short-medium term, Nvidia can make a lot more money by continuing to sell its cards, where it's almost a monopoly, rather than competing in the overcrowded product space.

We don't yet know if there's actually any gold in the mine or not this time around, but we know for certain that there is money to be made selling pickaxes to the miners.

There is probably more money to be made selling the tools.

I wonder if any shovel manufacturers invested into mining during the gold rush...

They lose their customers. TSMC succeeded because they didn't compete with their customers. When they started, they focused on making chips for others. Any other company that that had a Fab could offer the same service at the same quality back then. Yet folks choose TSMC because there was no conflict of interest. If Nvidia starts competing with their customers, that conflict of interest will force some of those customers to look else where. I suspect Nvidia is releasing these things to add value to using their GPUs, kinda like offering free tools for your OS, notepad, calculator, browser, etc

Nothing, in the same way that TSMC could cut Nvidia and OpenAI out.

Vertical integration is incredibly powerful, but it requires mastery of the whole stack.

Does Nvidia understand AI consumers the way OpenAI / Anthropocene does? Do they have the distribution channels? Can they operate services at that scale?

If they can truly do it all and the middlemen don’t add any unique value, Nvidia (or TSMC) can and should make an integration play. TBB I’m skeptical though.

> Nothing, in the same way that TSMC could cut Nvidia and OpenAI out.

one thing is not like the other.

They’re exactly alike. The question is “why doesn’t an upstream supplier just use its own products and cut their customers out?”

If you prefer, we can ask why TSMC doesn’t cut Nvidia out and just design AI chips. It gets to the same place: expertise as a supplier in one stage of a value chain is not fungible to all stages.

I disagree. Nvidia designs chips and has demonstrated it can move between manufacturers (TSMC to Samsung). TSMC does not have know-how to design nvidia level of chips or develop supporting infrastructure (software and sales) because a) they have demonstrated they don't have any b) other players which do have capacity like AMD and Intel have also demonstrated they don't have any.

Nvidia can move its vertical, TSMC not really. If you would've mentioned Apple for example, then maybe.. since that know-how is licensable to an extent (Arm).

But it's entirely possible to me that TSMC lacks some of the hardware knowledge that Nvidia has. A fab has a lot more than just TSMC equipment in, hasn't it?

They kinda need to do it to justify their price

https://ngc.nvidia.com/

There’s no need to cut them out, the symbiosis is working perfectly.

Huh? Their position in the market is much better than that of OpenAI and Co. Selling hardware and services is sure business. Training models is by contrast a risky business, and even if you attain SOTA, you cannot relax because your competitors will be on your tail. From a purely pragmatic perspective they also do not have the manpower to compete seriously in that space.

Doing so would mean incentivizing customers to seek alternative routes and migrate to competitor. Thus with the risk of losing the current moat.

Given they are releasing models, whose to say that they don't have teams working -- not secretly -- just not in the open?

They literally have paid platforms to use their models.

Then they would quickly discover that their moat isn't as strong as people think it is.

They would have to pay for them.

middlemen to what?

The AI race is one of the most impressive examples of Capitalism making the market efficient. Or at least I've witnessed in my life.

We went from Google having complete control. To Open AI releasing GPT2 which really inspired a lot of people to try it. Then GPT3+ convinced the world to try it.

After that, Gemini, LLaMa, every type of fine-tune... The noteworthy thing is that LLaMA was good enough that ChatGPT had competition. Then within 1 year of that, we have a dozen companies with models that are good enough.

The competition has been the best type of brutal.

Good enough for what? I've been playing around with local models that often get mentioned here (llama3.3, mistral, etc.) and they routinely provide incorrect code that does not even compile or implements algorithms that have nothing to do with the task at hand, generate invalid JSON like '{ "foo": bar - 42 }', write nonsensical statements like "CR1616 has double the capacity of CR2032", etc. I'm yet to find a useful application for them that they can actually solve at least somewhat reliably.

I use it as a confidant for bouncing ideas and situations. I will specifically ask it to use dialectics, or phenomenology, or whatever.

Otherwise, I just use the higher quality stuff online. I only use local stuff if I specifically don't want the data saved.

Yeah, I would like to know this. From my perspective, even frontier models by the big players (4o, 3.5 Sonnet) can be unreliable at times, and are at best just walking the line of usefulness for a lot of "exact" tasks (for me: programming, approximation and back-of-the-envelope calculations, expertise on subjects I'm unfamiliar with, a better Google, etc.).

The only deal that would make sense for me is to get something more accurate, and these open models just go in the wrong direction. I've observed similar behavior to what you mention. I'd really like to know how people use them and for what tasks so that their performance is acceptable.

In agentic settings, cost and latency are also a large factor since tokens are consumed invisibly, so I think a lot of these systems are waiting for a trifecta of better accuracy, better cost and better latency to make them viable. It's unclear that this is coming, at least it hasn't been the trend so far.

o1 is 10x better than 4o and 3.5 Sonnet for non-trivial coding tasks. I don't even bother with the other models for coding-related tasks, it really is a big difference.

For isolated coding tasks I've been using o1-preview instead of Sonnet for a while now, I just didn't mention it. Haven't had a chance to test o1 proper, but I assume it's also a jump in performance. However, for more "holistic" tasks which need to take into account a larger view of some other modules/systems/interfaces, I've found o1-preview can get really confident about weirdly incorrect things that end up being harder to debug than the more straightforward hallucinations of Sonnet, and so I mostly revert to Sonnet in those cases.

I tried not to make too big a fuss about the exact models I'm mentioning, since it's pretty clear that the strongest open model, Llama (discussed here), is not comparable to inference-time compute models.

And for agentic settings which I mentioned above, o1 just tips way too much into expensive & slow territory to make it useful, so my prediction is that it will be limited for direct (chat-based) consumer use for the time being.

Yep. Calling 4o a “frontier model” when o1 is available seems questionable.

I only use 4o when I need web search incorporated in results.

Can it really be called efficient in the capitalist sense until they figure out how to actually turn this stuff into a viable business? Apparently OpenAI is even losing money on the new $200/month tier.

https://finance.yahoo.com/news/sam-altman-says-losing-money-...

I believe they’re profitable on all unit economics other than the $200/month tier. The users who opt into this are the absolute “highest” users so you end up with an adverse selection issue with this plan.

That doesn't seem to gel with the fact that they lost $5 billion overall last year, mostly before the $200 tier was even available.

They said profitable unit economics, not profitable as a company.

So P/L incrementally, ignoring capex and salary?

I have the local models, why am I supposed to care how a young company is wasting money?

Every local model worth a damn is also the product of a company wasting money, so if you want better local models in the future then it's still your problem.

Lots of 'what ifs'

but I still have these models on my computer.

I'm not sure how a few multinational mega corporations "competing" with each other is an impressive example of capitalist market efficiency. After all, this is isn't GPTx vs Gemini vs Llama vs Claude - it's Microsoft vs Google vs Meta vs Amazon. None of which are fair actors in the global marketplace.

We have offline models and multiple online models of growing high quality.

Its impressive.

(and if you want to see the flip side, our regulatory captured Medical industry still uses faxes)

Wait, which megacorp is Claude?

Amazon by proxy, like how GPT is Microsoft by proxy:

https://www.aboutamazon.com/news/aws/amazon-invests-addition...

> The competition has been the best type of brutal.

I’d counter with “The competition has been the worst type of brutal.”

https://finance.yahoo.com/news/ai-destroyed-google-promise-c...

Tech’s quest for the best chatbot so they get all the grift dollars has torched climate change progress.

"Torched climate change progress" is a wildly overblown take: https://www.sustainabilitybynumbers.com/p/ai-energy-demand

It’s well known that humans have a tendency to hallucinate!

What about the patient that got their misdiagnosed illness finally solved?

What is more important, the pains of Humans, or exhaustion of resources.

You want to go tell patients that rocks in the ground and unconcious trees are more important than their pain? Or tell a student that their educational future is less important than tress?

I think you could argue that competition indeed makes the market efficient, but that we shouldn't conflate that with capitalism itself. Capitalism, in my opinion, can at times prevent competition due to the required capital investment to compete. E.g. even OpenAI with their golden bullet couldn't get there without the capital investments from big tech? Might be wrong here of course.

I specifically mentioned Capitalism because it required the investment.

It was Capitalism, not market efficiency.

Fair enough!

I will agree that the free market has really shined here as far as product development is concerned. I have a hard time believing any government effort short of a war-time incentive would have produced anywhere near the same results.

That said, I think we're also going to see exactly where capitalism always fails - the negative externalities. If AI goes off the rails and ends up deliberately or incidentally wiping us off the globe, it's likely to be because of this relatively unregulated space race.

I have given up on AI folks using a scientific definition of “world model,” yet I am still amazed at how viciously dishonest NVIDIA is being here:

  “Cosmos learns just like people learn,” the spokesperson said. “To help Cosmos learn, we gathered data from a variety of public and private sources and are confident our use of data is consistent with both the letter and spirit of the law. Facts about how the world works — which are what the Cosmos models learn — are not copyrightable or subject to the control of any individual author or company.”

Cosmos definitely does not learn facts about how the world works! This is just a ridiculous lie. It accumulates a bunch of data hopefully demonstrating how the world works, and hopefully some facts fall out of it. Given that this failed completely for Sora, which obviously knows nothing about physics, I am confident that Cosmos also knows nothing. It has no facts, just data. And unless they somehow integrated touch sensors it doesn’t even get physical data the same way toddlers do. So “learns just like people learn” is also a lie.

Some AI hype is people getting ahead of themselves and believing their own marketing. But here NVIDIA is just lying their asses off, presumably to stoke investor hype, but also because they’re trying to monetize a bunch of copyrighted data they stole. These are bad people.

Reply on news.ycombinator.com