blazespin

trump literally said he wants venez to return the oil it 'stole' when it nationalized.

There is a significant risk of uncertainty in all of this, the most damaging aspect really. If AI improves, and it is threatening to, then growth in SaaS may decline to a point where investing in it needs to be reconsidered.

The problem is, nobody knows how much and how fast AI will improve or how much it will cost if it does.

That uncertainty alone is very problematic and I think is being underestimated in terms of its impact on everything it can potentially touch.

For now though, I've seen a wall form in benchmarks like swe-rebench and swebench pro. Greenfield is expanding, but maintenance is still a problem.

I think AI needs to get much better at maintenance before serious companies can choose build over buy for anything but the most trivial apps.

Nobody collapses, everything just shrinks.

And we're seeing that in the labor numbers.

Sometimes things are harder to see because it's chipping away and everywhere at the margins.

who likely wins, fify

if you read the paper that is the intention, to guide stuff like lean.

i don't think llm is a great pure rlvr

Advanced math solving, as the results indicate. Informal proof reasoning is advancing faster than formal proof reasoning because the latter is slow and compute intensive.

I suspect it's also because there isn't a lot of data to train on.

Verifying math requires something like Lean which is a huge bottleneck, as the paper explains.

Plus there isn't a lot of training data in lean.

Most gains come from training on stuff already out there, not really the RLVR part which just amps it up a bit.

More training data on advanced math. Lean is cool, but it's mostly about formalizing stuff we already know.

You can't process untrustworthy data, period. There are so many things that can go wrong with that.

Given they made a geopolitical accusations and dozens of mainstream publications repeated the "thousands of requests per second", this seems like a grossly negligent flub that should not be dismissed as a mere typo.

I frequently, alone, do 1000s of requests over a period of time, especially ones that are mostly cache hits, which can be $10-$50 in API costs.

This was not a "large scale" attack by any means.

KIMI just proposed linear attention. I mean, one breakthrough, and blammo, the whole story changes.

That was funny. Weird the pdf is missing all those pages. You'd think they'd just redact.

the way it will work is ai slop will rank high, and pomelli will generate the best ai slop.

Good thing to do regardless of the source, AI or Human, right?

I do verify by using topics I'm an expert in and I find hallucination to be less of an issue than depth of nuance.

For topics I'm just learning, depth of nuance goes over my head anyways.

Very unlikely Emmett lied. What would be the point.

No sub. Did they just re-tweet reuters or did they separately confirm?

Also in the news today, the 86B share sale is back on.

I mean... come on.

Post this on a thread or something. Unattributed complaints like this are dangerous.

The difference is nearly everyone else doesn't stand to seriously benefit from the implosion of OpenAI.

Yeah, I can't imagine why DeepMind would possibly want to see OpenAI incinerated.

When you have such a massive conflict of interest and zero facts to go on - just sit down.

also - "people I respect, in particular Helen Toner and Ilya Sutskever, so I feel compelled to say a few things."

Toner clearly has no real moral authority here, but yes, Ilya absolutely did and I argued that if he wanted to incinerate OpenAI, it was probably his right to, though he should at least just offload everything to MSFT instead.

But as we all know - Ilya did a 180 (surprised the heck out of me).

The obvious answer to twitter. We need vanity metrics though if it's going to win the optics battle.

Some kind of push mechanisms would be great too.

One of the most admirable things I've seen done in a loooong time.

If there is another board and Ilya is not on it, I mean... ffff it.

Yeah, that's an interesting point. But I think with appropriate RAG techniques and proper citations, a future LLM can get around the copyright issues.

The problem right now with GPT4 is that it's not citing its sources (for non search based stuff), which is immoral and maybe even a valid reason to sue over.

A hostile relationship with your cloud provider is nutso.

I think without looking at the contracts, we don't really know. Given this is all based on transformers from Google though, I am pretty sure MSFT with the right team could build a better LLM.

The key ingredient appears to be mass GPU and infra, tbh, with a collection of engineers who know how to work at scale.

If they didn't want to be responsible for running and paying for an actual service like ChatGPT, they should have spun it off to someone who did.

The notion they could just build it and they will come is ludicrous. Sam understood that and was trying to figure out a model that could pay for itself.

Obviously there will be mistakes made along the way. That's how it goes.

Don't forget. ChatGPT has competitors. A lot of them and they're getting pretty good.

The difference with OpenAI / GPT is a dozen or so primary engineers plus a few billion dollars for GPUs and you have a competitive version.

And if those primary engineers get sucked out of OpenAI, OpenAI won't be able to compete.

OpenAI is a different animal.

SamAltman has the cache to pull out those engineers. Particularly because Ilya's vision doesn't include lucrative stock options.

The staff calls the shots. The money will go wherever the talent is.

Yeah, the seed of this disaster was probably planted a long time ago, like when they created the hybrid, when they got money from MSFT.

Maybe Ilya doesn't want to be at the forefront of AI building I guess. He probably thinks he can lead it better by not being at the table.

Or who knows.

Yeah, the message from the board hinted at something very bad. Why scare people like that?

If they just said philosophical differences or something that would have been sufficient.