Hacker News Reader

Comment by himata4113 | Hacker News Reader

I think this is as good as time as any to bring up that fable/mythos weights are one mistake (malicious or not) away from being leaked to adverseries or available in a random torrent.

Imagine this, fable weights are likely distributed to hundreds of datacenters with likely thousands of people directly or indirectly having partial or full access. I just don't quite buy that a 'world ending' fable/mythos model would be treated like this, mythos I could maybe believe that it runs inside government compliant datacenters which have a proven track record, but something as valuable as a 'world ending' model invites state sponsored actors to put in significantly more effort into exfiltrating it.

Whatever the real story is I doubt this is as ground-breaking as anthropic claims it to be.

View on HN

We've been told models are too dangerous since gpt2.

There comes a point where you not only want the boy to stop crying wolf, but hopefully be eaten by one.

I don’t fully agree with this sentiment. Just because we were told something and it didn’t come true before, it doesn’t mean it can’t come true now, and that the capabilities are there now.

At this point we have enough of real evidence from project glasswing like the massive Firefox security patches from Mythos findings. This isn’t crying wolf.

I’m very glad that they’re actually being grownups and not yolo’ing something this important, and are working with groups until we can secure critical infrastructure before making this more available.

That example is brought up a lot, but in retrospect the concerns about Gpt2 were pretty valid: https://techcrunch.com/2019/02/17/openai-text-generator-dang...

>OpenAI said its new natural language model, GPT-2, was trained to predict the next word in a sample of 40 gigabytes of internet text. The end result was the system generating text that “adapts to the style and content of the conditioning text,” allowing the user to “generate realistic and coherent continuations about a topic of their choosing.” The model is a vast improvement on the first version by producing longer text with greater coherence.

>But with every good application of the system, such as bots capable of better dialog and better speech recognition, the non-profit found several more, like generating fake news, impersonating people, or automating abusive or spam comments on social media.

Does OpenAI suspend users that they detect impersonating people, spamming people or generating social media posts though? It feels like they belled the cat just to charge a toll for abusive usage, and now that we've seen LLMs used for real warfare, the moralist angle isn't very defensible. OpenAI and Anthropic are both willing to help kill people if the price is right.

No, they don't

> or automating abusive or spam comments on social media.

Actually, the biggest problem is the automation of inane comments on X. Which is admittedly quite surprising - I would have agreed with OpenAI at the time.

And Guinness thought the t-test was too dangerous to announce publicly over a century ago

I don't get the analogy.

Guinness didn't want to make it public because he was afraid competition would start using it, and then lose his company's advantage.

In the current context, the retention didn't happen because of Anthropic. On the contrary, the company wanted to offer Mythos/Fable.

I think that is a little bit disingenuous, Guinness just saw it as a trade secret. Not something that was too dangerous to release to the public for the public's own 'good', just that it could be bad for their business.

Mythos somehow leaking and becoming usable by all humanity in a self-hosted manner would probably be the optimal long term outcome.

We can’t buy affordable memory for Fortnite but everyone is running Mythos locally? Is this the year of Linux on the desktop too?

I think they meant more "every country can host it", as opposed to "every human hosts it at home".

I'll take what I can get.

If models leak and the AI bubble implodes, memory could become cheap again.

Not if happens before the new memory factories get built

It's time for this bubble to pop already. We need to get past this centralized, unsustainable, undesirable, Big Brother stage of AI and get on to the "AI hosted in your kitchen" phase before we mess up the balance of power too much and effectively backdoor the US constitution to power hungry technocrats.

so... long term here meaning until the next frontier model comes out?

In a year you can watch a documentary on the mythos of Mythos (that outdated model in case you forgot already).

It honestly is shit, I tried 2 simple prompts, both times got blocked within 2 minutes.

Just wait patiently for the chinese labs to inevitably distill Fable without the 'policy layer/neurons' and offer it for pennies.

That one will be fun to see.

Sounds awesome. I can barely use Fable because or Anthropic's obnoxious fearmongering. It almost always gets downgraded to Opus 4.8. If the chinese somehow distill it into an open weight model, then I'll finally be able to harden my projects and networks.

i believe it's more complicated than that. i know that nvidia offers TEE for their overpriced offerings. i would assume they make use of that so the weights are encrypted.

this doesn't mean it cannot leak but it would be a major undertaking.

this is why anthropic isn't that worried about having Elon service their models. the workflow would be something like handshaking with the nvidia TEE, provisioning it with your keys and then uploading encrypted weights. there is probably also a timer in there so you can't continue operating the nvidia box with the stolen weights without a heartbeat signal.

There's so many points of failure before it ever reaches gpus.

Hollywood solved the problem. digital movie distribution to cinema doesnt leak anymore until they are made available on streaming platforms

hollywood doesn't have to worry about state sponsored actors and hollywood does infact still leak movies early, but usually in some unfinished form with a bunch of directors text, missing vfx, etc.

Im of the opinion it’s all marketing and the government has continually shown themselves to be insider traders.

I don't think Anthropic is claiming it's world ending? Just that it has offensive cybersecurity abilities which can be dangerous

Cyber-warfare is pretty world-ending these days. Our relatively peaceful world is built on top of mutual destruction.

See: https://news.ycombinator.com/item?id=48633023

Why would you believe anything an intelligence agency says? Half their job is spreading fear and obfuscating the truth.

And that mistruth appears in delightfully insidious ways, like seemingly well-meaning comments opening up about "half" the job being lying when the point is to shift the Overton window away from the actual numbers, whatever they might be.

(obviously not serious, but it's fun to probe what we can actually reason about when every message might be adversarial)

Half? What’s the other half?

lookin at their high school sweet hearts private DMs

What they claim is immaterial.

Security has always been and always will be a game of cat and mouse.

We all need cyborg cats to hunt cyborg mice. There's no other compromise unless you want a rat infestation.

Fido the Rat Thing

Yup, but apparently our cyborg cats can only be kittens and the cyborg mice are probably going to be like 4 feet tall. At least according to the US government.

We could also encourage people to bring us cyborg cobras.

Is the model structure going to be easy to reverse engineer just from the weights? Also, I'm going to guess it's an MoE and thus it's possible there's no single machine that hosts all of Fabel / Mythos.

kvcache residency requirements and general latency for good throughput wants good locality, but you're right it could be split across multiple different parts of a single datacenter, but as I mentioned before the weakest link is before the model is ever loaded onto the gpus.

as for reverse engineering I doubt it's something that state sponsored actors would struggle with for too long.

Remember the Clipper Chip back in 1993 because personal computing was considered too dangerous a technologie to be accesible un-controlled/suprvised to the general public?

Most likely it'll try to rewrite the whole world's CSS and get the custom properties wrong.

It's probably a huge file though, I would guess it's at least a multi terabyte file.

The spotify leak on the front page a while back mentioned a 300TB torrent

I suspect that might have been more than one file - I wonder how many files make up the shipping bundle for Fable 5?

Yeah, and I also don't buy the semi-conspiratorial beliefs of some of my friends who think there is super intelligence somewhere being built.

If fable is the best we got everybody still has some more time till the apocalypse I think.

most of it is overstated because of marketing. in fact the ban looked like an inside job by the current administration to play with the stock value. if the ban did anything it was to make every n00b agree that Anthropic was so far ahead of its times it needed to be banned.

Stock value?

Not publicly traded as they haven't IPOd yet (the part where those closely held shares become publicly traded), but Anthropic does absolutely have shares, and there is a valuation on those shares. I don't quite understand how the administration is playing this, beyond the normal corrupt protection racket stuff they're doing all the time now, but Dario is absolutely gearing every public apocalyptic prediction on juicing the "stock".

I mean, if the matryoshka doll rent-a-gpu farce that is SpaceX somehow is worth $2T with their garbage child porn xAI, Anthropic must be worth quadzillions.

I feel like I'm loosing my mind with the way people are falling for the marketing with every iteration. Is it better, sure but it's still just a LLM.

These days an LLM + a harness can one-shot playable Minecraft clones. This tech has come a long way since it was first released.

You are saying: it turns out that one-shot playable Minecraft clones are actually pretty simple. Maybe it seemed a hard problem to programmers, but why not just say that the verifiable-rewards training has shown that their skill is unusually simple?

What a great example of the no true Scotsman fallacy.

I am saying 3 years ago there wasn't a snowballs chance in hell I could one shot a playable Minecraft clone, and there has never been a snowballs chance in hell a human developer could do so in 45 minutes.

The difficulty of the task and human performance on that task hasn't changed. LLMs performance on that task has changed dramatically.

it could almost certainly regurgitate something like minetest

its in the training data, probably many times, and it was then, too

And that's exactly why 3 years ago we saw people one shotting it in 45 minutes :)

"Some idiot on Twitter did a game tech demo" has been a thing since, like GPT 3.5 or 4.0?

It has. We can see 3.5 started with text adventure games https://medium.com/@duncanariey/using-chatgpt-to-make-the-it... and with 4.0 it moved up to pong https://x.com/skirano/status/1635736107949195278

As the generations increase, so too has the complexity of the games it can one-shot.

But nobody has nailed "this is a game anyone wants to play" or "professional game development life cycles are shorter" so it's not particularly impressive.

I'm still waiting for any evidence of LLM economic impact in the real world.

Yeah, well I doubt any economic impact will from LLMs will ever come. I doubt anyone is working on nailing those things as we speak. I doubt the results my team and I are getting with agentic engineering are real. I doubt anyone can take increasingly better tools and translate that to economic value.

If you're doing such amazing things why the need to gush about Minecraft tweets?

Why wouldn't I be?

Those demos are put out by non-engineer AI-bros who start from a blank slate and just test raw one-shot performance of the model.

In comparison I'm aware of how much additional performance you can extract out of models with a team of professional engineers that are balls-deep in harness engineering.

That delta is very large.

It's exciting.

Can't wait to see what unusually simple Erdos problem LLMs will expose next, hiding in plain sight for decades and seemingly intractable for professional mathematicians who weren't aware just how simple the problem was.

Reply on news.ycombinator.com