Hacker News Reader

Comment by hgoel | Hacker News Reader

Well, there go any such claims of dangerousness in future models, regardless of if they are true or false.

No one's going to risk building anything important on these models if the government will randomly order the use of the model to be discontinued by all foreigners, regardless of if they are in the US or not. Just a matter of a foreign company catching up to take the commercial market for such models (though, as the US often does, they'll ban the competitor, so actually we'll have a situation where the backend uses a different model in only the US).

View on HN

I think it’s more like “there goes the semiconductor boom predicated on monetization of ever larger models.” Once the IS government acts out of capricious fiat because a model becomes “too good” and they demonetize it, the entire shell game collapses. It’s times like these, with oil scarcity planet wide, fertilizer scarcity, and now ham fisted meddling in the bubbles expansion, we can be thankful we have an octogenarian senile stable genius with twenty two specialist doctors and a disdain for the rule of law at the wheel!

The thought that this would also destabilize the AI bubble did come to mind, but the current government loves to crash the market on Fridays, only to backpeddle on Mondays.

A related thought though, the AI boom is predicated on the idea that everyone's going to want or need all this "mass produced" intelligence. But what happens to that when you go from being able to claim to have a total market size of ~8B people, to ~400M peoole? I think the reason to push ahead at any cost evaporates.

It's honestly not the worst strategy: make the dangerous move when you have the most tolerance, and then everyone can figure stuff out and make the landing on Sunday.

Options and futures don’t wait and a lot of stuff trades 24x7. You can do your puts right now, and banks and market makers will meet you now if you’re big enough. The landing for Main Street will be more of a horrible traffic accident that happened days ago and they just woke up in the flaming wreck of their financial life.

There’s no back peddle once you’ve demonetized by fiat for being too big. Once you doo it you prove you will do it again for the very reason the bubble is inflating. It’s a binary one way door and it’s already happened. It’s like killing the supreme leaders entire family and maiming him and expecting he will be happy to meet with you, that ship has sailed and magical thinking won’t undue the incredible atrocity you visited on him - you’ve created a mortal enemy for all time. This is an administration of mental gnats.

Yes I rather think “just do the Friday-Monday thing with it” will get applied, but without the realisation that once this trigger is pulled, it can’t now be “un-pulled” at a later date.

Agreed. The timing here is interesting as well. 5:21PM ET on a Friday. Like they know this could roil markets and they're trying to buffer that a bit (and maybe they're really hoping this deal with Iran is actually real this time and figure that will help offset the effects?)

39 times is the charm I guess?

The Trump administration would never do anything to manipulate the markets. /s

From reading the post, I think it's more likely that anti-jailbreaking is going to become much more strict and prone to false-positives.

> We received the directive from the government today at 5:21pm (ET). The letter did not provide specific details of its national security concern. Our understanding is that the government believes it has become aware of a method of bypassing, or “jailbreaking” Fable 5. We reviewed a demonstration of this specific technique being used to identify a small number of previously known, minor vulnerabilities. These vulnerabilities all appear relatively simple, and we have found that other publicly-available models are able to discover them as well without requiring a bypass.

But no matter how conservative they make the anti-jailbreaking, the risk doesn't go away. There are so many logic "holes" that are ambiguous and can blur the line between a jailbreak and legitimate use.

If every time a jailbreak is discovered, the model has to be turned off and jailbreak prevention updated, the effect will be the same regarding how willing users are to adopt it.

Also this falls into the "right to bear arms" thing: if LLMs are limited legally, then illegal LLMs will be the superior choice. This is pretty much the plot of Cryptonomicon and Corey's take on I, Robot

Except there’s a large hardware barrier to entry, which for now seems effective.

Related note. Cryptography has been subject to export controls for years and manufacturers bend into pretzels to meet the laws, regulations, and policies.

Anti-jailbreaking and passport verified access to model families.

I don't get the emphasis on known vulnerabilities. The jailbreak already works on previously known exploits? That seems a bit weird.

Likely models by Anthopic can no longer be reliably trusted as it'll subtly sabotage your codebase you're working on.

Gov just need some national security orders for Anthopic not disclose it to the public and to implement whatever they've done to Fable 5 to existing models.

No no, you never ever build ON the models. You build WITH the models.

Never let a 3rd party LLM be the core of your product or it can be changed or taken away at any moment.

What you do is use the frontier models to build a deterministic set of tools that does what you want and MAYBE put in a small core of LLM for the ambiguous stuff you can't make deterministic (yet).

And make sure you can swap that LLM core to any other provider (even local) and have a playbook ready for that.

I mean, lots of Americans would risk building something important with it in that case.

With how much foreign talent is involved in the tech world?

its establishing a bifurcation in the tech workforce at private companies into citizens and 'foreign nationals' for security reasons. that's not a very pretty precedent. pretty destructive given the pervasiveness of international workers in us tech. its just going to encourage organizations outside the US to further develop their own training methodologies and models.

this cleaving of the us from good relations with other people is sold as a consolidation of strength. Made from a position of baseless hypernationalism, its just going to make the US much less relevant on the world stage.

Yes, it's actually a consolidation of weakness

Americans didn't build the current AI tech.

Reply on news.ycombinator.com