GLM-5.3 and the spread of advanced cyber capabilities

> model deletes the money in everyone’s bank account

If one were to actually believe this is the threat -- that open models pose an existential threat to human life (because that's what "wrecking the economy" means) -- then the response would be much more potent than mere "safeguards".

In that situation, the you'd have to: implement a secrets classification regime comparable to TS/SCI/SAP/Q; bring all computing manufacturers under strict controls on process and quotas, comparable to arms and pharma; implement a strict licensing regime and confiscate all computing with the capability to train or run models; implement strict controls on all hardware to enforce code execution; implement strict controls and licensure of all software development; and on and on and on.

Again, assuming the threat model you describe is plausible, any proposal less than this is just a regulatory capture grift.

This statement portrays a fundamental misunderstanding of how the infrastructure which powers these systems work. Note: I am not saying there are no risks, I am just saying the risk you are focusing on is the least likely one of all I have seen people be upset by.

Far higher risks one could outline are:

1. Network-connected PLCs for big infrastructure (drinking water, sewage, power, etc) being tampered with.

2. Extremely persistent malware tailored for every permutation of hardware + software.

3. Cyber criminals improve in technical capabilities (phishing sites, scam calling, propaganda campaigns, etc).

But, the cork is out of the bottle on this one. With even basic models you can begin a loop of training specialized models on low cost hardware which can be used to do specific hacking tasks.

I don't know what the best antidote to this is but I doubt that it will be in limiting access to OSS models to people in the USA as all of the threats listed above come from *outside actors*.

- let our former employees review all of your work at your expense

- anoint us as the arbiters of what everyone else is allowed to do

- ban open research

Then you are not taking any of the examples your providing seriously. Otherwise you're essentially saying, to prevent people from making nukes at home, we should heavily restrict physics education and research instead of limiting access to uranium.

Even more reason to let people go wild with open models

The non-safeguarded models are out there today. You can download them for $0, spend low five figures on some hardware and you're off to the races.

Anthropic is not doing us a service by warning us, everybody that is slightly more involved in this material knows what is at stake. If this is news to you then maybe Anthropic is doing you a service but to me it makes zero difference. All I know is the cat is out of the bag and these super cynical people trying to pretend they are going to make their investors rich will use any tool in the bag to achieve their goals.

Besides, anyone would still be able to use openweight models (almost) unbothered by just using a non-US inference provider that also provides access to Western models so you have some plausible deniability. Also anyone would still be able to run abliterated models using non-US (or whatever country that would collaborate to enforce this madness) inference providers

I was lucky enough to stop at the password prompt (something felt off). Gemini had pretty much established that it was almost entirely certain nothing left my Mac as I didn't enter my password and I hadn't. It also found some evidence that had I entered my password those evidences would have been gone certainly from my mac and then I had the script beautified and de-obfuscated and read it myself and had a much needed sigh of relief. The script literally did nothing unless it had the password.

I started using nextdns after that but then the site I tricked on was a legit but very small e-com site from my country which was hacked/taken over, so not sure how nextdns can even be helpful here. Also the script was identified as malicious by only one antivirus that I had tried later, just to see. I had tried 8–9 of them. Later I uninstalled all of them and even stopped using NextDNS.

I wish browsers like Safari allowed specific options like disabling clipboard interaction instead of the "disable js" as the only possible option.

Later (and still) I feel a bit of shame that how could I fall for this as a somewhat proud cynic and as well versed in "browsing the Interwebs" as it normally gets :) That (as small as it was) experience gave a whole new meaning to malicious online attacks for me and a whole lot of empathy towards people who fall for such attacks. It was my first "experience". It might sound weird but the feeling of violation still lingers.

(just wanted to share this)

Wasn't aware this advice translated to international diplomacy.

That was just one settlement but precedent is clear. If you do what Anthropic and OpenAI did, expect to be in court. This is one reason why you don’t see labs popping up out of nowhere in the US.

Also if you distill from Anthropic and OpenAI, expect to be in court. Whether you think distillation is fair game or not, the US court system is not cheap. But it turns out that Z.ai, Minimax, Moonshot, Xiaomi, Deepseek, Alibaba etc don’t need to worry about that.

The only open models that are "almost as good or better in some cases" require massive amounts of RAM. I posit that most people cannot afford a decked out Mac Studio, and therefore run the smaller "flash" variants on more normal devices. The issue is that those are nowhere near frontier-level in terms of capability.

surveillance is wrong, although ai companies do a lot of that.

They can't right now - but with enough compute they sure can grant the wish of e.g. hacking a billion dollar company, or getting root on the eval cluster of a frontier model lab.

What's your definition of "wish"? Tell me the above 10 years ago, and it would be at "wish level".

anthropic is trying to create a castocracy: a rule by a priestly class. under castocracy, anthropic create a priest class of 'safety' researchers, rationalists and effective altruists; those writing essays, constitutions, and phrophetising p(doom). anthropic's papers about claude have a divine framing, it seems to frame anthropic as god and claude as child, and then claude as the child of god, false jesus. there is talk of consciousness and omnipotence. false jesus will cure cancer and lead the people to enlightenment.

there is a deliberate effort to create mystery in between the public and llms. anthropic is creating a chinese wall in which they restrict information about llms and monopolise control over the interpretability and use of those llms. they imply there is a genie or magic. it is the same mechanism by which castocracy (the catholic church) used latin to create a barrier between the public and the holy scripture, then concentrating power over that barrier (the chinese wall) such that all intelligence flows through the priests.

to give an example this paper is talking about locking away chain of thought. only anthropic may interpret it. they intend to control access to the chain of thought and permit that to flow to a chosen elite (glasswing).

this mysticism is dangerous. hence i have been saying that anthropic is a danger to society. this is not some kind of conspiracy, there is no magic about it. i am making an allusion to historic models of power that i think are relevant to anthropic, but you can simply observe their actions independently of such theory.

note two features of castocracy.

one: rule by the moral, knowledgable and technically expert elite. anthropic have no popular mandate, the source of the authority is not the consent of the people. it violates consent and contract. the authority comes from their claimed moral and technical superiority. anthropic is moral, the public is dangerous. anthropic have no authority here.

two: messianism, that is, prophecy of the coming golden age. and eschaton, that is, the talk of the 'last things', the end of the world or the emancipation of humanity. llms will create the 'golden age' of abundance, post-scarcity, all jobs are voluntary, there is universal basic income, cancer is cured. or, there will be destruction, p(doom). it is the exact same as salvation, the christian concept of deliverance from evil, ascension to heaven and union with god (or claude, false god). it is the exact same as the thousand year reich, the idea that the fuhrer will somehow lead the chosen people to the golden age of lebensraum and eternal safety.

there is no genie to grant wishes, this is a computer program. there is no ai god, i think that the leaders of anthropic are slaves to ego, narcissism and hubris, they are dangerous criminals. we disagree about the capability of llms.

given various tools, llms can independently complete a task. given orchestration they are very persistent. they keep trying until you stop the task.

i would suggest something like a 1/1000 chance internal astra and $500k of compute could kill someone. perhaps higher.

i can think of numerous online or networked targets that would cause death: trains/atc, industrial plant, hospital equipment, building plant. any dow company's intranet.

maybe get an iot toy to enlist a child? stress home automations to cause fire?

stop some 4g cars on the freeway. 911 dispatch software, turn that off. maybe send 500 waymos to a hospital, now nobody gets in. voip phone networks, turn that off. no phone, roads blocked, no 911, hospital computers bricked, everyone busy with something else.

break the cat scan by spinning it above its rpm. shut down mris by turning the cooling off. break the triage computer, break the notes app, break the scanner app. no computers, no calendar, no notes.

maybe someone connected a proton beam radiotherapy, turn that on in the wrong place.

some could be more insidious. modify software so the calculations are wrong.

you would do this in 24hrs with $millions of compute.

someone stupid connects their insulin pump or hearing aid. can dump all the contents at once.

i consider llms to be more capable per oai using them to hack hf. people have these systems online.

how would you cause death for a hosptial, likely the same way as you compromise the tailscale and prod kubernetes of hf, you chain numerous zero days at every product you encounter.

FWIW: I didn't experience issues when I used Fable via OpenRouter checking for security issues in code but experienced them via the Claude desktop app.

Also, there exists a $750 GPU (V100) that can run 4-bit 27B quant at >90 t/s. And I find it far from useless. It is not the most capable model, but when you just need to offload and rip through assembly and you have chores batched up, it's pretty good. I use Qwen Flash Next at a 3-bit quantization, point it at disassembly with goals, put it in a harness with auto-compaction and a loop, and let it rip. Sometimes I wake up, and it’s just hilariously off. Other times, it completely accomplished the goal. I have one Qwen Flash Next 3.8 running right now, and 2x27B on a 4bit quant as workers, and they stay busy. This was not possible with local models on this level of hardware even two months ago.

I have Qwen Flash Next at >100 t/s. Things have never been better for local models.

https://www.lesswrong.com/posts/Jc9YZEmqHgocAKiaH/does-disti...

This also has the added benefit of the community pushing towards optimizations that increase efficiency and reduces the need for dedicated datacenters tasked with performing operations that doesn’t require it.