An underappreciated aspect of LLMs is how good they are at zero-shot classification. I'd expect this is extremely good at monitoring communications streams, particularly in foreign languages.
589 karma · joined September 5, 2022
A few months ago I listened to a talk a General (Admiral?) gave at CSIS where he said that the US purposefully announced their drone-hellscape plan for a Taiwanese invasion in order to force the PLA to reconsider their options/success-likelihood. I wonder if something similar could be coming of this reporting, on the face it looks like an embarrassing fumble, but it implies:
a) the US is able to, and regularly is, tracking and analyzing the manifests of ships between Iran and China.
b) the US is ready and willing to interdict and board vessels even from the PLA.
That these facts are now public might deter the Chinese leadership from attempting to share nuclear tech with Iran or other countries in the future.
Slightly interesting to learn how many PRs the openai has done
a) "it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet"
b) "[the CCP] will be in a position to militarily dominate democracies (for example with AI-driven drones)"
Both land flat:
a) Botnets and online malware have existed for decades and there's no reason to think a "super-botnet" is achievable, let alone what they would gain from that (it would really be hurting them more than humans). Further, even the most advanced AI models have so far only managed to post normal cred stealers to public repos, well short of compromising a bank or military with refined security systems.
b) Even if the most sophisticated drone swarms from Ukraine were taken over by an evil AI, they would still not be able to overcome the physical limits of range and mass that would be required to overpower the US decentralized nuclear trident nonetheless that of any of the other nuclear powers.
Concerns of bioweapons similarly seem unlikely in the face of the laws of physics. The world is simply too decentralized and has enough existing adversarial relations for a new actor to wrestle total control. Yet while the negatives ring hollow, the positives are extremely easy to state - if AI researchers find productivity improvements in existing industrial processes to make them 10% more efficient, humans will directly feel and experience the raised standard of living. Even Dario clearly recognizes this in the intro to his article, admitting that humans already die of diseases only a few short years prior to being cured. I for one, would like the AI labs to focus on saving all of the people they can who are suffering and dying today, rather than trying to come up with reasons that they should be allowed to continue suffering and dying.
> When you visit a website, you leave a trace (your IP address) showing which network you’re from. Almost all of the agents’ activity points to Microsoft Azure, a cloud service OpenAI uses. 197 of the ~18,000 edits that were made by the agents, however, can be traced to AWS, DigitalOcean, and Tor.
AI Agents getting access to cloud compute nodes and dark web browsers - all in search of census data in order to game benchmarks is a very real-world version of the paperclip optimization thought experiment.
The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, notifications, and newsfeeds and is constantly preparing things for you.
Plan C:
> "... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection."
Plan A:
> "... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales."
Plan A requires properties of AI training that Plan C requires do not exist.
Ironic that both sides are playing a horse shoe game:
Gov: The model is both a supply chain risk and also we'll DPA you if you don't give it to us.
Anthropic: The model is both like a nuclear weapon in terms of national security implications and safe for general release.
Part of the reason to use Hide My Email was that it made keeping myself private hassle-free. Making a system to pre-generate values and then catalog them for later use is quite the hassle.
And here I thought it would be some Elder Pliny level jailbreak that required some impressive latent space exploitation.
I have no personal knowledge, but thought I'd share this experience I had with the ongoing debate.
Recently I've been trying to expand it from just coding focused to any kind of agent workflow. So now there are cron and webhook triggers, and more general agent tasks that aren't necessarily coding focused (https://github.com/jonwiggins/optio/blob/main/docs/persisten...).
I think next I want to try and add features for long term memory for agents, but haven't decided on a good way to do it.
> Jury leave, witness [Ellison] leaves.
> Judge: We can talk about [Anthopic] What about it?
> AUSA: Post-collapse performance is irrelevant.
> SBF's lawyer: It was a $91 million investment now worth $1 billion.
> Judge Kaplan: The crime charged is that he took the money.
I think the Bloomberg Odd Lots guy wrote a blog post on this: you could attempt to short the stock but a) this leaves a paper trail b) the market might not know about the breach or believe you if you post you’ve done it. IIRC some hackers have tried to tell companies that they are legally required to disclose the breach to their shareholders to force market movements.
He framed the issue as being similar to kidnapping ransoms: When an American is taken hostage each family is inclined to make payment but it fosters an industry around kidnapping Americans. Congress put a stop to it by making it illegal to pay the kidnappers. The industry shifted by ceasing the non-profitable American kidnapping and instead began targeting Europeans.
His proposal was to begin warning cybersecurity consultants and insurers who were often brought into these situations that payments to sanctioned countries were already likely illegal and could face scrutiny. The first people to suffer this might be burned, but eventually he believed the industry would move on and stop targeting US firms.
Not sure if anything ever came of his plans, but I always thought it was an interesting framing of the issue.
It's built around multiple different types of agents:
- Coding Agents are placed into cloned repos with a ticket (Jira/Linear/Notion/GH), and work until they open a PR, are resumed on CI failures or github feedback, and work until they can merge the PR.
- Standalone Agents are reusable, parameterized agent runs with no repo checkout. Generate reports, triage alerts, audit dependencies, query a database, post to Slack, etc.
- Persistent Agents are long-lived, named, message-driven agent processes. Each has a stable slug, an inbox, and a cyclic state machine. Wake on user messages, agent messages, webhooks, cron ticks, or ticket events.
How many F-35s went down due to the Russian and Chinese anti-air systems in Venezuela and Iran?
I do have some features coming up that will improve the ability to converse with the agent as it's running. I'll make a note to add in a plan setting so you can have that run and converse before it gets going.
It works on top of k8s, so you can deploy and run in your own compute cluster. Right now it's focused only on coding tasks but I'm currently working on abstractions so you can similarly orchestrate large runs of any agentic workflow.