Mythos is the best cybersecurity news in a decade
sfstandard.com
sfstandard.com
Infosec for decades has been 99% “hey I found some low-hanging fruit” only to get treated like a liability by the company you report it to, if you got acknowledgment at all. Because of Mythos though, now Artificial Superhumans can find these same vulns, and anyone could be running such an intelligence! Even better, the rich untouchable people operating this particular Artificial Superhuman can’t just be suppressed or ignored by the other set of rich untouchable people that have routinely not cared in the past. So long as it makes anthropic money, maybe we’ll actually see actual improvements in security!
IMHO the main thing thats interesting about AI assisted bug hunting is that it changes the balance of power from people who had a lot of free time & attention to the state and big business, who have money and frontier model access. It's a broadly "conservative" development in the sense that it distributes more power to groups who've already got it.
Waiting for the cyber "proxy wars" where state A equips deniable groups x, y with frontier access to undermine state B.
This presumes there is such a thing as "every" vulnerability. It is possible that ever more sophisticated, complicated, and abstract attacks become possible/discoverable as one applies more intelligence to the problem.
IF it is indeed possible to make a piece of software completely secure, then yes, more intelligent systems make the situation better, because it will always be possible to audit a system before it is ever released and make it completely safe.
That is a very big if and, as far as I am aware, remains to be seen if it's the case
-edit- They mention this possibility themselves further down, so the authors know this is a completely speculative point/article. They don't even try to make an argument about why one possibility might be more likely than the other. This article is useless.
For a purely remote attacker (although maybe we have to get clear on what distance counts as "physical proximity" because we need to clarify what phenomena spy satellites, for example, can observe), it seems pretty straightforward to me that there is such a thing as actually secure software.
You can make a very strong model of what the software computes and then prove that it never does some undesired thing. It's not common to do this at all, and even formal verification work may not use very strong models or models that capture some important part of the behavior, but it is possible to mathematically reason about what software does and doesn't or can and can't do.
To summarize some of the problems that I partly just mentioned (in no particular order)
(1) We may not have the will, the skill, or the economic demand to make software secure in a very strong sense.
(2) Attackers may subvert our infrastructure or organizations so that we don't actually apply the processes or controls, or run the software, that we expect.
(3) Physical proximity (for active or passive attacks) might sometimes include distances that are actually attainable for attackers. Maybe there are passive or active attacks involving lasers that can be mounted from multiple kilometers away, as an example. In that case most software users might not be able to be sufficiently isolated from the attackers to be protected against those attacks.
(4) Software or hardware other than the specific software whose security we're talking about might be compromised in its supply chain in a way that people don't have a plan, or resources, to detect or mitigate.
(5) Some systems might be compositionally insecure (their pieces might be secure in some relevant model, but the pieces might interact in a way that isn't secure overall, for example related to timing and concurrency problems).
(6) Our proofs of security for cryptosystems rely on unproven hardness assumptions for various primitives, some of which might turn out to be wrong.
(7) Some security properties, especially related to communications security, might be inherently unattainable even with correct software. For example, there's an argument that Roger Dingledine (Tor lead developer) once told me about that implies that no anonymity system is perfectly secure in the long run against a very powerful active adversary, unless the system is willing to make extreme trade-offs like shutting down completely in response to any attack. So it might be that we can't actually build any useful communications system that can absolutely guarantee perfect traffic analysis resistance, essentially because of inherent architectural trade-offs.
But I don't want to lose sight of the idea that you can actually meaningfully reason about what software does and so there is such a thing as the software being correct or incorrect, relative to some specification or goal for its behavior, and correct software actually does exist (which computes correct outputs for every input).
You can affect memory cells for example by repeatedly writing to cells next to them.
So if you have write access to some part of memory, you can affect memory you might not have access to because of hardware effects. This breaks assumptions in software but can be done completely remotely.
I guess that branch prediction attacks are an analogous phenomenon, although slightly less physical.
So to re-phase the question to more clearly have an answer: does there exist an attack which no one will ever be able (for all time) to come up with a defense against? (the very existence of such an attack would end the (open) internet, wholly and completely, if the only winning move is not to play...)
There will be an exhaustion of possibilities in the end. New attacks eventually run out after each surface area is hardened against those attacks.
In the limit, defense wins.
There is only one case (that i see) where this may fail. if there is a 'predicament' with the state of security: ie, if securing against attack A requires you to be insecure against attack B and vise versa (this could be a 'whack-a-mole with many different kinds of attacks' situation). But that would be 'provable'. So if such a case exists, we will know about it. And it may be true that predicaments like this could be exercised if they even can exist, we might still be able to avoid/mitigate them.
So large bets on defense winning in the end.
I’ve never seen us so aligned on a goal! Wiz is doing pretty well for itself also…
I think it's a total overreaction. But the edict was passed down, and here we are go.
Granted, given that most cybersecurity news over the past decade has been grim, both could be true...
A. Individual developers get sorta cheesed when an automated tool implies they wrote code with a TOCTOU or off-by-one error. They look for reasons to diminish anything a tool might say
B. Most of these tools do a very bad job of identifying architectural flaws. I spent a year trying to explain the Confused Deputy problem to my coworkers and why I wanted to use capability based security. They ripped it all out when I moved on to another team. Their product continues to have security problems because they forced the association of credentials between domains and didn't do a great job of it.
This is largely on me. I should have spent more time socializing the solution.
C. Management never wants to pay for tools they view as "vitamins" (as opposed to "pain killers.") This is mostly changing as ransomware attacks are on the rise.
But... Long story short... Yes... This will hopefully cause people to use scanning tools. But in a year they'll slack off and complain it's too expensive.
Mythos lays bare the folly of allowing procurement to drive technical decisions instead of IT back in the 1980s. We had KeyKOS and then EROS, but settled for ambient authority based junk because it seemed cheaper.
Social engineering requires a lot more organisation from attackers.
In a physical sense, Anthropic is giving access to who we believe are the "defenders", aka the United States DoD and Israel.
I'll believe it when I see it, but it's almost certainly just marketing drivel.
hn@ycombinator.com