rakejake

Plausible deniability - The line of attack is in their sessions/prompts data. Just make the prompt pointed enough that the search space is tractable and use your ginormous compute.

> "Of course we don’t know whether that is true"

Yep. Who is verifying these claims? We all know how trustworthy Altman & Co are.

Yeah, I think you can't just throw money randomly at problems and expect results unless you know a line of attack that can get you all the way. OpenAI chose the line of attack only after it became known to them via rumors. They "front-ran" the researchers.

Exactly! This is the real Occam's Razor explanation.

Are you saying there is no search space intractable to LLMs? That wouldn't be possible. AIs are statistical pattern-matchers on steroids. The prompt is key to getting anything useful out of them. They are incredibly useful and major game changers but ultimately that does not alter this fact. People (including OAI) have already tried to solve Millenium Problems with it. That OAI woke up last week and suddenly decided that throwing their researchers armed with millions of compute on one particular idea to a problem is highly suspicious in itself.

Even if OAI had zero data from Buckmaster's sessions, this is in very poor taste and highly unethical. You are front running a researcher just to be able to say you did it first? Tao is right - OAI is treating math results like oil. This is the like Exxon getting a whiff of a massive oil field and racing to the punch by deploying their full crew.

OAI started working on this only after they found out it was close to being solved. They threw a team of researchers who spent sleepless nights + a ton of compute. This is not exactly healthy academic competition - it's like if you spend a year hunting for oil fields and finally find a very promising area to be explored, only to find that Exxon tapped their entire exploration unit to go all in and and find it overnight just to stake claim to the discovery. Tao said it right - math should not be treated as a non-renewable resource to be mined.

OAI doesn't need to mention Buckmaster's name directly in a prompt. They just need to select a basket of sessions that is guaranteed to contain Buckmaster's and then direct the LLM to attack only a specific method/angle. This is trivial to do while maintaining plausible deniability about not using his work.

I'd think nothing is "safe". Anything you say can and will be used by the LLM if it has enough statistical similarity to the prompt. Call it "Ma Random Rights"

Research equivalent of front-running.

I don't think OAI should be given the benefit of doubt. They are doing the research equivalent of front-running. Knowing where to look is one of the main challenges in research. Tristan's argument from his essay was that it is hard to brute force with a vanilla prompt (even for seasoned mathematicians) unless you knew very specifically what to mention i.e the search space would have been intractable even for OAI's compute budget.

"deidentified data" isn't much to go by. Say I prompted the internal model this way - "Hey there's a solution to a unsolved problem X. The solution uses a less known Method Y so don't bother wasting time with the usual methods. Take papers A, B and C as references. Oh btw, here's the last year's worth of data of all prompt sessions that mention this problem. Pay special attention to the ones that mention Method Y and sub-keywords Z,W".

This is obviously all speculation but the timing is very suspect. If OAI actually did this (and I suspect whatever they did is pretty much close to this), I think it is highly unethical.

Try to increase the complexity of the work you're doing. It's not that AIs don't makes mistakes but they are able to pattern match to larger and larger pieces of the problem as more people use it and more iterations of the model are released. So your challenge is now in steering it in a way that these errors are minimised while reducing bloat.

Try to think of yourself as a professor who's trying to come up with a problem statement worth solving. The AI is your "lab" that will help you run experiments.

Yes, that does track with my personal experience. More context, more params and no quantization is probably it. But my hunch is that all the training data they've been getting in the past year also plays a part here. More than any other lab, anthropic's focus on coding right from the beginning gives them access to the best training data (several githubs worth). Most of this code comes with human feedback and anthropic even has data on how many went to production, got reverted etc. No need to pay for human labeling when your customers are doing it for you. This is their secret sauce.

>>> the model is (obviously) good at security

Out of curiosity, are you one of the people who has access to the model? If yes, could you write about your experimental setup in more detail?

Sure, I am not precluding the possibility that they've trained a genuinely great model. All I am saying is that the "this model better than that model" is moot when on one side you have model weights, and on the other side a whitepaper and some accompanying comments on the danger.

I'm not that old but have been here long enough that I remember when GPT-3 was considered too dangerous to release. Now you have models 10x as good, 1/10th the size and run on 8GB VRAM.

>> Test it yourself, GPT 120B OSS is cheap and available. BTW, this is why with this bug, the stronger the model you pick (but not enough to discover the true bug), the less likely it is it will claim there is a bug.

I guess this is the crux of the debate. All the claims are comparing models that are available freely with a model that is available only to limited customers (Mythos). The problem here is with the phrase "better model". Better how? Is it trained specifically on cybersecurity? Is it simply a large model with a higher token/thinking budget? Is it a better harness/scaffold? Is it simply a better prompt?

I don't doubt that some models are stronger that other models (a Gemini Pro or a Claude Opus has more parameters, higher context sizes and probably trained for longer and on more data than their smaller counterparts (Flash and Sonnet respectively).

Unless we know the exact experimental setup (which in this case is impossible because Mythos is completely closed off and not even accessible via API), all of this is hand wavy. Anthropic is definitely not going to reveal their setup because whether or not there is any secret sauce, there is more value to letting people's imaginations fly and the marketing machine work. Anthropic must be jumping with joy at all the free publicity they are getting.

Maybe they did use small models but you couldn't make the front page of HN with something like this until Anthropic made a big fuss out of it. Or perhaps it is just a question of compute. Not everyone has 20k$ or the GPU arsenal to task models to find vulnerabilities which may/may not be correct?

Unless Anthropic makes it known exactly what model + harness/scaffolding + prompt + other engineering they did, these comparisons are pointless. Given the AI labs' general rate of doomsday predictions, who really knows?

The word "profound" is a bit overused when it comes to movies. I agree that The Battle of Algiers is an excellent film, one of the best ever made even. One Battle After Another is also excellent but it is not really political in the way the TBoA is. It uses a political setting very effectively in a chase thriller. A movie like The Parallax View is a better comparison. That movie used the post-60s paranoia very effectively in a great suspense thriller.

Yeah, I'm watching a lot of Charlie Chaplin movies in preparation for my new role as a tramp.

I agree on principle. But there is going to be a painful transition where people are still reckoning with the new capabilities so I understand where all the fear/sadness is coming from.

Tbf I think the golden days of being a software dev are over even if the AI were to stagnate and never improve. The spectre of AGI is enough for higher ups to demand more output which will in turn require more hours to be put in by devs. A project that required 2 months will now be allotted 3 weeks because "Agentic coding increases productivity".

> I don't know if the market will have fully internalized that knowledge soon enough

This exactly. I am neither a boomer nor a doomer. It has helped a lot both at work and in accelerating my personal projects. But now that the C-suite and middle management has jumped on the agentic bandwagon, I'm unsure where this will go and what casualties ensue. At the very least, in the short term there's going to be a lot of "Now that we have agents, this project should be achievable in half the time".

This started decently enough and then the author went all over the place. I'm not sure why detailed explanations of neural nets and smart contracts were needed here. It really feels like trying to ram in a tech solution for what is effectively a market/social problem.

Using computers to aid in designing is not specific to Kanchipuram saris. While I realize people always approach it from the POV of saving a dying art, I'm unsure if K.saris can really fall under that umbrella. Clearly the demand is there and the issues here arise due to inefficient and possibly corrupt market practices rather than the art itself dying. A lot of space was used to explain the lopsided economics on the supply side but there's not enough attention paid to the demand side and the marketplace dynamics.

Excellent comment that really gets to the crux of the matter. Countries like China and India see themselves as civilizational, America sees itself as a perfect marketplace - it exists to feed its customers's wants and whims as efficiently as possible. I don't necessarily mean this in a demeaning way, it is what it is. In some sense, America is a state-level example of hedonic adaptation with its positives being improvements in quality of life and development of new tech, negatives being a bully in world politics, endless wars and bloodshed.

In general, hedonic adaption ends either with internal retrospection (shifting from pleasure to purpose) or an external disruption. In America's case, the former is extremely unlikely IMHO - the American people will not put their money where their mouth is because they enjoy the wealth generated this way. It will be upto external disruptors to check on Uncle Sam's endless thirst.

Rather than think of it as a pivot to hardware, I looked at it as MS trying to corner their share in the consumer market. Mobile and Social were the hot things back then and mobile threatened MS's dominance of the OS market. MS ultimately failed but they still owned the enterprise market and continue to keep their lead in desktop market share.

Ballmer doesn't strike me as an idiot and definitely not bland. He's one of the more colorful tech personalities. MS's almost unassailable lead in enterprise could be attributed to him and the pivot to cloud could not have happened without this. But he definitely fumbled hard on mobile (Windows Phone), Surface (IIRC the initial ARM laptop was a major flop and had a close to 1B+ writeoff) and the disaster that was the Nokia acquisition. I'd say he left at the right time, just as it was becoming clear that MS's bets on Windows Phone and hardware in general weren't paying off.

My point was more that MS hasn't had an industry changing product in a while. Google became joint-SOTA in AI and seems poised to take the crown with the next Gemini, and also in self-driving cars and quantum computing. They've kept their cash cows going while also being up to date on the tech that might upend their business model, so in a way they've cracked the innovator's dilemma which is definitely not an easy thing to do. A lot of HNers even wrote them off after ChatGPT and the disastrous Bard. Apple has a successful mass product in Airpods, a moonshot in Vision Pro and the insane Apple Silicon which they executed over more than a decade.

Nadella did well in the last decade to consolidate the MS stack (Teams, Azure, Office) and to invest in OpenAI when he realized MS's internal efforts wouldn't yield the expected output. He has protected their turf and made some strategic acquisitions like Linkedin and Github to keep their lead in enterprise software. From the POV of Wall Street performance and stock returns, he is a definitely a great CEO but so are Cook, Pichai even Ellison.

I often wonder why Satya Nadella is so venerated on HN compared to say, Cook or Pichai. As innovators, MS lags way behind both Google and Apple. I can't think of one bleeding edge product released during Satya's tenure. Say what you will about Apple and Google, they still consistently put out products that make you sit up and pay attention. What has MS been doing other than squeezing the MS Office and Azure cash cows?

I guess Atlas is a good name for a web browser. But I'm surprised their first release is Mac only. Does it indicate they are targeting some kind of power user (programmers, creatives etc) or is it just the first platform they could ship by the deadline?

Will they be able to take any significant marketshare from Chrome? I suppose only time will tell but it will be a pretty hard slog especially since Chrome is pretty much synonymous with "browser" in most of the world. Still, I don't think anyone at Google is breathing easy.

Unless the filmmaker has a track record of misrepresentation or negative representation, I tend to give them the benefit of the doubt.

In any case, if the movie irked you that much, I don't think there's anything I can say to change that. Peace out.

Well there are no "good" characters in this film so how would one add a positive portrayal of blacks? Maybe "Pat" could have also been black but he's an ex-bomber turned paranoid junkie.

I think the character traits were what they were because the story doesn't work otherwise. I don't think it was PTA's express intention to showcase negative black stereotypes.

The review does not mention Deandra (played by Regina Hall) at all, among other black characters who weren't negative representations per se. Deandra is very prominent in the second act of the movie and her responsibility and dedication to the mission is quite apparent.

Reviews are very hit or miss nowadays (mostly miss for me), but the verdict for One Battle After Another is absolutely correct.

For those of you who are on the fence wrt watching this movie, the politics and the revolutionaries simply form the backdrop for the story. The movie is ultimately a chase-thriller and the cinematic pleasure on screen is just incredible. If you are a fan of superbly shot and staged set-pieces, this movie is for you.