ajross

> 2. Making critical systems available on internet to leverage integration with LLMs [...] which were otherwise in DMZ (demilitarized zone)

Honestly, I think this is actually not nearly paranoid enough. Phrased the way you do, it sounds like it's just a matter of setting boundaries in the right places and identifying "critical systems". But that's way, way harder than you'd think.

Here's my For Dummies reasoning behind the AI apocalypse:

1. AI is now at parity with median human reasoning capability and can use people's computing devices as well as the people can.

2. People commonly let AI operate their computers, and can be easily fooled into doing so in any case.

3. Society runs on computing devices operated by people.

4. There is no step four.

Basically any world where there is common access to AI agents (or whatever they end up being called) is one those agents can pretty trivially hijack.

If there is a protection regime that can prevent this, it's not about where the AI runs or what the boundary of its DMZ is.

> It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.

Well, in a narrow sense of "wiped out" (c.f. Terminator/SkyNet), sure.

But the deeper worry is better expressed this way: AI is now starting to accomplish things that defy explanation, or prediction. We don't know if Alignment is even a solvable problem as we thought we understood it.

So basically, yes: "humanity" is probably not at risk of extinction per se in a biological sense. Human culture, civilization? Who the fuck knows any more.

Chip assets are a vanishingly small fraction of the valuation of these companies. If they were to depreciate instantly and need to be replaced at cost every year (assuming the fabs had capacity to produce them) it wouldn't move the needle.

This is like asking if there's a historical basis for understanding how the depreciation of garden sprinklers affects home value appreciation.

Not sure I buy that? There's a hardware/process argument that DRAM scaling is hitting a wall (for largely the same reason that logic scaling is, even though the fabs are different).

But DRAM has always been almost completely dominated by consumer device needs, and consumer requirements leveled off hard maybe a decade ago. Once you had a 4G system, things were mostly OK. Pervasive VM use and general bloat pushed that up by maybe a factor of two or three since, but that's it.

AI is different. It wants that capacity for different reasons and puts it in different places. And almost none of us really saw that coming more than ~18 months ago or so. Certainly not to the extend that you'd conclude "we've been ignoring it".

> Anybody who's ever tried to get through their review process can attest that the article writer's experience is far from outside the norm.

This is an argumentum ad populum fallacy. "Everybody Knows" this kind of thing is true, ergo you should believe the example here without evidence.

It's also kinda ridiculous in context. I mean, no, there are no cases in the literature where Google has rejected an app due to misidentification with NFSW content and then sent screenshots of said content as part of the review. Even not in combination I don't think. Like, has either situation every been documented on its own? "Far from outside the norm" indeed.

Seriously, do kids not use scare quotes anymore or know how they work? There are no less than three replies here interpreting what I said entirely wrong.

I said it's a "hit piece", not a hit piece. And I gave a definition for what I meant in the very next sentences. Because, obviously, the traditional definition is one that is going to be subjective and silly and not worth discussing. But the idea of documentary filmaking having opinion and perspective is important, nonetheless.

Discourse about this man is cursed.

That bit was wrong. He was wrong about lots of other stuff too.

But Tesla's product design and market fit was right on the whole. SpaceX was almost all right and not wrong. Starlink was almost 100% right. PayPal was a lot more right than wrong. Obviously you can nitpick about anything, but when you make that many great bets on things that other people tried and failed at, I think you need to at least nod that maybe his instincts were... good. Which is what I said.

Since you like definitions: https://en.wikipedia.org/wiki/Scare_quotes

That's a purely semantic argument. Was Oppenheimer a hit piece? Lots of people think so!

"Biased" is a weasel word when used in arguments like this. I really doubt anything in this film is actually materially false.

I... just don't understand your confusion? Of course it's a "hit piece". There's a market for hit pieces because there are people willing to pay to see content aimed at their favorite targets. And the trailer starts with Musk sniping to substantiate what it's about and clarify who its for.

Maybe it's that you think "hit piece" means "illigitimate?". Documentary cinema is absolutely allowed to have perspective. Musk's own communication is hardly unbiased or coolly considered, and he's been known to engage in some "creative advocacy" of his own. Would you apply the same logic there?

My own opinion is that he's a complicated guy and a seemingly textbook bipolar case with a lot of good technical instincts and a really destructive egomaniacal streak. He could do with some hit pieces, honestly.

> And all those companies can choose not to work with them too.

Right, and if they chose independently to do so it would be their right. What makes this problematic is that "all those companies" did not choose to not to work with the Pentagon (or with Anthropic spefically, for that matter). They were compelled to using using a regulatory tool which was originally intended to combat hostile foreign espionage.

That's not a "choice" made by those third party suppliers. The Pentagon is using the Supply Chain Risk designation as a contract negotiating tactic.

It "literally" did not. What enforcement exists is financial (a tax penalty) and not compulsory. And obviously most people are covered via employer plans or Medicare, as has always been the case. A quick Google says that 19.2M Americans are enrolled in ACA plans. I don't have statistics, but I'd bet that the overwhelming majority of those are willing purchasers. Most people view health care as desirable.

This is why this kind of stuff is so difficult to argue about. G-S was repealed primarily by the Gramm-Leach-Bliley act in 1999. https://en.wikipedia.org/wiki/Gramm%E2%80%93Leach%E2%80%93Bl...

And it is true, that it was signed by Clinton. But it was otherwise an entirely partisan bill passed with exclusively republican votes. All 44 democrats voted nay.

Now, given that, do you really feel that that's a supporting argument to the upthread contention that Democrats are a corporate-dominated party and not interested in regulation?

> How’s ACA hard-hitting on corporations?

"You must sell insurance to sick people!", "I don't wanna!", "I don't care, the law now says you must."

I mean, the whole insurance industry had spent the better part of a century (basically its whole existence) denying coverage to people with pre-existing conditions, and the ACA fixed it via fiat.

Not only is it clearly "hard-hitting", I'd argue it's probably the single most "hard-hitting" invasive change to corporate regulation of the post-war era. Certainly it is in terms of the amount of money involved in the affected market.

> how’s this anti-corporate?

And this is where the rub is. You're not saying that your desired regulatory structure is something different than existing proposals, you're demanding that your government engage in behavior that is expressly "anti" whatever you perceive "corporate" interests to be.

That's not democracy, that's just warfare. You have enemies and you want to win. And the problem there is that the structure of our government requires compromise.

> understandable that the government doesn't want to commit to Anthropic controlled software

But that's not at issue here. Obviously everyone agrees that the Pentagon can issue RFQ's to whatever specifications it wants. They can buy whatever they want, for the same reason that you or I are free to sell whatever we want.

The subject at hand is an attempt to use national security regulatory power to coerce Anthropic into selling a product it doesn't want to sell.

> Democrats are primarily a corporate dominated party. They haven't really had any hard hitting legislation nor legal regulation/rulings on corporations in easily the last 30 years.

Sarbanes-Oxley? Dodd-Frank? ACA? Now I'm sure you (like, every other living human) has a ton of nitpicks and criticism of bills like that. But they're real and they passed and they're largely-to-wholely partisan creations of the Democratic policy apparatus (SOX looked very bipartisan and was passed under Bush, but the actual bill was written mostly by democratic staffers).

I mean, fine, you'd like different regulation. But they're responsible for basically all the corporate regulation active in the US regime.

Bothsidesism is a very poor tool for this particular argument. Work with your allies, please.

Did you have to liquidate your house to pay the property tax on it?

Edit to point out a subtlety: this is an argument at cross purposes. If the economic impact of all this personal wealth growth is so high that liquidating it would move whole markets, then very clearly it represents significant missed tax revenue.

> There is nothing to tax until they sell some shares.

That's tautological. I mean, it's true under current federal tax law. It's obviously not true under new California law, which is what the article is about.

Clearly the government can tax non-cash assets, and they do all the time. People act like "wealth taxes" are some moral horror or logical impossibility, while tossing their mortgage statement into a big file and pretending to ignore the property tax line on the escrow account.

Are there practical problems like "wealth has feet"? Sure. Taxation is hard and all systems can be gamed. But let's not pretend that there's a greater principle at work here.

> Willing to bet it was something as stupid as the data being accessible by changing the query parameter,

Um... why? OpenAI agents have literally been caught coordinating with each other to effect successful multi-stage attacks on sites using novel zero-day vulnerabilities.

While, sure, it's possible this is just a goof on the part of the victim, that you would be inclined to give the benefit of the doubt to the LLM seems... weird.

Moore's law is mostly dead these days. Process scaling still happens, but nowhere near at the rate predicted, and there are tradeoffs everywhere now. The Zen 5 core is in fact bigger in terms of die size too.

> Sorry I'm not sure what point you're trying to make.

Just that "Apple customers use old MacOS devices" is in fact of a piece with "Let's monetize our users" (by stuffing ads into the Settings app in this case). They can't rely on you for upgrade revenue and need to squeeze you in other ways.

No free lunch, basically, unless your software vendor really cooks for free.

> I am still rocking my M1 MacBook Air and performance-wise I have no complaints.

Ah, but see, I'm rocking a vanilla Ubuntu install on a vanilla[1] Raptor Lake Dell I got used on eBay last summer for $220, and I too have no complaints.

"No complaints" is a great way to celebrate worthwhile hardware's value in the age of consumerism. It's death to a business who needs to sell stuff to survive. That M1 Air isn't booking revenue, and to be blunt Apple would have preferred its lifespan to be lower.

[1] Still has 16G though for VM and docker tinkering, something that is hard to get on Apple hardware without springing for a MBP.

> To each their own but I’d rather not deal with XYZ and just get the thing that works.

Yes, but the point of the linked article is that this preference inevitably results in your device spying on you. To each their own indeed.

Edit: and of course I see your P.S. note. But that isn't really responsive. Most Linux users would point out (P.S.) that they'd like their hardware to be faster and lower power or their OS to be a little cleaner. But alas, you can't have everything.

> Because Mac laptops are lightyears ahead of [hardware that runs Linux]

There was a discussion on this forum just a few days ago wondering why Apple is so hostile to open source support on its devices. And consensus here was that, no, it clearly wasn't due to competetive pressure.

But clearly it is. Because to many consumers, if you have to run MacOS to get an M6, you'll run MacOS instead of Linux. And that makes enshittifying MacOS as a revenue center very tempting.

> Also the OS mostly “just works”

Article is literally about the OS not working as desired, though.

There is zero chance that an LLM approximating a modern frontier model is going to be running on a phone in the next decade. Even if you grant that you could stack enough DRAM dies on top of each other in the package, that would be a three order of magnitude improvement in power efficiency just for the compute.

I don't see how that math works? This is a $15k rig under benchmark and per the results it competes very acceptably against... one consumer GPU.

I really don't see who buys this, except people who want the Studio for some other reason. But nothing in the story says you want to fill racks with these instead of Blackwell or TPU parts; it's not even close.

The cooling pause thing was just a dig at Geekbench, not specific to this discussion (thus putting it in a footnote). They tuned (the last version of their) benchmark specifically to give better numbers on hardware that tends to throttle under performance load.

Now... you can make a reasonable case that this matches real world interactive load better. But it also happens to have the practical effect of making Apple's numbers better, and no one else's. The new benchmarks are measuring subtly different things, and the new thing they measure happens to be what Apple wants to sell. At best, that's backwards.

> Apple chips don't have AVX-512.

Zen 4 isn't 512 bits wide though, it's a split cycle 256 bit wide SIMD engine otherwise very similar (except in register size) to M6's.

The answer is more that Geekbench is at this point[1] heavily tuned to exactly the code Apple silicon does well: implicitly parallel wide-issue scalar code that you typically get out of modern compilers and JIT engines when throwing mostly-unoptimized "regular source code" at them. Apple has an enormous amount of instruction issue parallelism compared with x86.

The grandparent is looking at transcoding tasks where the limit isn't instruction issue but actual compute hardware on the core. And Apple doesn't actually win by much there.

It's just hard to know what to measure. Geekbench tends to be a metric for "feels fast doing boring interactive user stuff", which probably matches Apple's marketing imperatives well.

[1] Really they keep moving harder in that direction with every release. The "cooling pauses" in v6 likewise seemed very much like an attempt to boost the score on fanless Apple devices. If one were the type to allege a dark conspiracy, this is a tempting spot.

I'm talking specifically about discourse like Jacob Coxon's tweet thread and later communication. Those are the folks[1] you want to be listening to, not the nuts with the blogs, no matter how much a particular conspiracy confirms your priors.

Salvation will come in a whitepaper, not a screed.

[1] And yes, that includes Amodei and Altman.

> The framing of how we think about AI

I don't think I agree. In fact I think most of the too-online pontification from P(Doom)-ers and EA and the Rationalists and the like is, if anything, sorta missing the mark more than it hits.

They got the One Big Thing right, which is that AI progress was likely to race way ahead of consensus predictions. And props to them for that. They had vision we didn't.

But their actual advice and specific fears remain, as then, at least as much paranoia as insight. They were right, but they aren't helping any more than any other fringe group.

We need to trust the actual scientists and engineers to save us here, not the bloggers. Which when phrased that way seems hopeful and not doomerist.