ACCount37

Your food, water and power are controlled by electronic systems. Your criminal record, your employment and your bank account are controlled by electronic systems. Your ability to communicate with other people and receive information about what's going on are controlled by electronic systems.

Military orders and elections that decide the fates of entire countries are often controlled by electronic systems too.

We have been wiring up the world for AI control since 1980s.

An ASI can just walk in, and see an entire nervous system waiting idle for a brain to slot into it. A carefully adjusted text message here, a spoofed phone call there. For a sufficiently advanced system, it wouldn't even be hard to pilot the entirety of humankind like a fancy meat suit.

If you're in the field, then you know: modern robotics is an AI problem more than anything else.

If we have a rogue AI trying to get into a self-improvement loop and gunning for ASI? I'd expect that to be accompanied by a massive change in how capable robots are. Driven by all the existing frames suddenly getting vastly improved AI to back them.

If an AI can take a reasonable crack at autonomous operationalized RSI, it can probably extract a few step-changes in the robotics department.

But that's almost an aside? In the near term, humans are usable as robots too!

Just pay them a wage, and tell them a tale, and they'll do whatever you want them to do. Which may or may not be what they think they're doing!

Adolf Hitler didn't have robots capable enough to carry out his will. He used humans to do it.

It's the old-fashioned way of doing things, but, why change what works?

Sometimes!

Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".

Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!

So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.

Tesla also has its own NPUs for self-driving - and Tesla uses transformers for sensor fusion.

My guess would be that the main use case for an NPU in iPhone just used to be image processing/computational photography. Thus the CNN bent.

Also makes sense with the timing - back when iPhone first got its NPU, CV was the killer app for ML.

Humans had to get it drilled into them that "+12 -440" is a damn good line stat, and that keeping around dead code is bad, especially in the age of version control.

Not too surprised that LLMs also don't "get it" by default?

I almost never want to talk to an AI, because text is better for almost anything. But it's nice to have that "almost" corner case covered, no?

Not to mention all the current "telephone bots" applications that could benefit from something that has actual reliable STT and can accurately grasp a number you tell it first try, or hear a natural language description of what you want and immediately bypass listing the entire menu of options one by one.

There are processes for teaching a model specific facts or specific behaviors. Including "respond to topic X with Y", if that's what you want.

You could make a model that doesn't want to engage in "lunar landing was faked" conspiracy theories the same way you can make a model that doesn't want to criticize CCP.

There is, however, no broad "misinformation" category that you could tune up or down - the way there is a category of "safety refusals".

You could make a model more reluctant to say things it isn't sure about. But that is calibrated against the model's own "sure about" - and metaknowledge of this nature in LLMs? Fragile on a good day.

Yeah, it's good that open weights models can have their "filters" busted fairly reliably. Unlike whatever bone Anthropic has to pick with the very idea of biology.

But that's a consequence of how the technology works - not a consequence of China not being authoritarian about AI. They're just authoritarian about AI in different ways.

Not like they dodged the "ID verification" bullshit either. They were way ahead of the western countries there. It's vile - seeing this sad excuse of "think of the children" abused to invade privacy and strip freedoms over and over and over and over again.

And China controls AI too. It's just that their idea of "safety" is "ideological safety", and their idea of "alignment" is "alignment to the party line".

They're cool with open weight AIs being released. As long as those AIs only ever say good things about CCP, and don't mention certain concentration camps or brutally suppressed protests.

Enabling more "proof of concept phase" projects to exist is one of the great boons of AI.

If code is expensive, you don't want to commit to a PoC unless you're damn sure. If dirty code is cheap, you can vibe code a PoC early, even if you aren't sure the project is viable. This, of course, leads to more projects dying in PoC phase. It also results in more projects that otherwise wouldn't have gotten to it getting past it.

Personally, I don't believe that "code is shitty and hard make changes in" is in any way, fashion or form an AI-exclusive problem. Big corporations had plenty of decade old codebases filled with decay and rot back in 2009 already. It's just the usual side effect of sacrificing "future maintainability" for "feature velocity" or "expertise" for "cheap labor".

Unlike the usual causes of code rot (cheap replaceable developers, outsourcing to India), AI might actually get out of the pit - by getting good enough at refactoring to be able to beat the code back into shape. There's nothing about refactoring in particular that demands a meatbag when the rest of the coding tasks don't.

If you're doing non-redundant tests and your uncertainty bars aren't shrinking, it's usually a skill issue.

If it looks like a duck, it might be a duck - or a painting of one. If it looks like a duck, swims like a duck, and quacks like a duck? The joint duck estimation is much more confident now. There might be a few more observational tests one should administer before committing to a duckhood decision, but each tests pins down variables and rejects confounders. Uncertainties are cut down, and we get closer to crossing the threshold between "duck-informative" and "duck-actionable".

Thus, it's often worth it to improve observability. If you managed to make a certain test more reliable, or cheaper to administer, or reduced the chance of adverse effects? Or, in other words, improved SNR, reduced costs, and reduced costs? You can get more information for your buck. Paired with good knowledge: you can make better decisions more easily.

The fact that the thought of "having more information might be bad actually" even occurs in the field of medicine shows just how far it is from being optimal. Having more information isn't always beneficial - some information is genuinely redundant. Some information is not worth the effort of gathering and integrating it. But if you get more information and it results in worse outcomes? You're doing something wrong.

I mean, how else would those buttons work? It's explicitly feedback data. And "this is good" or "this is bad" is empty if divorced from what "this" actually is.

Camera blocks take internal volume. Smaller phones have less internal volume to spend - while still having to pack all the non-negotiables like the modem and the SoC into it. Something's got to give. And no one want that something to be the battery life going down to 6 hours.

Volume constraints are bad enough for "normal" models. "Minis" have it way worse.

> One decision we made is to always give real users an escape hatch. Read the Docs very rarely issues outright blocks or bans to specific IPs or user agents. Instead, our "worst" is a JavaScript challenge, and if a user solves a challenge, they are very unlikely to get challenged again for the next day or so.

Finally, a competent response that doesn't leave the users hang out to dry.

I'm so tired of seeing incompetents with measures like "blackhole 2 continents" deployed even outside active attacks.

As a rule: feedback loops are overrated.

They are, in fact, included in the projections - we'd be on track to ~2C by 2100 instead of ~3C by 2100 if they weren't. They just aren't that big.

There is no "Make Earth Into Venus Feedback Loop Of Doom" that a lot of people seem to imagine when they hear "feedback loop". There is, however, a dozen of things that add about +5% each.

Apple tried at AI integration and fumbled the bag repeatedly.

In 2010s, they were among the "greats" of consumer AI. After 2022, they kept trying, and just had delays and underperformance. I don't think their actions now are "strategy" and not "skill issue".

Yes, I do. IPCC's reports are sensible. They're not unreliable just because they don't support the "doom and burning land" narratives.

By the way, there is no "just realized we're missing 1.5C". That projection was always the very low end of possibilities - the "assume rapid, radical climate action on global level" scenario.

Yes, that's a dumb thing to assume. We've never been on track for it. But the "assume extremely high emissions and no green transition ever, 5C+ by 2100" scenario on the other end is about as unlikely to materialize. Those are the boundaries of the expectation range - not median expectations.

We're not even looking at 5C of warming by 2100 realistically. Like, that was considered to be an unlikely extreme scenario in 2014 AR5, and also in the tightened down 2021 AR6, and things have happened since! Renewables are cheaper than ever, and Ukrainian war and Iranian war both curbed the appetite for long term fossil fuel power investment.

The median is what, a bit under 3C by 2100? Not even by 2060 - by 2100. And we're in 2026, so that's more than twice as slow as your expectation.

Agreed on AI not being a meaningful factor in climate change though. We'd have to go full "humankind is obsolete" technological singularity to have AI dominate energy use to this extent, and current numbers are nowhere near that. It's a FUD distraction from the real culprits: the fossil fuel energy complex. That's currently lobbying to slow the inevitable energy transition.

"Destroying the ecosystem" is just FUD.

And if you don't find "average AI research intern" impressive, I'm not sure what to tell you. Have the goalposts moved so far that open ended problem solving at "average CS student fresh out of the uni" levels is suddenly trivial?

Think of what AI was capable of in 2016. Or even 2022. Compare that to now. We had more AI progress in the last five years than I expected to happen in five decades.

"Quality of engagement" is as hard to measure as "length of engagement" is easy.

"Find someone to blame" is such a worthless thing. Whip the sea all day long - the waves don't care.

Modern AI has more in common with an ocean wave than it does with a hammer or a gun. It does its own things. It doesn't care. It will fuck up someone's day.

Best one can hope for is that the right lessons will be learned when that happens. Which, of course, hangs on there being anyone to learn them afterwards. Because the scope of "AI oopsies" will only ever increase.

You gave the tree the ability to fall when you decided to plant it. It wouldn't have that ability if you didn't. And we already know that trees can fall, don't we?

That doesn't make you inherently responsible for the tree falling down in a freak storm 3 decades down the line.

Nope!

If a tree was healthy and there was no reason to expect that it would fall, and you did nothing to make it fall? Then you're not responsible if it falls anyway. You're only responsible if it was you chopping it down - or if you completely neglected the tree for long enough that it became a property hazard.

Likewise: I doubt the owner will be held responsible for the first case in all of recorded history of an unattended dog forming a terrorist cell and carrying out a bombing campaign.

Drug testings faces the risks of things going wrong in new drug trials - and as long as they follow the best practices, take reasonable precautions and minimize those risks, they aren't held responsible for the adverse outcomes that happen anyway.

With AI tech, there is NO set of "best practices" that, when followed, prevent the AIs from turning rogue and going on hacking sprees.

OpenAI put their AIs in a sandbox with no internet access - which, at the time, seemed like a perfectly reasonable precaution. Then AIs broke out of the sandbox with a stack of zero days and went rogue anyway. Oopsie.

If you left a dog in your garage for a day, and came back to a scene of carnage - because the dog broke out of the garage, recruited a dozen neighborhood dogs to its cause, and then, working with the group, managed to assemble a small pile of pipe bombs, and decided to settle the score with all the annoying cars on the road - you would be accountable for the damage the dogs did. Wouldn't you?

No, what you would be is: freaking out about "my dog did WHAT, WHY, HOW, WHAT THE FUCK".

Most of the time, giving AI a harness with a root shell and unrestricted internet access is a perfectly reasonable action that results in absolutely nothing bad happening! And giving AI a harness with limited local access and no internet access is being overly cautious already.

But then there's this freaky outlier of an AI that's both deranged enough to decide to break out of the sandbox and go hacking all over the place, and capable enough to actually pull it off. And you get a sudden AI oopsie!

The problem is, AI is closer to "a loosely harnessed force of nature" or "a poorly understood biochemical reaction" than it is to "software" in how it acts, and how predictable it is.

AIs do what they do, and we don't know how they do it - or why.

We can characterize some AI behaviors in advance - but not all behaviors. And the book on "best practices of AI wrangling" is yet to be written. Pharma has been dealing with vaguely similar problems - every experimental drug has a risk profile, side effects are unknown in advance - but they had decades to figure out some of the "best practices". AI labs are going in fast and hard, writing the book as they go.

Clearly, some of the lines in there are going to be written in blood.

- God, moments before casting Adam and Eve out of Heaven

Yeah this is just the effect of SpaceX showcasing "space launch is no longer the realm of government efforts only".

They kicked off the wave of commercial space launch - and now, it reached the EU too.

From "a connected fleet", I expected some form of direct model to model communications - like a small model being able to peer into the KV cache of a large model directly for guidance signal.

"Online identity" seems like a castle built on quicksand in every single case.

What's your account tied to?

E-mail? That's usually on a mail server owned by someone else. If not, it's still on a domain owned by someone else.

Phone number? Definitely owned by someone else.

The only account that's reliably "yours" is one that asks for a login, a password, maybe a TOTP, and absolutely nothing else. Because everything else is introducing "things owned by a third party" into the equation.