danpalmer

Was this a "build better sandboxing" and "don't tell people to eat glue" safety leader, or a Roko's Basilisk believing safety leader?

A lot of the "AI safety" types are very focused on the latter and not at all concerned with the former. We need both, but we clearly need a much stronger focus on the problems we are seeing now, and much less on the hypothetical problems we might see in the future.

The "Flash" branding is doing a disservice here, it makes it sound like the cheap and bad model, but 3.8 Flash is really quite good, especially for it's price.

Flash-Lite is closer to what Flash originally was, both in terms of the pricing and the intelligence level compared to other models in the same generation.

This proves my point. This is a hobby-build, because it wouldn't be prioritised for most businesses (including Apple, clearly), and it was launched in Dec 2025 so was almost certainly significantly built by LLMs.

I'm not saying that these tools didn't exist (there are better examples than this one), but hobbyist, or open-source tools are examples of the prioritisation not mattering. Most of the little projects I've built outside of work have been objectively terrible uses of time for the result.

This is exactly the sort of software that was hard to prioritise before LLMs, and trivial to build with LLMs.

There's nothing interesting going on here, and I don't mean that as a criticism. There's a clear schema, no particularly clever UX needed, you can build almost all of this out of standard UI components, and the problem is well defined.

The only challenge is the actual time input for coding, and that's mostly gone today.

This experience is with Antigravity both internally and externally, and I have done quite a few side-by-side comparisons with the same prompt across a number of different Google and non-Google models.

I've tried Codex as a harness too, and that was nice. I don't find a significant difference between Antigravity and Codex. Codex has more features but I don't use them.

3.8 Flash is my daily driver and produces pretty excellent results all round.

Expected value usually assumes these things happen in isolation, and they don't. They are good at representing the isolated upside, but rarely do they account for the downside.

In this case a 1% chance of $1bn represents an expected value of $10m. If you accept the cost of litigation as $10m (for example), then your expected value is actually zero. And if you think about the outcomes of the 99% of cases, bankruptcy is hugely painful.

One can always play silly games with expected value. If the "value" of a human life is $10m (supposedly a figure used by some governments), you could pose all sorts of expected value scenarios, but when it's your life that all goes out of the window.

Fair point, it does seem to be changing recently. It seems less religious and more practical than at Anthropic.

I hope you're not serious in drawing an equivalency between a mainstream religious belief and that level of extremism.

My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.

Famously Anthropic exists because OpenAI didn't believe deeply enough in that risk.

As much as I cannot disagree more with the author's fundamental premise (religion), it is refreshing to see someone claim to have a belief and then follow through, change their behaviour to match their belief, at no small cost to themselves, and in pursuit of doing the right thing.

The religion part or the AI criticism part?

This explicitly documents that it modifies the image. Passport authorities typically explicitly tell you not to do that. They modify the images in a standardised way.

> And no I'm not paying $25 for some idiot at CVS

Neither am I. Don't do this. Just send the passport authority your image as they tell you to do.

I can't stress how bad an idea it is to use this for passport photos.

Passports are likely the strictest, most important ID that most people will ever own. The reason the photos have strict requirements is to ensure that the ID can be effective. Putting a photo of not you on your passport is such a huge self-own.

It's also not just about whether your passport authority accepts the photo, it's about whether every single border you cross accepts it, and crossing a border in a country you aren't a citizen of is about the worst place to have your ID questioned.

I don't mind taking a hard line on safety (this is even good!), and I don't mind controlled testing without model safety to experiment with safety systems or harden things.

The bit I don't like is taking a hard moral stance on what you are allowed to do with the models, while simultaneously taking the guardrails off themselves and then marketing the results of that. "Look how good our model is when it does things we don't let you do" is a pretty bad marketing line.

And this is all in the face of Anthropic stating that they think this is an existential issue for the human race. It's a bad look.

I'm not saying they don't restrict them, I'm saying they don't try to both take the moral high ground about it and simultaneously do marketing on the basis of it.

They're not saying "this is an existential risk" while pushing hard on exactly that risk.

I don't see the same level of hypocrisy from the other frontier labs.

Indeed, it's a sort of Academic Supremacy – "we're smart so we get to control the world". I think SV tech has had an aspect of this for a long time, but Anthropic do seem to be the clearest version of it in a while. Until regulation catches up.

Anthropic: You absolutely cannot, under any circumstances, use Claude for bio-engineering. It could literally end humanity.

Also Anthropic: Claude discovers a new way to edit your genome!

> Fun fact: Waymo pay $4 to the San José Department of Airports for every pick-up or drop-off at SJC.

This is pretty normal. LHR charges £5. SYD charges A$5. Airports are often highly congested and pushing more people to take public transit instead of private drop-off/pick-up is a big lever.

All of the "hacks" (from Google and others) were the same issue – the vendor Irregular running the models unsandboxed.

Specifically, the model hacked when run on 3rd party infrastructure without the necessary sandboxing. Given this was to test/benchmark certain capabilities it's also possible that this was a model without built-in guardrails.

Why do you think that? Waymo hasn't done that anywhere else.

I don't read it as informal. Childish or lazy perhaps, at best.

When you intend it as "inviting informality", you're implicitly doing this because "formality" is too much effort. The thing is you're not inviting, but rather demanding. You have decided the conversation is informal and low effort, and that's how you'll treat it, without considering the person you're communicating with.

This is of course all a lot of strong statements and these things don't matter as much as this sounds. I don't feel _that_ strongly about these things, but the lowercase thing always strikes me as just plain weird, and deconstructing why I feel that way this is where I get to.

> It's reasonable to assume that if AI drop-in-replaced all those knowledge workers, AI companies could credibly charge somewhere in that order of magnitude, because that's what the market is already bearing

This assumes you don't change the market, but at the scale of (checks notes...) "all knowledge work", that just doesn't hold.

For example if you put 1bn people out of work, you now need some sort of safety net to bail out much of that workforce, a truly unprecedented change. You also lose tens of trillions of dollars of tax revenue.

One solution might be to recoup that cost and lost tax revenue from businesses by raising corporation tax. If corporation tax went from low tens of percent to high tens of percent, would those businesses be able to afford all that AI? No. Same order of magnitude? I doubt it.

There are many possible futures there, but the simplification made in the parent comment is completely unrealistic. The article is right in calling out the valuations as crazy.

It's a tech bro thing. Altman does it too, and I've worked with people in the past who do it.

I read it as "I'll take literally any conscience for myself no matter how minor, at any cost for you no matter how big".

Sure, but installing a chess program is child/teen level general ability, and playing chess well is highly trained expert level ability. Which one are we sold AI as being?

> The reason ... is because the cost of one bug is many, many orders of magnitude higher than software. Both in dollar cost and in schedule cost (it takes months ... and if you messed up and need to spin a fix, it costs tens of millions of dollars, not counting any design engineering cost).

Aren't you just describing waterfall? That's still very prevalent in software engineering, and pretty much any other type of engineering – civil, chemical, building, architecture, drug discovery.

It's typically true that software can fail faster and cheaper, but it's also true that the costs are still vastly higher to fix later in the process.

We've had technology beating humans on memory for millennia, and we've had technology beating humans on computation for many decades now.

The tricky thing with LLMs is describing what they actually do. They are too clearly beating humans on some things, but what exactly? Memory – already done, they're bad at basic computation (all LLMs just write code for actual computation/calculation). And as you say, they do badly at more abstract concepts.

What do you mean, there's a screenshot link right there in the docs: https://docs.rs/crate/relm4/assets/screenshots /s