palata

Yeah I think we agree. I just wanted to note this:

> but the AI companies/engineers to their credit have been very clear on that risk

I am not a big fan of that. It's too easy to say "I make a ton of money harming society, but I'm good because I am telling me that you should regulate me". If someone told me "I hit my wife, but you should not criticise me because it is not illegal, and I have been telling for years that it should be illegal. But as long as it is legal I will hit her", I wouldn't suddenly respect them just because they are apparently very well aware that what they do it bad.

Fine with those engineers explaining how it is on their side, but not whining because other people think what they do is unethical. It sounds like "please, I need you to tell me that I am a good person even though I take money (and pride) contributing to something I know may hurt society".

> The difference is that 99.9% of software is just not as important as even your average two-lane bridge.

You miss the point. LLMs are not the two-lane bridge here, they are the trigger. Sure, when an LLM makes 99.9% of software fail, nobody cares. But what happens when it touches a single, very critical instance of software?

> Oh no, HuggingFace got hacked. So what?

Ever heard of NotPetya? Blocked Maersk unwillingly (as a collateral). Could have resulted in a global economical crisis. There is a nice Darknet Diaries episode about that.

> Also, even civil engineers very rarely go to jail when a building or bridge collapses.

At the very least they don't go public saying "well it happens, building bridges is hard". Usually it goes more "it shouldn't have happened, and we need to get our act together and make sure it doesn't happen again anytime soon".

> Tricky problem, isn't it?

I get the irony. And I also totally get why someone getting an insane salary would see no reason to not work on it (and cynically think "if I am not getting rich having fun here, someone else will... may as well be me"). Who am I to say I wouldn't do the same? I strongly believe that sending astronauts to space is fun but useless, and taxpayers shouldn't agree. But offer me to go to space? I go tomorrow. Ironic, isn't it?

I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing. And comparing yourself to Sully (while saying you don't).

> but don’t attack staff directly

Sure, agreed.

> I literally missed my sister’s wedding a few weeks ago to help clean up after some of the recent incidents

If you're not paid for that, you should just leave. But my understanding is that OpenAI employees cannot complain about their conditions, can they? Hard to feel for a millionaire saying "I earn orders of magnitudes more than you, but I work 50% more", I think (I don't know if it's the case here or not).

> all I hear on X is how I “don’t know how to configure a sandbox” or that I should be in jail

I think it can be two very different comments. "They don't know how to configure a sandbox" is condescending. "Someone should be in jail"... well that can make sense. When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.

The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure. It's hard to build a dam that does not collapse, but that's not a reason for civil engineers to vibe-build a dam and then say "ok it collapsed, but look it's hard to get right".

> Very few organizations can survive that scale and growth, and yet OpenAI did a very good job handling it (and should be commended).

So if OpenAI results in people dying, it is fine because "really, it's very hard"? I'm trying to think about a company that would build dams at such a scale and then use it as an excuse for only a fraction of them collapsing from time to time.

> To say it lightly: this surprised the fuck out of us. We had not expected it this soon. Suddenly we weren’t dealing with just a small jump in capabilities; we were talking about a different sport altogether.

Last example: imagine a geoengineering company coming out with a "solution" to global warming. Suddenly it goes out of control and it screws us even more. Would we say "please, have some respect, having that kind of impact is really hard and it honestly surprised the fuck out of us", or would we say "maybe you shouldn't have done something dangerous you didn't understand"?

> I recently watched the movie Sully.

Okay I can't resist it. It's inverting the argument. In Sully, we accept the fact that they took risks they shouldn't have because it turns out that they saved lives (and they prove that doing what they should have would have resulted in more casualties).

Now if the plane had crashed into a tower and killed thousands of people, of course we would blame the pilots!

OpenAI resulting in security incidents is NOT AT ALL the same as Sully saving lives by not following the rules. What a weird comparison that is?

If Starship crashed in Central Park, I wouldn't expect a SpaceX engineer to write a blog post saying "Look, luckily it didn't kill anyone, and I'm sick of reading comments saying "someone should go to jail". It's very hard to build a rocket, and people who think otherwise are just misinformed. Give us some slack, crashing into Central Park surprised the fuck out of us".

Maybe... legacy?

I guess we disagree on the meaning of "relatively easy". I do it a lot, I consider it boilerplate. It's not like passing 10 strings is exponentially harder than passing 1 string, is it?

I agree that the more you mix, the harder it gets. Maybe I'm just never working on "serious" projects, but I have never been in a situation where I had to mix 10 languages.

Well once you have a C API, it is relatively easy to call it from any other language.

The painful part is to have to go through a C API (modern languages can express much richer APIs and of course there are different constraints on the different runtimes, e.g. GC).

The annoying part is that each language adds overhead (its runtime). I wouldn't call it painful (I don't have much to do about it), I say "annoying" just because I would rather minimise the amount of code I ship.

I agree that mixing languages adds complexity. And I say that as someone who routinely does it, because many times it's better to reuse a mature/audited component than rewrite it from scratch.

Sometimes it's about the libraries. E.g. writing Computer Vision is nicer in C++ right now (IMHO) because most CV libraries are in C++.

Similarly I like to do video stuff in C just because I call gstreamer/ffmpeg directly in C, rather than having to bridge everything.

In that case, a real problem if you sell hammers is that you might not be able to prevent people from hitting their wife with them.

Not sure it is a reason for making hammers illegal?

For some reason webtorrent never took off, though

To be fair, it has always been like this. People actively became dependent on the Internet, on StackOverflow, fast Internet connection, cloud services, etc.

At a company I had setup scripts to build our packages, and the CI was running those scripts. Someone more junior (only a few years, not decades) found it strictly superior to remove my scripts and replace them with GitHub actions: people could now know even less about it (as in, no need to know how to copy-paste and adapt the recipe for a new package), but now it depended on GitHub. GitHub is down, nobody can build anything anymore. And it happened once every few weeks, so people would just go have a coffee during the outage.

You know what happened next? That person got promoted for their good work. That was before AI.

> part of the answer is obviously "it's hard to have a classifier that smart" and "it'll have false positives" but even a very bad safeguard would have stopped this.

And maybe "they are running after glory, not safety"?

Still we had a peak of conventional oil in 2007, I don't get why I'm getting downvoted for that fact.

This is wrong. The peak of conventional oil is documented around 2007. Since then, Europe's economy has been slowing down (if you look at physical indicators like how much is transported by trucks or what surface is being built each year, not artificial ones like the GDP).

Global peak is estimated to be around now (a few years ago maybe?).

> If a power generation system costs more to operate than the value of power we get out of it, it's probably not very useful.

There is more than money. There is stability (not depending on foreign oil, peak oil, etc), there is climate change, etc.

Sure, but then that's not "profitable" in the direct sense of "makes rich oligarchs richer".

I'm all for improving the quality of life for society. In the end the goal is to live well, not to destroy our living conditions for a few oligarchs to make more money. But "profitability" is not generally optimised like this, unfortunately.

Healthcare is profitable like schools are, or roads, or public transportation, or sanitary systems: the goal is for us a society to live better.

When something is profitable in the sense that a few oligarchs make more an more money, I see it as a bug, not a feature.

Profit is not everything.

> That being said, many countries consider it a national security issue not to be held hostage to foreign oil.

Spot on. That, plus... peak oil.

Choice, or incompetence? Or maybe structural incompetence: those who decide what's being built only care about maximising profit, those who know how to build stopped caring because... why would they?

When whoever is in power finds it fine to build such terrible products full of ads, the only thing worth doing is the minimal amount of work required to get your salary, if you ask me.

Interesting insights.

I kind of feel like as a human, I like reading a good blog post explaining something interesting. Can be a tutorial or just "something challenging". That helps me understand a concept, or build a mental abstraction.

I can also point an LLM to such a good blog post as a resource ("I want you do to X the way it's described in this post").

Sadly, though, I feel like people are less and less motivated to actually read the blog post. They will ask the LLM to summarise the good blog post, which may not be so constructive. Or will just completely ignore it. And if you are not used to reading good blog posts, how do you make the difference between a good one and AI slop?

A third reason I was writing blog posts was that I thought it was a portfolio. When applying for a job, I could point the employer to my open source code and to my blog, and they could judge me based on actual stuff I produced. My experience is that nobody cares: they will redirect you to their vibe-coded leetcode goddamn tool and check if you can invert a BSP in less than 4 minutes.

I have failed interviews in companies that build software on top of a library I wrote because a junior felt I was not good enough at leetcoding. Fun fact: the junior in question obviously wasn't better than me: they just had read the solution before the interview so they could feel like they knew better.

Actually they don't work because those apps suck. Typically the banks are responsible for preventing their own apps from working there. Because they don't care or because they follow some bullshit security-theatre policies, or because they just don't understand security, I don't know.

It's not Google that adds the code that makes your banking app fail: it's the goddamn bank itself.

But I agree with your point that the reason it sucks is not the AOSP-based system, and it's sad to blame the system when it's the app (or many times just Google) that is to blame.

Oh, that explains it. I'm still on Opus 4.8 (5.0 was too annoying), and I thought I had missed a few releases...

> Does that mean that I can no longer trust Opus with not snitching my AI research to Anthropic either?

You never could trust it. Fundamentally you leak everything your LLM uses to their servers.

For a smaller manufacturer (e.g. Fairphone), that may mean a lot. If I remember correctly in 2021 Fairphone was selling 100k phones per year?

Also I am genuinely interested: is it that hard to meet GrapheneOS' requirements? Feels like it shouldn't take years of R&D for every new iteration. And also that it's about getting the right deals with the suppliers, which is not really an R&D issue?

I changed bank because they stopped supporting GrapheneOS for bullshit reasons (they support my completely unsecure and outdated /e/OS but not GrapheneOS, and they call that "security").

The only way a bank cares is if enough users complain and leave for another bank. Banks (like all companies, but especially banks) care about money and nothing else.

GrapheneOS is installed with their keys which you can verify, right?

My understanding is that the problem is not on GrapheneOS' side. We need more manufacturers to care and build devices that meet GrapheneOS' requirements.

My hope is that Motorola will be very successful with that. GrapheneOS currently has half a million users and growing. The more users it gets, the more interesting it is for manufacturers to actually consider it.

Google being Google, if Motorola gets a relatively cheap phone to run GrapheneOS (the flagship is quite expensive IMHO), I think people will flock away from Pixels (I definitely will, and I will tell everyone and their dog that they should when they renew their phone).

> Switch to android which has even more ads?

I am on Android, I have no ads.

> The average person has no idea what lineageOS is

The beauty of it is that the average person can buy a LineageOS-enabled system without having to know it's not "normal Android". Look at /e/OS for instance, and hopefully soon GrapheneOS will come preinstalled on some Motorolas.

> There are secure ways to whistleblow

... when your adversary is the NSA? We're circling back to my original message.