Significant raise of reports

Yeah that attitude really makes no sense, and I don't see why AI finding security bugs would make people "finally understand".

I suspect it's just an excuse for Linux's generally poor security track record.

And this is the best-case scenario. Because once updates become opt-out it simply becomes an attack vector of another type.

If the updated code is not open source, you are trusting blindly that not some kind of different remote code execution just happened without you knowing it.

I think part of it is that, especially at the kernel level, it can be hard to really categorise bugs into security or not-security (it has happened in the past that an exploit has used a bug that was not thought to be a security problem). There's good reason to want to avoid updates which add new features and such (because such changes can introduce more bugs), but linux has LTS releases which contain only bug fixes (regardless of security impact) for that situation, and in that case you can just stay up to date with very minimal risk of disruption.

And if you're the kind of person who cares about that, you pay a vendor that gives you 10 years on the same distro version.

Or just use an off-brand RHEL I guess.

Details are important, but my mental model has settled as: Security bugs are being use in a manner to how politicians use think of the children. It's used as an auto-win button. There are things to me that compete with them in priorities. (Performance, functionality, friction, convenience, compatibility etc); it's one thing to weigh. In some cases, I am asking: "Why is this program or functionality an attack surface? Why can someone on the internet write to this system?"

Many times, there will be a system that's core purpose is to perform some numerical operations, display things in a UI, accept user input via buttons etc, and I'm thinking "This has a [mandatory? automatic? People are telling me I have to do this or my life will be negatively affected in some important way?] security update? There's a vulnerability?" I think: Someone really screwed up at a foundational requirements level!.

> Linux devs keep making that point, but I really don't understand why they expect the world to embrace that thinking. You don't need to care about the vast majority of software defects in Linux, save for the once-in-a-decade filesystem corruption bug.

The point is that all of those bugs are now trivial to exploit and so will be exploited

>it takes effort to familiarize yourself with new features, decide what should be enabled and what should be disabled, etc.

What features? I update my rolling release once a month and nothing changes for the last 10 ish years. Maybe pipewire/pulse thingy was annoying and bluetooth acted a bit. With docker on rpi I even upgrade the whole zoo of things by just rebooting.

but this simply isn't true. everyone thinks "oh well my use cases will never hit any of those bugs", but then there is one person in your org who hits that particular bug and it drives them batty. it is a retro-justification for doing things the wrong way "For the Right Reason". like... no one would be like "NEVER change the oil in your car unless the light goes off". we're not talking about Micro$oft here, where you literally have to pray to your deity of choice every time you click the update button. we are talking about the Linux kernel. i do not even need a thumb to count on one hand the amount of times a kernel update has significantly impacted my life. whereas probably 50% of my Windows updates break at least one of my peripherals, and OS X isn't exactly much better these days.

> Was software made before 2000 better?

At the time of release, yes. They had to ensure the software worked before printing CDs and floppies. Nowadays they release buggy versions that users essentially test for them.

I was a developer at Microsoft in the 90s (Visual Studio (Boston) and Windows teams). I won't claim that software back then was "better," but what is definitely true is that we had to think about everything at a much lower level.

For example, you had to know which Win32 functions caused ring-3 -> ring-0 transitions because those transitions could be incredibly costly. You couldn't just "find the right function" and move on. You had to find the right function that wouldn't bring your app (and entire system) to its knees.

I specifically remember hating my life whenever we ran into a KiUserExceptionDispatcher [0] issue, because even something as simple as an exception could kill your app's performance.

Additionally, we didn't get to just patch flaws as they arose. We either had to send out patches on floppy disks, post them to BBSs, or even send them to PC Magazine.

[0]: https://doar-e.github.io/blog/2013/10/12/having-a-look-at-th...

> Was software made before 2000 better?

Literally the moment everyone got on the internet, pretty much every computer program and operating system in the world was besieged by viruses and security flaws, so no.

Just think of 8 and 16 bit video console games. Those cartridges were expensive so just how sure they had to be they were bug free before making millions of them?

It was a simpler time. Not better. Not worse. Programs still had bugs, but they weren't sloppy UI bugs, they were logic bugs and memory leaks. If software was better back then, we'd still be using it!

I think there was a period where things got better but I don’t think it was pre-internet.

There was a point in time where both windows wasn’t constantly bsoding and Microsoft’s primary objectives weren't telemetry and slop coding.

Yes and no.

Yes. The incentives for writing reliable, robust code were much higher. The internet existed so you could, in theory, get a patch out for people to download - but a sizeable part of any user base might have limited access, so would require something physical shipped to them (a floppy or CD). Making sure that your code worked and worked well at time of shipping was important. Large corporate customers were not going to appreciate having to distribute an update across their tens of thousands of machines.

No. The world wasn't as connected as it is today, which meant that the attack surface to reasonably consider was much smaller. A lot of the issues that we had back then were due to designs and implementations that assumed a closed system overall - but often allowed very open interoperability between components (programs or machines) within the system. For example, Outlook was automatable, so that it could be part of larger systems and send mail in an automated way. This makes sense within an individual organisation's "system", but isn't wise at a global level. Email worms ran rampant until Microsoft was forced to reduce that functionality via patches, which were costly for their customers to apply. It damaged their reputation considerably.

An extreme version of this was openness was SQL Slammer - a worm which attacked SQL Servers and development machines. Imagine that - enough organisations had their SQL Servers or developer machines directly accessible that an actual worm could thrive on a relational database system. Which is mindboggling to think about these days, but it really happened - see https://en.wikipedia.org/wiki/SQL_Slammer for details.

I wouldn't say that the evidence points to software being better in the way that we would think of "better" today. I'd say that the environment it had to exist in was simpler, and that the costs of shipping & updating were higher - so it made more sense to spend time creating robust software. Also nobody was thinking about the possible misuse or abuse of their software except in very limited ways. These days we have to protect against much more ingenious use & abuse of programs.

Furthermore today patching is quick and easy (by historical comparison), and a company might even be offering its own hosted solution, which makes the cost of patching very low for them. In such an environment it can seem more reasonable to focus on shipping features quickly over shipping robust code slowly. I'd argue that's a mistake, but a lot of software development managers disagree with me, and their pay packet often depends on that view, so they're not going to change their minds any time soon.

In a way this is best viewed as the third age of computing. The first was the mainframe age - centralised computer usage, with controlled access and oversight, so mistakes were costly but could be quickly recovered from. The second was the desktop PC age - distributed computer usage, with less access control, so mistakes were often less costly but recovering from them was potentially very expensive. The third is the cloud & device age, with a mix of centralised and distributed computer use, a mix of access control, and potentially much lower costs of recovery. In this third age if you make the wrong decisions on what to prioritise (robustness vs speed of shipping), it can be the worst of both the previous ages. But it doesn't have to be.

I hope that makes sense, and is a useful perspective for you.

It is hard to say which of the 2 is the reason, more likely both, i.e. lower complexity enabled more exhaustive testing.

In any case some of the software from before 2000 was definitely better than today, i.e. it behaved like being absolutely foolproof, i.e. nothing that you could do could cause any crash or corrupted data or any other kind of unpredictable behavior.

However, the computers to which most people had access at that time had only single-threaded CPUs. Even if you used a preemptive multitasking operating system and a heavily multi-threaded application, executing it on a single-threaded CPU was unlikely to expose subtle bugs due to race conditions, that might have been exposed on a multi-core CPU.

While nowadays there exists no standard operating system that I fully trust to never fail in any circumstance, unlike before 2003, I wonder whether this is caused by a better quality of the older programs or by the fact that it is much harder to implement software concurrency correctly on systems with hardware parallelism.

Depends what you mean by better. It crashed more and there was a lot of data loss, but it wasn't explicitly evil so maybe on measure it was better.

It was the best of times, it was the worst of times.

Best/better because yes, QA actually existed and was important for many companies - QA could "stop ship" before the final master was pressed if they found something (hehe as it was usually games) "game breaking". If you search around on folklore or other historical sites you can find examples of this - and programmers working all night with the shipping manager hovering over them ready to grab the disk/disc and run to the warehouse.

HOWEVER, updates did exist - both because of bugs and features, and because programmers weren't perfect (or weren't spending space-shuttle levels of effort making "perfect code" - and even voyager can get updates iirc). Look at DooM for an example - released on BBS and there are various versions even then, and that's 1994 or so?

But it was the "worst" in that the frameworks and code were simply not as advanced as today - you had to know quite a bit about how everything worked, even as a simple CRUD developer. Lots of protections we take for granted (even in "lower level" languages like C) simply didn't exist. Security issues abounded, but people didn't care much because everything was local (who cares if you can r00t your own box) - and 2000 was where the Internet was really starting to take off and everything was beginning to be "online" and so issues were being found left and right.

Define better.

Before 2000 fixing a bug the user would notice was expensive - you had to mail them a new disk/cd. As such there was a lot more effort put into testing software to ensure there were no bugs users would notice.

However before 2000 (really 1995) the internet was not a thing for most people. There were a few viruses around, but they had it really hard to propagate (they still managed, but compared to today it was much harder). Nobody worried about someone entering something too long in various fields - it did happen, but if you made your buffers "large" (say 100 bytes) most forms didn't have to worry about checking for overflow because nobody would type that much anyway. Note the assumption that a human was typing things on a keyboard into fields to create the buffer overflow. Thus a large portion of modern attacks weren't an issue - we are much better at checking buffer sizes now than there - they knew back then they should, but often got away with being lazy and not doing it. If a vulnerability exists but is never exploited do you care - thus is today better is debatable.

In the 1990s the US had encryption export laws, if you wanted to protect data often it was impossible. Modern AES didn't even exist until 2001, instead we had DES (when you cared triple DES which was pretty good even by today's standards) - but you were not allowed to use it in a lot of places. I remember the company I worked for at the time developed their own encryption algorithm for export, with the marketing(!) saying something like "We think it is good, but it hasn't been examined near as well as DES so you should only use it if you legally you can't use DES"

As an end user though, software was generally better. They rarely had bugs anyone would notice. This came at the expense of a lot more testing, and features took longer to develop. Even back then it was a known trade off, and some software was known to be better than others because of the effort the company put into making it work before release. High risk software (medical) is still developed with a lot of extra testing and effort today.

As for the second part - software back then was plenty complex. Sure today things are more complex, but I don't think that is the issue. In fact in some ways things were more complex because extra effort was put into optimization (200mhz CPUs were the top end expensive servers, most people only had around 90mhz, and more than one core was something only nerds knew was possible and most of them didn't have it). As such a lot of effort was put into complex algorithms that were faster at the expensive of being hard to maintain. Today we have better optimize rs and faster CPUs so we don't write as much complex code trying to get performance.

There are some rose-colored glasses when people say this.

Programs didn’t auto save and regularly crashed. It was extremely common to hear someone talk about losing hours of work. Computers regularly blue screened at random. Device drivers weren’t isolated from the kernel so you could easily buy a dongle or something that single-handedly destabilized your system. Viruses regularly brought the white-collar economy to its knees. Computer games that were just starting to come online and be collaborative didn’t do any validation of what the client sent it (this is true sometimes now, but it was the rule back then).

It appeared better, because there were fewer features and more time to develop and test. But it's also a lot of nostalgia, because everything moved slower, the world was smaller, there was a lower standard; people will usually remember the later versions of a software, or never even encountered the earlier versions. Without the internet and every one bitching about every little detail, the general awareness was also different, not as toxic as today.

Delta, JetBlue, American Airlines and Alaska Airlines have free Internet as long as you are enrolled (for free) in their loyalty programs.

JetBlue and Delta use ViaSat. I only fly Delta for the most part and ViaSat was available on all domestic routes I’ve flown except for the smaller A900 that I take from ATL to Southwest GA (50 minute flight). Then I use my free unlimited 1 hour access through T-Mobile with GoGo ground based service.

I’ve been considering that this might be an outcome of AI-written software and it’s the one aspect of all this that I’m actually unequivocally happy about.

Most software written at companies is shit. It’s whatever garbage someone slapped together and barely got working, and then they had to move onto the next thing. We end up squashing a never ending list of bugs because in a time-limited world, new features come first.

But that only really applies when the cost of good software dwarfs that of barely-functioning software. And when the marginal cost of polishing something is barely longer than it took to write it in the first place? There’s no reason not to take a few passes, get all the bugs out, and polish things up. Right now, AI can (and will) write an absolutely exhaustive set of test cases that handles far more than a human would ever have the motivation to write. And it will get better.

If a company can ship quality software in essentially the same time as it can ship garbage, the incentives will change rapidly. At least I hope so.

Yeah I don't think that is true at all. Plenty of software today is very well tested, and plenty of software back then was pushed out with insufficient testing due to short deadlines (some probably caused by the fact that they had to press CDs).

Not all software are done with the same quality, whatever the epoch.

It was possible to work with Ada as soon as 1980 wherever high guarantee of reliability was taken seriously, for example.

And not everyone is Knuth with a personal human secretary in well funded world-top institution.

In 2000s, Microsoft which was already sitting on insanely high mountain of resources released Windows Millennium Edition. Ask your greybeard neighbour if you are too young to remember. While commercialisation started in 2000, it is the last MS-DOS-based Windows version and so represent the pinnacle of what Windows 9x represented, before the big switch to a NT inheritance.

As always, the largest advantage of the good all time, is selective memory. After all, people that can remember know they survived the era, while present and future never provided much certainty on that point.

It feels like rose-tinted glasses. While lots of low-hanging fruit had to be plucked to be shippable, there was still plenty of software which mandated specific hardware/software combinations or (worse) had major bugs which weren't patched but had workarounds documented in the manual, and if you weren't actively reading the manual, your newly-purchased software just wouldn't work (and if it was something low-level, that may mean you have to reinstall the OS).

Then there was stuff like rwall, which could be used to scrawl a message across basically every terminal connected to a networked Unix box in the world by accident [0][1], and it was far from the only insecure-by-design Unix software in widespread use.

It's interesting to watch youtubers like clabretro [2], NCommander [3], and Old Computers Sucked [4] who have documented the slog that was setting up and patching networking equipment, obscure Microsoft products, Netware, Unixes and Unix hardware, old Linux distros, etc. We take so much for granted these days. We don't even have to think about C/++ standards compliance outside the occasional compiler bug, much less the myriad of mutually-incompatible POSIX implementations that helped Microsoft win the Unix wars.

The fact that you can just build a PC with no prior experience or IT knowledge after watching an hour-long youtube video rather than having to spend weeks researching hardware compatibility or futzing about with IRQ levels, recompiling kernels, and messing with autoexec.bat/config.sys is a testament to how far we have come. You don't even have to think about drivers anymore unless you have specialized equipment.

[0]: https://news.ycombinator.com/item?id=31822138

[1]: https://news.ycombinator.com/item?id=35759965

[2]: https://www.youtube.com/@clabretro

[3]: https://www.youtube.com/@NCommander

[4]: https://www.youtube.com/@old-computers-sucked

The quantity of tests, known as penetration attempts, that most critical software survives today in a networked environment, is magnitudes more daring that the easily-cracked software printed on CDs. I really don't understand how this argument about software made 26 years ago really stands any reasonable ground.

I don't think it was better, but it was generally less buggy or higher quality for sure. And I think it's both because of testing and because of lower complexity.

For example, desktop apps are actually much less complex than web based applications. It doesn't really seem that way, but web apps are distributed, with a client and server on separate machines. That, to me, introduces a whoooole new layer of error conditions and flakiness.

Also, QA has been largely cast aside. Nobody is really doing manual click-around type testing, and nobody is writing user guides anymore.

There's an anecdote I remember reading somewhere: When an 'embedded systems' engineer was to present a web-based product they were tasked to build, the managers/reviewers were puzzled they couldn't find any bugs. Asked about this, the engineer replied: "I didn't know that was an option".

Definitely a different mindset/toolset is required when it comes to building systems that have to be working autonomously without "quick fixes" from the web.

My gut feeling and expectation is that people will be turning their internet off at night, and at all times. At least for a while until this whole new security situation somehow settles with newly invented automation.

May sound weird, but as author of previous comment noted - a lot of appliances need not be connected ever and still benefit humanity.

“Reversing was already mostly a speed-bump even for entry-level teams, who lift binaries into IR or decompile them all the way back to source. Agents can do this too, but they can also reason directly from assembly. If you want a problem better suited to LLMs than bug hunting, program translation is a good place to start.”

Huh. Direct debugging, in assembly. At that point, why not jump down to machine code?

Around 70% of security vulnerabilities are about memory safety and only exist because software is written in C and C++. Because most vulnerabilities are in newly written code, Google has found that simply starting writing new code in Rust (rather than trying to rewrite existing codebases) quickly brings the number of found vulnerabilities down drastically.

What do you mean "a priori understanding codebases"? Quantify it and let's test specifically what you mean. Linux is huge.

The fact that it's millions of LoC is borderline irrelevant in that context, you don't need to have it all in context to find bugs in a handful of files.

> Can linters find these? Perhaps fuzzing?

That's what syzbot / syzkaller does, as mentioned in the article, with somewhat similar results to the AI-fuzzing that they've been experiencing recently.

The issue that Linux maintainers have in general is that there are so many of these "strict correctness and safety" bugs in the Linux codebase that they can't fix them all at once, and they have no good mechanism to triage "which of these bugs is accessible to create an exploit."

This is also the argument by which most of their bugs become CVEs; in lieu of the capability to determine whether a correctness bug is reachable by an attacker, any bug could be an exploit, and their stance is that it's too much work to decide which is which.

> I suspect it's not very significant: we're flushing a 20+ year backlog, and generally the rate at which vulnerabilities are created is lower today.

The thing is: if these new AI tools can find a backlog of old bugs, these tools can very obviously be used on code that hasn't been pushed yet. And they'll find potential bugs there too. And so the rate at which new vulns are created is soon going to be even much, much, much lower.

Now of course I'm talking about serious projects like the Linux kernel in TFA: real stuff that powers the real-world. If we're talking about OpenClaw who decided to launch a startup based on a "Write me a clone of MySpace but with a Web design from the 2020s" prompt, then all bets are off.

The nice thing with using AI tools to find bugs is that there's not much ambiguity: a bug, if proven to be a bug, has to be squashed. It doesn't matter how it was found: even the AI doubters can accept there's a bug and that something has to be done about it.

Using AI tools to fix bugs in the Linux kernel is IMO much more impressive than "Write me the 10 000th MySpace clone but using a Web design from the 2020s".

To clarify, the talk is by an Anthropic researcher, though given the subject of LLMs, "entropic researcher" also makes some kind of sense.

It will be gamed, just as pagerank was.

agreed, attackers can use these AI tools to scan open source code and find bugs very fast... if project maintainers do not have access to such tools, it because an ufair fight

It can be correct and slop at the same time. The reporter could have reported it in a way that makes it clear a human reviewed and cared about the report.

Slop is a function of how the information is presented and how the tools are used. People don't care if you use LLMs if they don't tell you can use them, they care when you send them a bunch of bullshit with 5% of value buried inside it.

If you're reading something and you can tell an LLM wrote it, you should be upset. It means the author doesn't give a fuck.

If I read the sentence correctly they're saying that past reports were AI slop, but the state of the art has advanced and that current reports are valid. This matches trends I've seen on the projects I work on.

I think they are saying what you want them to say. In the past they got a bunch of AI slop and now they are getting a lot of legit bug reports. The implication being that the AI got better at finding (and writing reports of) real bugs.

Is there a reason you’ve copy pasted the first paragraph from the link? It doesn’t add anything to the discussion, and also doesn’t help as a tl;dr because it’s literally the first paragraph. Genuine question!

Could you back up your assertion that Willy Tarreau — who used to maintain the Linux kernel — is “an AI enthusiast”? I can’t find anything about it.

> but you never really see any gathered data backing these points up.

https://www.anthropic.com/news/mozilla-firefox-security

?

Sort of a tautology to just assert that someone saying good things about AI is an AI enthusiast and therefore their opinion should be dismissed. He also happens to have been a kernel maintainer, his experience as he's describing it should count for something.