The things nobody wants to pay for
lwn.net
lwn.net
In the case of open source, that algorithm usually identifies volunteers. Sometimes it identifies businesses. Sometimes people come up with clever ways of funding projects (Red Hat the enterprise was genius from this perspective, they organised a solution to tragedy of the commons in core infrastructure and made a lot of money doing that).
I suspect there does need to be a regular social renewal to resolve the tragedy of the commons problem. I expect that the network of OSS foundations and companies will regularly becomes corrupt and ineffective and need to be purged then replaced. That implies there is a fine line of healthy complaining - people need to communicate where the problems are by complaining, but it is useless trying to blame people like kernel maintainers for poor testing. It is good to stop and appreciate how beautiful the dance is; this is one of the purest free markets we have in the world and it is pretty amazing how well it hangs together since it looks like a hearty sneeze could bring the whole thing down.
Wouldn’t the solution offered on LWN work great?
Just excise a tax (well, I guess a licensing cost) for open source use to any corporation that is above x$ amounts of revenue. Make it a tiny percentage (0.1%?).
2022 revenue:
- $400 billion, Apple
- $200 billion, Microsoft
- $120 billion, Meta
- $80 billion, AWS
$800 billion * 0.01 = €800 million of “sovereign OSS fund” funding.
And that’s just the big ones. Bing tells me the 2022 combined revenue of all NASDAQ listed companies is ~$6900 billion. That’s $6.9 billion in OSS funding you’d skim off!
What’s more, with how much that would improve open source projects, those companies would see enough productivity improvement that despite the 0.1% in skimmed revenue, they’d see a net benefit.
Google makes open source Android, Chromium, Go, Dart, Flutter, Lit, ANGLE, Dawn. They also spend millions on open source.
Microsoft makes C#, F#, VSCode, Electron, Powershell, and further Microsoft is providing free hosting for probably 95% of all open source projects (github) as well as free CIs (github actions) and free JS/TS library distribution (npm). They're spending millions on open source too.
I don't know what Meta provides except React and LLaMa but that's still millions
Taxing requires administration so what you'd really be doing is giving a bunch of non-developers money in the form of salaries. The rest they'd give for their friend's projects, or whoever bribed them with kickbacks etc...
And the intentions. It is fine to care about individual requirements but not all target the common society.
Flutter and Chrome (Blink) are mere exclusive Google projects, they just shall strength Android, ChromeOS and their cloud business. Same for Microsoft stuff, they add stuff which strength their cloud business - with Linux - or WSL.
WebKit is developed in cooperation and there is WebKitGtk. This benefits the common. Also LLVM helps the common. But Apple tries to avoid the GPL whenever they can. The core of WebKit is luckily LGPL. This is also one of the reasons why many prefer GCC (GPL).
btw.
The sovereign tech fund (from Germany) is a good idea. A state must be interested in good infrastructure. Software and especially Linux is infrastructure.
America had once back in time something similar by pure luck. AT&T was restricted to enter new markets. Out of nowhere they started developing UNIX, and C and provided the source and documentation. Sadly the restrictions where lifted. As far as I know this was the last time that BigIT was actively regulated in America. It was good and accelerated literally everything.
Chromium is used by Electron and through that 1000s of projects. Element, VSCode, Slack, Discord, Figma, Signal, Twitch and thousands of open source projects. It's also the basis for Edge, Brave, Opera, and several other browsers.
Flutter is also used by 1000s of projects both commercial and open source.
Android is also used by thousands of projects, Fire sticks, Refrigerators, TVs, USB Stick Computers, Security systems, and on and.
All of these are "helping the commons". I've personally written 4 Electron apps. 2 are open source. They wouldn't exist without it as I wouldn't have had the time to make apps for multiple platforms.
Isn't it a non-idea? I'm seeing articles saying they spend order of magnitude single digit million dollars [0]. That doesn't sound like it'd be enough to cover the costs of much. Just look at the list of maintainers of the linux kernel, for example [1].
Germany would be doing much better if they had a few big software corporations, they tend to contribute more money voluntarily. They should focus on cutting back red tape and fostering some entrepreneurship.
> It was good and accelerated literally everything.
This position doesn't make sense, look at the the AI industry appearing out of nearly nowhere and is moving so fast even the people in the market can't keep up. And is mostly open source. We might be approaching the limit of what is physically possible.
No changes to the situation are needed if we want things to move faster. Maybe winding down IP protections to maintain the speed, once the lawyers get involved that'll kill some of the momentum.
[0] https://sciencebusiness.net/news/cybersecurity/germany-launc...
[1] https://www.kernel.org/doc/html/latest/process/maintainers.h...
I’m also afraid that our Fax machines were even a priority.
btw. Germany has a law which enforces the endpoint and declares that customers attach their own modems. Democracy allows citizen to decide - we can prioritize people’s needs over companies?
This channel I think was the A channel of an ISDN line which had 2 b channels.
I once had a Cabbie in NYC tell me about Citibank getting the first Fax Machines installed in NYC. They connected the Manhattan Office tower with the Queens office.
They remember it getting installed and there was a long lang time before anybody used. The opinion was that it didn't make sense compared to all the better ways to move documents.
The scheme that was posited could potentially land us in a really perverse place. We'd be paying Apple way more than they put in because they don't use much but they pour out projects that others use.
My suspicion is that this probably works for some of the big companies, but for places like Apple, we don't want to get into this situation at all. This would just be a new revenue stream for Apple for doing something that they would have done for free anyway.
If Apple is a net creator of useful open-source software, I don't see this as much of a problem.
If it results in a more viable/better open-source ecosystem overall, I'm more than happy to see all the large contributors to that benefit. (I'm ideologically for open-source software; I'm not ideologically anti-Apple/anti-company.)
A little off topic but this line of reasoning scares me. The end result of this kind of thought is "demonstrated need". I sincerely believe there should be an end to means-testing and anyone should be eligible for public benefits regardless of income or assets.
Perfect, they would on the same terms as everyone else be able to apply for funding from the OS-fund to support this development – given they comply with license restrictions, etc.
Plus, when you're asking for a tax on say Apple, where they get to cough up multiple billions, then expect them to invest plenty in lobbying and lawyers.
In other words this plan falls into the category of "sure, that sounds great, but there is simply no path for this to become reality."
The difference between FOSS and OSS.
> In other words this plan falls into the category of "sure, that sounds great, but there is simply no path for this to become reality."
And yet despite the extreme market capture of Meta we now have things like ActivityPub and soon WhatsApp will have support for 3rd party messaging services.
Dare to dream. Things can improve.
Edit: I guess HN prefers to stay stuck in quagmire rather than dream of improvements. Sad state of affairs.
The dichotomy between Free and Open Source software centers around political ideas around protecting user freedom.
[0] https://opensource.org/osd/
Edit: re: your "hopes and dreams" edit - I think it's safe to say reasonable people have no problem with proprietary source-available licenses (like the BSL, for example). Just don't call those types of licenses "open source". They aren't.
Those who attempt to free-ride on the goodwill Free and Open Source has amassed by using improper terminology to describe proprietary source-available licenses are what create frustration.
Thanks for making me aware of the term source-available and the BSL.
> Just don't call those types of licenses "open source". They aren't. Those who attempt to free-ride on the goodwill..
Well, to be honest, organisations can't lay claim to general language concepts.
Many many people use "open source" to just mean "source openly accesible", and use "permissive source" for stuff licensed with GPL/BSD/Apache etc.
The fact that many people use the phrase incorrectly is exactly the problem that needs correcting. Licenses gave very real legal consequences, and calling yourself Open Source when you're not is not a good way to create confidence with potential customers.
>> Edit: I guess HN prefers to stay stuck in quagmire rather than dream of improvements. Sad state of affairs.
Dreaming in itself doesn't lead to improvement. There's nothing wrong with dreaming per se, but it won't solve the problem raised by the parent article.
There are other course lots of ways to "improve" the situation. Microsoft has a model that generates mountains of documentation, and they find a way to pay for it. That model though (commercial software generating sales revenue) is not compatible with OSS licenses. I'm guessing we're looking for an "improvement" here that doesn't involve changing away from OSS licenses.
Even if that wasn't a deal-breaker though, fixing problems with new taxes is seldom a viable solution in the real world.
Taxes are seldom so specific - occasionally things collected here are used over there but mostly it's a giant pot. So your suggestion would be better described as "create a govt fund, seed it with billions a year and issue grants from that."
Certainly the money would have to come from somewhere, which means either raising some general tax, of cutting spending elsewhere. In case you haven't noticed there is little appetite in politics to increase spending. One party regularly lowers taxes and increases the deficit. The other wants to expand social services and increase the deficit. All sides see an increasing deficit as bad. (Go square that circle)
Interestingly European Govts -are- spending on OSS, and issue grants. But there's little appetite got this in the US.
So, by all means dream. By all means throw out ideas. But accept that most ideas are bad. This one is bad I multiple directions. That's no big deal, it'll help inform your next idea, which may be good.
It doesn't fit either of the definitions
This is America, new taxes are nearly impossible to pass, tax cuts are much easier to pass.
But having tax deductions solves the problem. There is an incentive to support open source, but the companies can decide themselves where the effort goes.
No, you absolutely can't set such an organization and expect it to work. You are marginally better giving the task to some government body, but expect failure either way.
I don’t know what you were expecting.
I think you are missing the actor there. Who is going to enact and enforce this tax? I can't think of any entity that could and would.
Are there companies that are gonna try to dodge it? Sure. Same happens with companies that refuse to open source modified GPL code. But the majority plays ball. Same with this funding, we wouldn’t be looking for perfection but for an improvement on the current situation.
That's an auction, not a market. Markets are two sided, there are differentiated (multidimensional) offers and bids that are seeking to get matched. Much more complex
None of that necessarily invalidates the point that people are paid the minimum amount possible for a given pool of labor supply
At any given _instant_ in time there is a steady state where there is queue of people willing to buy and a queue of people willing to sell time.
There are no market makers in the job market. What OP says is true
People working on software projects could apply for grants, which would be reviewed by a panel of peers and experts, then funded, and the impact of the software project can be reviewed. I would think that judging the value of a software project would be easier than many basic science research, because the impact of software is more immediate and easier to measure. Yes, any system that involves money will be susceptible to some amount of fraud, waste, and deception. But most people still support funding science and medical research.
> People working on software projects could apply for grants, which would be reviewed by a panel of peers and experts, then funded, and the impact of the software project can be reviewed.
That's the dream. The reality is that the money will get allocated according to politics and corruption. The money will come with strings attached, and a lot of extraneous bureaucratic requirements. An industry will spring up that is specialized in getting as much grant money as possible.
When it's common practice to slip the town clerk a $50 to get your deck permit pulled in less than a month, then we can start to conclude that maybe government's corrupt.
The government is systemically corrupt because it is not doing what we the people authorized it to do. It is not protecting everyone's basic rights equally. It is favoring particular special interests (which include politicians themselves--Congress has single digit approval ratings but incumbent reelection rates in the nineties).
This kind of corruption does not involve obvious violations of the letter of the law like overt bribery. But it's still corruption--and indeed it's worse than simple violations like bribery because it involves the entire system, using a process that was intended to do one thing to actually do other things that go against that one thing.
The problem here though isn't lack of money. Linux has lots of money. Adding more money from governments won't change anything. The article says there are hundreds of people paid to hack on the kernel. The problem is that the kernel development culture (sorry, it has to be said) is still a 90s hacker culture, and the people who get employed to work on it are hackers. Hackers like writing code, and dislike writing documentation or unit tests or fiddling with CI systems. The fact the kernel is written in C doesn't help (makes common testing techniques like dependency injection more painful).
The idea that it's companies to blame for lack of docs is rooted in this culture. The people who understand what the docs should say are the ones writing the code to begin with. How will these technical writers know what to write, exactly? Usually in a company they'll do that by interviewing the people with the knowledge, who are mandated to attend the interviews by their managers. In the open source context you can't do that so this boils down to "please hire reverse engineers at vast expense to work out things I already know but can't be bothered communicating" which is a terrible proposition. Who exactly are these unicorn reverse engineers who are skilled enough to work out kernel code but who want to spend all day writing docs? Where will you find them?
Likewise the lack of testing infrastructure. The idea that those terrible freeloading corporations just aren't helping is wrong - they have tried repeatedly and the kernel community has ignored them.
https://kselftest.wiki.kernel.org/
"Kselftest is run everyday on several Linux kernel trees on the 0-Day and Linaro Test Farm and other Linux kernel integration test rings."
https://foundation.kernelci.org/blog/
"The first KernelCI test development and coverage hackfest took place from 27th May to 4th June 2021. For a total of seven days, developers from the KernelCI team, Google, and Collabora worked to improve many different aspects of KernelCI testing capabilities."
If Linus and his subsystem maintainers decided to stop merging patches until they collectively had a good CI setup, money and staff would appear immediately. They have before, even when there was no commitment to do that. But in the early 90s people didn't really write unit tests or run CI systems, and so nor does the kernel today.
Don’t know when the Americans became anti-government but it’s been a while in the making. Hell, rabid anti-communism famously took root in the 50’s
On a different note, I think it's true that the Linux desktop still sucks in 2024. At this point I would be willing to pay a substantial amount for stuff to be clean, consistent, and "just work" on the desktop. Doubt we will ever get there, but maybe companies like Valve will clean it up enough.
I don't think Valve will fix it TBH. They only care about Linux UX on their hardware, i.e. Steam Deck. I don't see why they would fund things like KDE, or features in the kernel beyond the ones that they need for their products to work. (Not that I think they're assholes for this; just that it's unfortunate)
You just invented *BSD.
FWIW, I have never needed to dig into Windows' help system or Google to figure out how to use Windows Settings or the old Control Panel.
GNU has this random assemblage of packages:
https://www.gnu.org/software/software.html
So add the Hurd to that, along with an actual distribution mechanism (not a trivial task), and what you get is something like a BSD.
But a free software one? No - that's the point of this article: the free software desktop users aren't willing to put up the resources it would take to develop what you're seeking.
They definitely could have done it differently and in cheaper ways (just look at every embedded platform).
As for KDE, Steam Deck runs KDE.
In my experience, most Linux desktop environments are leaps and bounds ahead of Windows here. What specifically do you want fixed?
It doesn't really sucks. Instead, I'm happy I have no longer to put up with the growing number of Windows annoyances. The balance is definitely positive.
That said, there are a number of issues that I've found. Some of them are solvable via configuration, some of them are not. I'll omit a couple of bugs I submitted to KDE that they say are corrected in the new version. Unfortunately, it seems I'll have to wait a long time until that version is published and then reaches Debian stable.
- KDE look and feel customization is badly scattered in multiple menus. Some options are not accessible and need to be changed in scripts written in a KDE-specific language. There are a couple of very bad UX in the last category.
- Firefox, Thunderbird and other applications use completely different file dialogs with different conventions that I haven't found how to configure. Firefox is more difficult to configure than it seems. I started caring because it had tiny scrool bars. For some appearance configurations I had to look at Firefox source code.
- My Behringer external sound interface doesn't work with the default software. It works with Jack, but I've been unable to configure JackD to start automatically, so I need to start it manually every time I reboot. Mitigated because I always hibernate.
- Drag & drop works in a very weird fashion. When I drop something, a popup menu appears asking me if I want to copy, move or link the dragged item. There's no way to configure KDE to just move. I need to press shift before I start dragging and that conflicts with multiple selection... no way to move multiple files without the shitty menu.
What I found sad was the response to users' request in forums. Some users volunteered solutions and they were told that they won't be included because the fixes needed to be included in some config dialog, documented, translated to all languages... and noone will have time to do all that.
One thing that I had not thought about was the increasing complexity of code bases. I looked what I needed to compile a browser, not sure if it was Firefox or Chromium, 500 GB required. Firefox also needs Rust now.
I'm worried that the OSS projects have no enough volunteers and the barrier-to-entry is high enough to discourage more people helping.
Years and years of Windows and Linux taught me to avoid drag & drop, mostly, except for dragging files around in file managers (and even then, some of the Linux ones…). I learned that it would, too often to be relied upon, do something I didn’t want, cause a program crash (Linux is especially bad about this), put the program in some weird broken state that falls short of crashing, or simply not work.
It took years to un-learn this avoidance of that interaction, after switching to Mac.
Now when I occasionally poke my head into Linux-land again, I don’t have that instinctive avoidance of d&d, and so I try it a lot more. Bugs and bad behavior abound. Anyone wants to see Linux desktop be bad at some things, go hard on drag & drop for a couple days, you’ll see it.
This workflow it's being terrible in Linux, worsened with the lack of a directory tree pane in the file manager.
Dolphin mixes directories with files in the tree. That would seem to bring the same functionality, but no. I've tried to get used to one-click to open, but it opens the door to all kind of unwanted behaviour when combined with the D&D. I need to undust the Air to remember how it worked there. Sometimes I just want to make sure I'm only selecting one file and I can't.
Anyway I haven't found the funcionality unstable. Just unwieldy.
See https://wiki.archlinux.org/title/Firefox#XDG_Desktop_Portal_... for Firefox, you have to both install the relevant portal (e.g. xdg-desktop-portal-kde) and enable its use in Firefox (as the wiki page explains). While these instructions are for Arch, I assume something similar exists in Debian.
> I started caring because it had tiny scrool bars.
Does it have tiny scrollbars on Windows? I agree that they suck, though I've learned to live with it by now...
> I looked what I needed to compile a browser, not sure if it was Firefox or Chromium, 500 GB required. Firefox also needs Rust now.
Probably Chromium? I've compiled Firefox in the past and don't recall needing 500GB, though Rust was indeed required.
Does it have tiny scrollbars on Windows? I agree that they suck, though I've learned to live with it by now...
Try these "about:config" settings:
- widget.gtk.overlay-scrollbars.enabled -> false
- widget.non-native-theme.gtk.scrollbar.round-thumb -> false
- widget.non-native-theme.gtk.scrollbar.thumb-size -> 0.9
- widget.non-native-theme.scrollbar.size.override -> 18
Then I wanted to match bars (and some menus) colors with my theme so I found these:
- ui.themedScrollbar
- ui.themedScrollbarInactive
- ui.themedScrollbarThumb
- ui.themedScrollbarThumbHover
- ui.themedScrollbarThumbActive
- ui.themedScrollbarThumbInactive
- ui.menu
Colors are in the usual HTML #rgb format.
That would be GTK.
I hate having multiple conflicting ways to do the same thing. Even within something like Ubuntu I can think of eight ways I've installed software.
Steam regularly fails to open, and games compiled for Linux, not running through proton, regularly crash.
Hardware features randomly don't work. My laptop won't go to sleep when closed and runs full power while in my backpack. My extra mouse buttons that could be bound in windows without an issue didn't work on Linux.
Everything about Linux irritates me these days. I don't want to use other OSes for ideological reasons, but my problems have only increased since moving to Linux on my personal machine.
That said, I wonder if some of the pain you are feeling is self-imposed. If you just use a single linux distribution, it is generally consistent in its install methods. Wouldn't you find it weird if someone complained about how the MacOS install process differed from Windows? That's roughly the equivalent of what you are saying.
The power management issue is a real problem, and it plagues not just linux, but the PC ecosystem (x86 / intel,amd) in general. MacOS gets around this by tightly controlling their hardware and software to create a great experience. It's harder on the PC side due to the sheer number of configurations of hardware and software that have to cooperate to accomplish good power management.
This is the same root cause w/ 'hardware feature' support. A PC is a collection of independently designed hardware and software, smooshed together into some sort of computer. Making that work is difficult due to the sheer number of combinations that are possible.
My feeling is that this situation is ripe for solution by a company that treats their software and hardware as a complete system, and to deeply attack these challenges by curating (and eventually designing, if needed) components and software to polish the user's experience.
1. apt 2. .deb files 3. Ubuntu Software center 4. tar.gz 5. curl > sh 6. Docker 7. Flatpack 8. Snap 9. Compile it myself
On Windows it's usually just an installer. On Mac you either you brew, or drag a binary file into your applications directory.
Power and hardware is absolutely worse on Linux. I still dual boot on all my machines, so I can make direct comparisons on the same hardware. This isn't just an x86 problem.
I agree that we desperately need a solution.
Which alternative exists today that comes even close to this described utopian product?
Linus gets paid $1,5 million (from the Linux foundation).
I think given the central position and importance of Linus compared to the head of the Mozilla foundation seems a bit off.
A little while ago Linus announced that they had lost power at their home due to a large storm.
I was picturing large corporations sending in giant Tesla batteries and diesel generators and hooking it all up in hours. (free of charge of course). but as far as I know it did not happen.
It might have if he asked for it I suppose.
Linus seems like a most humble guy, given his celebrity status, his life's work and his high importance to the world.
(I do see things that he has several other income streams than just the Linux Foundation so that s good)
That's pretty insane. I used to view Mozilla as "good guys" but it seems they are more like a lot of other non profits that are run mainly for the benefit of their executives.
LLVM also deprioritizes general cleanup work, such as getting rid off passes that don’t work, and are rotting in the tree: of the top of my head, I can think of GVNSink, LoopFusion.
There are additional problems unique to LLVM, as it doesn’t have a dictator: there are multiple different dependence analysis in tree, for instance.
As it is an open source project, in an ideal world, it would e fair to assume they would somehow contribute back, but instead they just take it for granted as there is a group of people who make it available for free.
(please read to the end before posting your "wHy pUt iT oPeN-sOuRcE tHeN???" bigoted comments)
Don't get me wrong: It is perfectly legal for anyone to do so, as the project is available under a permissive license and you are not mandated to do anything to contribute back.
I'm just trying to clarify the point made by OP here, and clarifying that some infrastructure tooling such as compilers and debugger are often taken for granted, and even rich companies opt not to contribute back.
I just looked at 1 page of the most recent commits to the LLVM project and every single one of them is from a corporate developer. I can't imagine how to arrive at the conclusion that the industry takes LLVM for granted. The major LLVM subprojects that I follow are not just largely written and maintained by large companies, but were also initially invented and contributed by them.
Now compare that with the corporate user base…
Software in the modern economy is the "means of production" and software developers have given it away free to the business/investor class.
The investor class, happily funds software startup gambles that people pour their lives into knowing that most will fail but they will still statistically make bank.
Companies making billions of profits lay off people who have made massive contributions of open source ( see for example Ben Collins-Sussman https://social.clawhammer.net/blog/posts/2024-01-10-GoogleEx...) knowing that the software will still maintained by volunteers without them having to pay for it.
You are likely an excellent software engineer. Why would a company not have hired you to work on non-foss software?
Plus, let's not pretend most of us were not pirating all the paid stuff. Is that what we want our industry to be based on?
What's more, FOSS is so much more collaborative the docs, tutorial and communities made me grow so much faster than anything else professionally.
It's a positive sum good for everyone, not some adversarial zero-sum game.
It's so good that we need to be doing a lot more of it, and (I think we'd all agree) figure out more ways to allocate capital to FOSS.
The profit of the big corps almost never results from the FOSS itself.
When you make code libraries or other "infrastructure" software it's important to be as stable as possible so your users don't have to constantly maintain compatibility. I suspect if it was easier to upgrade RH would have kept it KDE around. Same can be said for some of the Gnome updates too, and other software as well. "Maintenance" should be minimized particularly when people are working for free.
First, I've taken to looking at the economics as shared infrastructure costs - companies look at the volunteer built pot and make a self-interested decision on participating.
Second, this observation - "commoditize your complements" - seems still as apposite as when it was published in 2002: https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
https://autkin.net/audiobooks/things-nobody-wants-to-pay-for... (8:13)
Put differently, Buffett could personally fund 10k open source contributors 200k per year for their lifetime (60 years) if he wanted to.
So maybe that's the solution? Convince billionaires that open source software is a philanthropic endeavor?
I'm curious how he earns so much? That seems like a lot for someone who is associated with FOSS, even if they are so influential/skilled.
Source?
Is it, though? This isn't the attitude we have towards things like USB-IF or the Unicode Consortium, both of which are non-profits that were started by for-profit companies trying to solve a technical challenge for their own, joint benefit.
Hardware and software companies need a working operating system to be able to make money.
Linux could get dramatically more resources if they could just drop their big-ego'd "CLI is god" mentality. Yes, Linux is a powerhouse OS, but it only appeals to a tiny slice of the population.
It looks like one can donate towards the LF as a whole [1], but the LF manages a ton of other projects besides Linux. So if was possible to restrict small donations towards the development of the Linux kernel, that would be great.
My state promised that money from the lottery would go to education rather than the general fund. It does, but that doesn't mean education got more money when they started the lottery.
It might also not look good on them if they used the unrestricted donations in a very unbalanced way to counter restricted donations the way you describe, so they might get less unrestricted donations in the future.
There are no guarantees that anyone is interested, but it seems hard to tell whether that's true in advance.
I know a certain git that might disagree.
All the tech billionaires and their ex-wives throwing money at cool and fashionable causes should redirect a small part of it to the industry that made them rich.
Me: Annotate main scheduler in linux kernel version 6 for interactive workload on multiple cores.
Bard: I'm unable to directly annotate kernel code, but I can provide a detailed explanation of the main scheduler's components and mechanisms for interactive workloads on multiple cores in Linux kernel version 6, incorporating images where appropriate:
Key Components:
Completely Fair Scheduler (CFS):
Primary scheduling class for interactive tasks
Aims for fairness and responsiveness
Employs a red-black tree (image below) to organize processes based on their virtual runtime (vruntime)
Processes with lower vruntime are given priority
Зображення: redblack tree data structureВідкриється в новому вікні
www.eecs.umich.edu
redblack tree data structure
Per-CPU runqueues:
Each CPU core maintains its own runqueue
Stores processes ready to run on that core
Reduces contention and improves cache locality
Load balancing:
Mechanisms to distribute processes across cores for optimal performance
Triggered when a core becomes idle or imbalanced
Migrates processes from overloaded cores to underloaded ones
Process Creation:
New processes are assigned to a suitable scheduling class (CFS for interactive)
Added to a runqueue, initially on the core where they were created
Process Wakeup:
When a process becomes ready to run:
Added to the runqueue of the core where it last ran or a suitable core (load balancing)
Process Selection:
CFS periodically selects a process from the runqueue
Chooses the one with the lowest vruntime (most deprived of CPU time)
Context Switch:
The scheduler saves the state of the running process
Restores the state of the selected process to resume execution
Per-CPU runqueues:
Minimize cache-line bouncing and inter-core communication
Improve performance on multi-core systems
Load balancing:
Ensures efficient use of all available CPU cores
Prevents cores from becoming idle while others are overloaded
Scalability:
CFS design scales well with increasing core counts
Maintains fairness and responsiveness in large systems
Scheduler domains:
Hierarchical structures for managing CPU resources across multiple cores or NUMA nodes
Allow for fine-grained control over scheduling policies
Preemption:
CFS supports preemption to prevent processes from monopolizing the CPU
Ensures interactive tasks remain responsive
Linux kernel version 6 is under development, so specific details may change.
The scheduler is a complex component with various optimizations and tuning parameters.
Understanding its behavior requires deeper exploration of the code and experimentation.