Hacker News Reader

Comment by rifung | Hacker News Reader

Why machine learning? The paper says that the need to handle the complexities of modern hardware made it so that the scheduler failed to do its fundamental job.

Machine learning sounds like it would add even more complexity but perhaps I just fail to see why machine learning is a good idea here. I don't see how you can predict workloads based off historical data and if you're going to try to predict workloads based off binaries, the overhead would likely outweigh any benefits you would get.

I'll admit I am no expert in machine learning but I have a hard time understanding why you would look at this problem and think machine learning is the solution.

View on HN

The only reason is because machine learning is fashionable at the moment and a lot of people like to suggest it as the panacea. I can't even imagine the terrible overhead that machine learning would impose in probably the most critical part of the OS.

Prediction can be extremely fast.

But learning is always slow. What is the point of machine learning if you disable learning?

Learning doesn't need to be done in realtime... it can happen async.

Go for it, historically there have been some simple Linux schedulers. It's not difficult to hack on.

There was a "genetic algorithm" patch years back, it basically identified busier processes and preferred to give them cycles. It kind of sucked as less busy processes would starve. We are at an interesting place now, we have a good scheduler with o(1) semantics and it's fair, but it's leaving cycles on the floor on certain hardware with certain workloads.

I would think a machine learning approach would be more expensive and it could potentially be difficult to explain why it was doing a certain thing.

There is a long held belief that a good Linux scheduler shouldn't have tuning knobs and we don't need to select the scheduler at build time. I could see that belief coming to an end as we run on smart phones up to supercomputers with performance and energy being key issues on both ends of the spectrum. Some of the more exotic heterogenous hardware is becoming very popular too and that may beg the question more.

According to this paper, they dropped the O(1) scheduler a few years ago for a CFS scheduler that's O(log n) but exhibits nicer behaviors in other ways.

"I don't see how you can predict workloads off of historical data"

... Think about that part, then:)

Reply on news.ycombinator.com