Hack the Kernel – Learn about operating systems online | Hacker News Reader

Hack the Kernel – Learn about operating systems online

ops-class.org

611 pointskrat0sprakhar55 comments

Thanks for sharing. I'm certainly bookmarking this page and revisit this course after wrapping up CMU's 213[0]. How does this compare to Udacity's "Introduction to Operating Systems?".

[0] If you are self studying, like me, CMU offers not only the lectures online, but the labs as well: https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.a...

The CMU class looks interesting but it seems the assignments are behind a login. A list of classes that have more open content - https://github.com/prakhar1989/awesome-courses#systems

I took 213 at CMU. Can't recommend it highly enough. A lot of other intro systems courses are based off that. It is a prerequisite for most classes moving forward at CMU. The textbook is also available online as a pdf (Google it). First 2 or 3 labs deal with bitwise, assembly (buf overflow attacks - bomb lab), then cache lab (matrix multiply performance), then signals, then implementing malloc, and finally using pthread API / threads to create a proxy server.

Actually you can find similar versions here. http://csapp.cs.cmu.edu/3e/labs.html

I think you can get self study versions of the assignments off the textbook web page

Sorry for asking dumb question, but after looking to their website I couldn't find lectures for Operating system course. Where could I find them? (I am talking about CMU course)

Does it need to login to watch lectures ?

Its way better than Udacity's intro to os course. Just read through all the assignment descriptions, and you'll get an idea of how much you're going to learn.

Wow, I've read about half the intro to computer systems textbook and had no idea there were lectures online. Very exciting. Thank you.

I've been going through some of these. These are great. Thanks for sharing.

I wasn't expecting much at first as an OS professional, but wow this is great! Flipping through the slides, it's just the right amount of info and context to develop a working knowledge and vocabulary without much time investment or getting overwhelmed or losing interest. Didn't look at the videos or assignments yet.

If you work in some type of ops role this will really accelerate your career.

If you are a programmer and forgot or did not take an OS internals class, ditto.

You will not unlock maximum throughput, minimum latency or a balance without understanding these concepts.

Thanks for posting this. I was looking for more operating system courses. I'm currently watching Kirk McKusick's FreeBSD Kernel Internals course [1], which isn't available for free (the first hour is [2]), but I thought was worth the money. I find is really amazing that I can watch a course taught by someone who is such an expert in UNIX-like systems.

Along with a general in-depth OS course, I would like to find a Linux-specific course. Does anyone know of a good Linux internals course?

[1] https://www.mckusick.com/courses/introdescrip.html

[2] https://www.youtube.com/watch?v=nwbqBdghh6E

The Linux Foundation has a bunch:

https://training.linuxfoundation.org/linux-courses

I find it hard to believe someone would downvote a recommendation for suggesting the Linux Foundation in response to a question asking if there were any good courses on Linux Kernel internals. The quality is excellent the price is inline with just about any 4 or 5 day vendor training class. I know of no other such class outside of a university setting.

If you had didn't have a positive experience then please share it.

Agreed. But its a bit expensive too.

Not a course, but I read nice things about this Linux Internals ebook: https://github.com/0xAX/linux-insides

I'm currently working on the Eudyptula Challenge, which has been interesting so far and is Linux-specific.

The creatively-named book Linux Kernel Development by Robert Love is also good.

UNIX-like kernels are easy, in that you don't have to have that much running before you can run "Hello World". I'd like to see more on microkernels. You have to have more pieces running before you get to "Hello World".

Also, the emphasis on virtual memory and page faults is becoming dated. Paging out to disk is obsolete technology. RAM is too cheap, and mobile devices don't page.

>UNIX-like kernels are easy, in that you don't have to have >that much running before you can run "Hello World".

Well, it depends on what you mean by "Hello World". Simply printing it out from the kernel? Easy. Getting a workable userspace up and running so a user program prints out "Hello World"? Hard. Also, you can have a microkernel that gives you a UNIX-like personality too (as is it is often the case when implemented), so I am not sure how one precludes the other.

>Also, the emphasis on virtual memory and page faults is >becoming dated. Paging out to disk is obsolete technology. >RAM is too cheap, and mobile devices don't page.

Uh, of course mobile devices use paging and virtual memory. You might be conflating virtual memory with swapping here.

Non CS-major here (in fact I did math). I don't know anything about kernels or micro-kernels. Easy cases, even grossly over-simplified cases would be relevant.

I know that every time I run a command into bash Linux is doing all sorts of fancy memory-allocation and queueing and more. All the books I am finding are either way too technical or grossly out of date.

Here is a cute course on Compilers: http://web.stanford.edu/class/cs143/

Actually they are not that out of date... Tanenbaum's Operating Systems and Computer Networks are both good. The BSD book by McKusick is excellent for more advanced students. For example, they don't cover SMP much, but locking and queuing, so you are prepared for studying SMP.

> Also, the emphasis on virtual memory and page faults is becoming dated. Paging out to disk is obsolete technology. RAM is too cheap, and mobile devices don't page.

I've got to admit, I thought of John Carmack advocating for pretty much the opposite direction for operating systems earlier this year, for making greater use of memory mapped files, unified virtual memory for CPU and GPU, and so on: https://twitter.com/ID_AA_Carmack/status/742191010013286400

I wonder what you meant by "mobile devices don't page."

> I wonder what you meant by "mobile devices don't page."

Mobile devices have memory-mapped pages, but they don't have VM or overcommit. Your process just gets killed if it's using too much memory.

Page faults are still important though… actually even more important.

Are we talking about the same thing when we say "mobile devices?" Like, Android phones? Because if there's something lobotomized about their virtual memory, that's something I'm curious about, if you could share a reference of some sort.

I don't know if Android swaps (almost certainly not) but iOS does not swap. It doesn't have a pager. Of course it does have "VM".

I dug into it a little. It's mostly for posterity at this point but if anyone wants to read about it, look up the Android lowmemory killer, which is sort of an enhancement to the OOM killer's functionality. It's a conventional enough Linux virtual memory otherwise.

He's a game developer, and struggles with the communications problems between the application and GPU. Remember that in Carmack's worldview, the game owns the entire display. It's not one window of many.

Let me quote Carmack on the issue of mmapped assets so other people can see whether you're truly presenting his world view or something you just made up.

On startup, applications would read-only mmap their entire asset file [..]

Splash screens and loading bars vanish. Everything is just THERE.

You could switch through a dozen rich media application with a gig of resources each, and come back to the first one without finding that it had been terminated to clear space for the others [..]

It will save power. Hopefully these are the magic words. Lots of data gets loaded and never used, and many applications get terminated unnecessarily to clear up GPU memory, forcing them to be reloaded from scratch.

There are some well founded reasons to be skeptical about this at first glance, until you realize that we are moving toward a world where hardware supports having CPUs and GPUs in the same address space (and the support is there on some mobile devices and HSA-supporting desktops). Expanding support for these traditional VM techniques to use the hardware to its fullest only makes sense.

Mobile devices don't page because killing a process on a device which is usually only showing the user one thing at a time, due to want of memory, isn't usually that big of a deal.

iOS does have VM though, and apps use it to implement home brew memory paging. Obviously not useful for the stack, so it encourages more explicit memory management. The mmaped region is bizarrely counted towards the total, leading to windowed memory schemes from the 1960s, where mmaps are remapped to access different chunks of data. Some poor design choices.

Also, app code is paged in, it just doesn't need to be paged out.

I have to say - the course is very well organised. For me atleast, the structure and flow is exactly how I approach my learning. A video overview -> a though provoking discussion -> the problems -> hints -> academic paper -> solution. Neat.

The professor who made this class is leaving UB next fall. I'd be lucky to be able to attend this one last course under him(the class is gonna be a full house this time) before he moves out of UB, a sad event in itself :(

While the course / content is itself good. The attitude of the professor is very questionable.

https://blue.cse.buffalo.edu/posts/2016-10-22-the-best-way-t... is a rant from him.

Quiet a few PhD students have quit from him and does not have a good reputation for being nice.

Please don't slander people without references from throwaway accounts.

Some long term HN habitués obviously (given the downvotes, though strong oscillations at the moment) think slandering people anonymously is okay. I'm curious why this would be. Is is because they think people would be penalised unfairly (e.g. by their institution) if they "told the truth" about someone? Wouldn't critism be more effective if you backed it with your identity?

Obviously when universities conduct class assessments they are able to do a census of actual participants. Here anonymity is sensible.

Your slander would be more acceptable if you had used content from a first hand experience, as it common on e.g. ratemyprofessor. Reading the (anonymous) statements about him online it does seem like he is a polarising figure, especially that the course works is a lot harder (in the sense of pushing peoples' capabilities) but many people rate him as one of their best lecturers... http://www.ratemyprofessors.com/ShowRatings.jsp?tid=1626392 https://www.quora.com/How-good-is-the-MS-in-Computer-Science...

Overall I come away with the impression that a whole bunch of students were totally unprepared for his class ("Need to know C", since when isn't C a prerequisite for operating systems???). The projects sound too big for the time available, something I remember well from undergraduate study.

How is it questionable, for writing an editorial? His attitude and demeanor in the videos I have watched seems to be one of someone who is enthusiastic and engaged. He congratulates some students in one for a milestone. He looks like he enjoys what he does.

Go bulls! I took Operating Systems with Kosar and worked on Pintos rather than the OS-161 modules. Honestly don't know what I was thinking, Challen's class seems far more interesting...

http://wiki.osdev.org/Main_Page has a LOT of great info on OS theory and practice as well.

Edit: added "as well." I don't want to give the impression that the OP is an inferior source.

I have taken this course and the assignments are fun and challenging. Learnt a lot.

Wow, this really great. Thank you for sharing.

I didn't any "about" link. Does anybody know who did this or what the affiliation is? In this Youtube link I see "Buffalo":

https://www.ops-class.org/courses/buffalo/CSE421_Spring2015/...

Is this SUNY/Buffalo State maybe?

The lecturer in lecture 1/38 is fantastic. Does anybody know who this person is?

University at Buffalo, SUNY. The person that teaches the course (who is the lectuerer) is Geoffrey Challen (https://blue.cse.buffalo.edu/people/gwa/)

Source: I took this class last semester (Spring 2016)

Thanks. Do you know if he made all this available then or is this something he did in conjunction with SUNY? The reason I ask it because of the domain and it simplicity of navigation.

This is godsend! Today I found some final notes and resources over the internet, wanting to put my C/C++ skill to good use and build and learn something. Always was attracted to OS (having OS class in a year but I couldn't wait), so thanks for sharing this so much! Can't wait to get started tonight!

its the best class i've ever taken. definitely worth solving the assignments!

Wow! Thank you for sharing. Learnt a lot.

This is really worth keeping bookmarked. Highly recommended.

That actually looks quite good! Can anyone who took this training vouch for it?

Yes. Mark Zuckerberg vouched for it. https://postimg.org/image/4b23cfczh/

ᕙ(`▿´)ᕗ thanks alot for sharing

"Ops?"

No, really, I'm asking.

In this context it probably means Operating Systems (i.e. low level programming)?

(But i've been seeing the same term used for like System Administration / "Support Services" / the I.T. Department in general, or something.)

Wow this looks pretty cool!

Do they have anything like this for learning how to transpile from lisp to other languages? =)

Not a course, but here's a nice blog post [0] on writing a lisp->JS transpiler (350 LOC) that was recently shared [1] on /r/haskell.

[0]: http://gilmi.xyz/post/2016/10/14/lisp-to-js

[1]: https://www.reddit.com/r/haskell/comments/57sg9q/building_a_...

Very nice. Look forward to using this.

Mark this, will check it later. Thx

Reply on news.ycombinator.com