Designing a New Rust Class at Stanford: Safety in Systems Programming | Hacker News Reader

Designing a New Rust Class at Stanford: Safety in Systems Programming

reberhardt.com

214 pointsksml52 comments

Hi HN, I spent last spring working on a new class at Stanford that's focused on common pitfalls in systems programming and how to avoid them. You can check out all the lecture materials (including recorded lecture videos) and assignments available here: http://cs110l.stanford.edu/

I'm planning on teaching this class again in the winter or spring and am looking for any feedback to improve it. I would love to hear your comments and suggestions!

This looks really cool! I especially love the assignments and I'd definitely take a crack at them. Thanks again!

Thanks for checking it out! I'd love to hear any feedback once you look through them

Thanks so much for sharing it.

I may be misunderstanding, but it appears that the code for the assignments is in a private GitHub repo that students of the class are invited to. Is there any other way to gain access?

I'm happy to email it to you! My email's at the bottom of the blog post. I'm also working on getting that starter code posted publicly.

EDIT: the code is now posted here: https://github.com/reberhardt7/cs110l-spr-2020-starter-code

Thank you! Finally going to take a stab at rust, glad to have some direction in that endeavor.

One aspect of systems programming that doesn't get enough attention involves tracing memory leaks (with valgrind, massif, etc). While it may be challenging for you to create realistic examples, doing so would help expose students to challenges they'll face in the real world.

That's a great suggestion! I'll try to design some exercises for the next time we teach this.

This is a step forward. I've been pushing for more language safety for what, decades, and it's good to see Stanford teaching it. It's important that students understand ownership concepts, because they exist in C, but the language doesn't give you any help in dealing with ownership. For years, I've been saying about C that the big questions are "how big is it?", "who deletes it?", and "who locks it?". Rust addresses all of those. So teaching Rust is useful to get students thinking in the right direction. Even if they get stuck maintaining old C code.

Looking ahead, new students are going to have to deal in future with huge numbers of CPU cores. So they need to deal with large numbers of threads. That's very hard to do well. This class addresses not just safety, but performance. That's good.

This is so far ahead of what I got from Stanford in 1985.

(I have some arguments with the Rust people, mostly around over-use of "unsafe" and too much language churn, but they're doing great work.)

Just FYI, Stanford also has a class on parallel programming (which I TA'd at one point). This goes into GPU and distributed programming as well as multi-core. Slides are online though I guess lecture recordings are not: http://cs149.stanford.edu

Great class, thanks for sharing the materials. Would you consider also making the code for the exercises available? I'm working through the slides but the exercises are hard to follow without the provided source code.

Yes! It's now available here: https://github.com/reberhardt7/cs110l-spr-2020-starter-code

Great work Ryan! Love to see macros get their own lecture.

- your fellow 110 TA (and 240 TA I guess)

Hi Divya! Thanks for reading!!

Is there any plan of releasing all the videos? Lectures 15-18 do not have videos linked.

Teaching a class about safety in systems code seems like a great idea, and at first glance the class content here seems useful and interesting. I do a lot of OS-level stuff at work (especially around virtualization and Firecracker), and while safety (and security) are obviously a critical topic for industrial systems, they don't tend to be things that new grads have thought about at all. Great to see that changing (even, as with all curriculum additions, it means covering some other stuff less).

> CS 110 [multiprocessing, multithreading, and networking in C++] is not just about how we do things, but also why – why are things designed the way they are, and if we get certain bugs or performance characteristics, why is that?

That's an interesting take, because I don't see Rust as being more abstracted in this way than C++ is. Obviously it's more abstracted than C, but by the time you get to "modern" C++ you're programming in a much higher-level language than C.

> I also think it’s hard for students to appreciate Rust without having first experienced C and C++.

This part does make sense. Explaining the "why can't we just write C really carefully?" piece to people who haven't experienced trying to do that is going to be harder. As we all know, it is possible to write safe C, but it takes a level of discipline and tooling support that is beyond most undergrads.

> that looks at what is often going wrong in systems and how we can improve practices to build better systems.

I'd love to see more research here too. There's some systematic studies of the causes of bugs in systems code, and obviously a lot of well-known bug patterns (see all of C's string handling). On the other hand, there seems to be fairly little research on the causes of more pernicious and subtle problems that become vulnerabilities (and data corruption, crashes, etc) in systems code.

I'm also biased here, but it seems like Firecracker (https://github.com/firecracker-microvm/firecracker) and rust-vmm (https://github.com/rust-vmm) would be great things to use as examples in a course like this. Not because it has space to explain the whole virtualization stack (it doesn't), but because things like virtualized devices are both fairly simple (at least at the simple end) and really safety critical.

> That's an interesting take, because I don't see Rust as being more abstracted in this way than C++ is. Obviously it's more abstracted than C, but by the time you get to "modern" C++ you're programming in a much higher-level language than C.

I agree, but C++ doesn't force you to write good, modern code, and so in lecture, we are better able to show examples of bad code that well-intentioned people might write and discuss the consequences. Also, you can't directly make syscalls in Rust without using "unsafe" or a library, and it's harder for students to see how programs interact with the kernel when that extra stuff is in the way.

I'm checking out Firecracker right now, and it looks like a really neat project! I'd love to talk more about virtualization in the class, especially since I feel that our current curriculum doesn't spend as much time preparing students for the modern cloud computing architectures they are bound to work with. Curious: can you think of any safety issues you've encountered in Rust or bizarre bugs that Rust's type system didn't save you from? Those often make great lecture examples, and while we always remind students that Rust is not a panacea, I don't think we had any good concrete examples.

I think another big benefit of Rust for systems is that you can explicitly mark functions/traits as unsafe if they require some preconditions to work correctly or if the trait must guarantee something that can't be checked by the compiler. That's a big benefit over C++ where the best you can do is documentation since the warnings in the documentation don't appear at the call site unlike how using an unsafe API requires an unsafe block/another unsafe function.

These comments struck me as particularly damning:

> As people usually say, Rust has a steep learning curve, and it’s really hard to get productive with it in a short amount of time. This is reflected in the 2019 Rust language survey, and it’s also reflected in the student frustration in the first few weeks of our weekly survey responses

> While I think Rust would be poorly motivated in CS 107, I think it is extra poorly suited for CS 110.

I have essentially no experience with Rust, and I want to like it. But I get the feeling that it isn't very user-friendly and that I would enjoy it much less than other memory safe languages like Swift, Go or Java or even sharper-edged languages like C++ with smart pointers or Objective-C with ARC.

Also given the large body of legacy C/C++ code still in use and development, I'm disappointed that clang still doesn't seem to support a memory-safe mode/ABI, as it could eliminate a large class of errors.

Maybe Unix made a critical error by adopting and promoting unsafe C; its predecessor Multics, written in PL/I, had essentially zero buffer, heap, or stack overflows over its entire lifetime (though it probably still had race conditions and concurrency errors.) ;-)

Compilers such as clang/LLVM and GCC do support a memory-safe mode (e.g. AddressSanitizer) which is not what you want because they add large overheads and thus are not intended for production use.

There's an ongoing effort to design extensions to C that allows efficient enforcement of memory safety, Checked C from Microsoft Research is an interesting and current effort in this direction.

For all the challenges of getting comfortable with Rust (which I 100% acknowledge), it does provide a very interesting tradeoff between performance, safety, and compatibility with C.

Asan and co are not just not intended/recommended for production use because of performance issues. They are designed as debugging aids, not security hardening features. As such they are likely to introduce security vulnerabilities, not prevent them (they effectively expose a larger attack area).

A couple of weeks of stifled productivity versus years of buggy, insecure, and inefficient software (e.g. because you'll be afraid to use asynchronous IO, threading, etc). I'll choose the former.

Rust is nice enough, but I'm not sure it's worth a university-level course on safe systems programming. There's a lot more of interest about ats[1]; f*[2] (specifically its 'low' subset); isabelle[3] (most notably used by the sel4 microkernel); et al.

1. http://www.ats-lang.org/

2. https://fstar-lang.org/

3. http://isabelle.in.tum.de/

This Rust course is a better intro to these topics for undergrads.

It serves the undergrads who want to learn languages popular in industry so they can get a job and hit the ground running, while teaching them how to make higher quality enterprise/startup apps.

And it's also a good intro to safe systems language design for the ones who want to make a research career in PLT, who will go on to study your list in more depth.

I don't see a focus on testing in this course. Was that by design? I feel like a lot of design out there forgets this important implementation detail.

No, it wasn't by design, but we just didn't have time to talk about it. I talked about testing extremely briefly in one lecture, but I think we will spend more time on this next time we teach the class.

Designing this class was hard because there's just so much stuff out there to talk about, and not enough time... Did you see any topic we covered that you think we could do without and talk about testing instead?

You could drop the OOP lecture and leave detailed notes on the course page instead. I'm not arguing for OOP v FP or anything like that, just that for either (and other) paradigms a specific testing approach might be better suited for this course. Talking about the complexity v. safety trade off might be better suited.

Unfortunately it seems Zoom failed to record Armin's voice in the first video, FYI.

Huh, that's a bummer... Not sure if there was a recording issue or if I made a mistake editing the video. Sorry about that :(

I wonder if either of the two teachers have actually worked on a large system before.

This comes across as really harsh. Did you mean to come across that way?

You are projecting your intent on the OP. I do not see anything positive or negative in their comment.

I don't think it's harsh. It is a reasonable question to ask. Academia is rife with people who have never worked in the industry, completely disconnected from reality.

From what I understand reading the course outline, this is an introductory class. Foundational material in a subject is often "completely disconnected" from the reality of the discipline. If I was an engineer taking a first course in integral calculus, should I worry that my teacher may not be a seasoned practitioner, or should I worry that they are good at getting me to understand the fundamentals? I can definitely see your argument if this was a professional practice class, or something vocational, but it's a 100 level CS class.

I think you should worry. One of my math professors taught us how factoring to a(x+b(x+c*(...))) is a great optimization in an introductory class, because he had no idea about CPU pipelines for example.

Horner’s method for polynomial evaluation is used because of numerical stability, not speed.

Could you elaborate?

I'm working on a project where the evaluation of a 4th degree polynomial is on the hot path. My micro benchmark (evaluating two polynomials + little bit of addition) shows that using simple Horner's rule is already somewhat faster, and implementing Horner's rule with fused-mul-add is more than twice as fast.

I would be very nervous about the integrity of a bridge designed by engineers whose sole source of knowledge was a teacher who has no experience in construction or design of bridges. That’s even considering the engineers may rely on industry standards bodies like AASHTO that prescribe much of the particulars of a design.

To my knowledge, no such standards bodies exist for software, making the designs of pure theorists all the more concerning. Maybe FRAMA-C? I’m not sure if that’s equivalent to AASHTO LRFD. Given that end users have almost no way of evaluating the safety or quality of software, similar to medical or legal counsel, I would think that there is an obvious need for some kind of regulatory body. Celebrity-bloggers seems like all too common a source of best practices.

One of them is in the thread here, why don't you ask them?

just because you have done something doesn't mean you can teach it and vice versa.

Just because somebody is a native speaker of English, doesn't mean they'd be good English teachers, true.

But somebody who only knows English from text books can't possibly teach it properly, no matter what.

As I say in the blog post, I'm just a grad student, but I have worked in many different places in industry, and from my own experience as well as what I know from others, there is a big need for better awareness of these kinds of issues. We wanted to start doing something to fill that need. I'm certainly not the most qualified person to teach this class, but that's why I wrote the blog post and am soliciting feedback here. Anyways, better to have something than nothing, right?

Morgan Freeman narration "While not always true, in this particular case, the answer to that question was most definitely yes".

Even if you hadn't any, industry experience isn't some magical pedagogical panacea. Much can be learned from individuals, even at the extremes of the academic-industrial spectrum.

This is a fine article but not a Show HN, so I've taken that out of the title. Please see https://news.ycombinator.com/showhn.html.

There's a quarter-length class with its assignments and video of the lectures included. Closer to a book, a thing-you-can-try, maybe.

I read that and wasn't sure if it counted, but I added "Show HN" because I uploaded all the course materials for people to "play with," and I'm mainly posting to get feedback from the community. Thanks for re-tagging it.

There's more to safety in system programming than "use Rust"..

What about Ada/Spark ? Fuzzing?

That's true, but we only have 10 weeks in a quarter, and this class isn't many units so there isn't much time in every week. In the blog post, I explain our rationale for focusing on Rust even though I'm aware there are many other things we could talk about. Primarily, I think Rust encourages good programming practices, and, by working through practical assignments, students are able to develop better habits and have a more cohesive experience than they would if we just gave them an assorted list of technologies to know. That being said, I do wish we had spent more time giving students a better awareness of what's out there and talking about useful tools such as fuzzers.

That's excellent, and Rust makes perfect sense here. But consider that OP has an important point (though not said very diplomatically): Ada/SPARK has decades of experience and research in safety, especially type safety, which has not received the same level of focus in Rust as memory safety has. In addition, Ada was the vehicle for many years (and still is in some places) for teaching systems safety and mission-critical programming.

You should consider familiarizing yourself with some of that Ada/SPARK research, experience, and pedagogy. The issues you're teaching aren't new, and there's a huge body of knowledge, experience, and anecdotes you may find useful or inspiration.

A couple ideas to get you started-- - "Safe and Secure Software" - https://www.adacore.com/uploads_gems/Ada_Safe_and_Secure_Boo...

- "Safe Dynamic Memory Management in Ada and SPARK." Quote from its first page: "As our main contribution, we show how to adapt the ideas underlying the safe pointers from permission-based languages like Rust or ParaSail, to safely restrict the use of pointers in more traditional imperative languages like Ada." - https://www.adacore.com/uploads/techPapers/Safe-Dynamic-Memo...

- The Ada Information Clearinghouse - https://www.adaic.org/advantages/

Rust is new and important and it's great that it's the focus of your course. But I also think you would do your students a service to show them that they can stand on the shoulders of giants in comp sci just as much as any other discipline. Much as the industry has moved back to what was fundamentally the IBM mainframe remote-system-and-virtual-machines service and licensing model [now we call them clouds, containers, and SaaS subscriptions], Rust is a relatively recent response to the same problems Ada/SPARK have decades of experience in handling. The lessons the industry and comp sci researchers learned 30 or 40 years ago do matter, and they can show us what kinds of solutions work and what may have have unforeseen effects.

The Ada Reference Manual and SPARK are under-appreciated tomes of the world's experience in these issues.

Thanks a lot for these helpful links! don't know much about Ada/SPARK but will spend some time going through these resources.

Reply on news.ycombinator.com