MIT 6.824 Distributed Systems labs rewritten in Go | Hacker News Reader

MIT 6.824 Distributed Systems labs rewritten in Go

pdos.csail.mit.edu

132 pointsmyko32 comments

I'm currently taking the class. C++ was used before. The main reasons the professor cited for using Go are that it has a very easy to use RPC library and that the labs are designed so that you will spend most of your time working on the distributed systems aspects of the problems, not dealing with the intricacies of the language or libraries used. Apparently he felt Go was better for that than C++, though he didn't spend any time comparing the two.

Another reason may be that it's easy to let students validate their work with the built in `go test`.

> Your job is to modify client.go, server.go, and common.go so that they pass the tests in test_test.go (which go test runs).

I wonder if it was would be easier (and legal) to just mock the functions so that tests pass, without writing any actual functionality.

I don't think you could do this without actually solving the labs the way we wanted you to.

I took 824 last Spring and the labs were often terrible because of the level of debugging necessary (but very informative and well made). In particular it was bad for those without much C++ experience like myself. I think the difficulty of testing and worrying more about small bugs rather than the concepts of distributed systems was a common complaint and it's awesome to see it being changed. This class was also taught by rtm the semester I took it.

"Taught by RTM" == "taught by read the manual"?

RTM = Robert Tappan Morris, http://en.wikipedia.org/wiki/Robert_Tappan_Morris

I took this class in 2008, in C++, and the RPC code definitely felt like a mess. I expect the Go version must be nicer, plus a fresh rewrite probably helps regardless of language.

Enjoy the course, it's a great one!

Having been educated only as an undergrad at a college ranked ~100, I have a question for those who've experienced a course like this at MIT or a similar institution. When I was reading the first lab's description I thought "man, that'd be tough to put out in a week, but it's doable." I saw that it appears to be a multi-week lab for graduate students and wanted to find out if this is normal difficulty for a course? Is there more homework outside of labs that's missing, or crazy quizes/tests that deal with much more theory or proof style work? I just recall my undergrad courses having those extra items, but in large part being the case that if you made all your labs pass all the test cases you could get an A in the course. Does this hold true even at the elite schools or is there much more hidden below the surface here?

The first lab is due Monday, six days after class began, so we were given about a week for it. The professor told us it was not supposed to be a very intensive lab, and its main purpose was to get us up to speed with Go and the scaffolding code that we are using. In addition to the labs and tests we are required to read a research paper for every class and write a paragraph answering a question about it, and we must also do a project at the end of the term. You can find examples of old tests at the OCW site http://ocw.mit.edu/courses/electrical-engineering-and-comput..., if you'd like to see what they are like.

I took this course in 2010 when it was in C++. In addition to reading papers, as mentioned below, there were one or two tests. I say this is one of the tougher CS classes, though probably my favorite.

I'm a CS major at Yale. Problem sets this difficult are assigned regularly with only a week or two to do them. Right now I have two due at once. It sucks.

The good news is that you can get an A on all the problem sets by just completing them to specifications. It's often unclear whether you will be able to make the deadline, but usually you manage. At least at Yale, I've found the tests to be pretty difficult as they are the differentiator in the courses.

I took this class in 2011 when it was still in C++ and as others have said, besides the weekly labs, it had significant amounts of reading, two quizzes and a final exam. This is the probably the class I have learned the most from.

having taken a couple of cs undergrad courses when i was a physics grad student, i can tell you that undergrads seem to get way more in the way of homework and assignments than grad students do (presumably because grad students need to keep up with their research, TA duties, etc.)

It is because grad school "classes" are a charade to keep up the pretense that an apprenticeship is a classroom education that merits tuition.

I'm the TA for the class. The first lab was intended to get students up to speed with go; the rest of the labs are significantly harder. That said, the labs are completely new, so we're not exactly sure how hard they will be for students. It's a process.

CMU's 15-440 Distributed Systems course did this over a year ago. See http://www.cs.cmu.edu/~dga/15-440/F12/

The professor, David Andersen, catalogued his experience on Google Plus: https://plus.google.com/115237092509505721130/posts/DKVcPLbq... https://plus.google.com/115237092509505721130/posts/AQqe7Xay... https://plus.google.com/115237092509505721130/posts/d2RE6gbj... https://plus.google.com/115237092509505721130/posts/3PFz3RUe...

At CMU, 15-440/640 Distributed Systems is also in Go. One professor does it exclusively in Go and the other gives you the option of Go, C, C++, or Java.

http://www.cs.cmu.edu/~dga/15-440/F12/

For those wondering where is Go mentioned: http://pdos.csail.mit.edu/6.824/schedule.html

Also here: http://pdos.csail.mit.edu/6.824/labs/lab-1.html

> You'll implement this lab (and all the labs) in Go.

I'm really glad to see this :-)

Go seems pretty good for things I might otherwise prototype in Python and then rewrite in C++ in order to get better static guarantees and performance. Does that sound fair?

CSP/Actor concurrency instead of exposing raw threads is a huge win. Focusing on interfaces instead of ancestry is also a huge win. However, it's really tough for me to get over Go's treatment of null and global mutable state.

It seems to me a bit of a waste to design a new statically typed language that intentionally allows the most common type error, as if Tony Hoare's Billion Dollar Mistake wasn't a mistake. One of the smartest things Bjarne Stroustrup did was disallow null in C++'s reference semantics.

It also seems a waste to give a language somewhat Erlang-like CSP/Actor-based concurrency, and then allow direct mutation of global state. This makes it much easier to introduce concurrency bugs, and also hurts garbage collector performance.

Maybe I just tried out Go too early. Does Go still use the Plan9 ABI and come with a copy of kenc so you can recompile your C source to be linkable against your Go code? Are the language creators still the only ones allowed to create generics?

Yes and yes. Though you don't need to manually do anything (dunno if you ever did), it still comes with the Plan 9 C compiler for bridging Go with C, and generics are still hardcoded.

Can this course be taken/audited online? My degree has a distributed course, but there isn't even a mention of anything like Paxos.

Many universities have DS courses based on the same syllabus and labs. I took this at NYU: http://news.cs.nyu.edu/~jinyang/fa12/ You can read the notes/papers and do the labs as instructed.

Where do you see Paxos? This looks like something simpler. It pretty much has to be as implementing Paxos would be too hard for a class exercise. I don't think it makes sense to do Paxos with 2 servers.

It appears that Paxos will be used in the third lab, http://pdos.csail.mit.edu/6.824/labs/lab-3.html.

In the 2010 version of this course, we implemented paxos on one of the later labs. It was considered one of the hardest I think.

How is Paxos hard? Inventing Paxos is hard. Implementing it is implementing a well-specified state machine.

If it was so easy papers like this would not exist: http://research.google.com/archive/paxos_made_live.html

All the lecture notes will be put online, and you are welcome to clone the code and do the labs. If your code passes the tests reliably, you would have gotten full credit.

Can we get more details? What did they use before? C++? Is there any code opensourced?

Check their site for 2012. It was C++. :-)

Reply on news.ycombinator.com