Stanford class on Information Theory | Hacker News Reader

Stanford class on Information Theory

infotheory-class.org

132 pointshuherto30 comments

I noticed that this one starts in March 2012. Since the others start in Jan and seem to last about 10 weeks (AFAIK), they'll probably be complete before this one starts.

Keep that in mind if you've already signed up for others.

This is actually a super useful note, thanks. I saw the headline and thought "another interesting course that starts in January, no way" since the others all start in Jan. Upvote!

the Lean Launchpad course is the only one I've seen so far that says it's starting in February.

The SAAS course also starts in February.

Is it just me that doesn't like these stanford offerings? I find it very, 'to the point' kind of teaching, AI and ML. I prefer to watch Profs teach to a live audience. They normally fill a sense of energy in live classes. The style of teaching in lumps of 5-6 mins on youtube is very boring. And also the assignments here lack a serious effort compared to their original offerings.

I started my learning way back with http://www.cs50.net. I thought it would be great watching ai-class.org, but it isn't. Anyway, I found a very good AI course from UC-Berkeley (CS188). It's a streaming of live class sessions and is very interesting. And the assignments are top-notch as they are original class room psets.

It would be great if the videos are sessions to live audience.

Here are the classroom lectures for ML.

http://classx.stanford.edu/ClassX/system/users/web/pg/view_s...

Also, the classroom lectures for the upcoming algorithms course are already up.

http://openclassroom.stanford.edu/MainFolder/CoursePage.php?...

Thank you !Is there the original AI class too?

Thanks a lot for the link. Good way to review and compare with online class.

I didn't like Ng's course on ML too much, it is very good if you just want to implement the algorithms, but it gives no insight on why they work.

I enjoyed far more Tom Mitchell's lectures, they gave me a much better understanding of the algorithms described:

http://www.cs.cmu.edu/~tom/10701_sp11/lectures.shtml

There are some comments on HN about it

http://news.ycombinator.com/item?id=3199718

great! Thanks.

I am actually already following ML from Harvard. http://www.seas.harvard.edu/courses/cs181/extension/

Seems these videos are also great.

This is a great tip. Actually, I was kind of hoping that Ng's course would be focused more on implementation and less on the why, not that the theory is not important, it's just that I have found a lot of other great places on the web to get the theory for free (like the lectures you mentioned).

Really? I found descriptions of Linear and Logistical Regression, as well as the descriptions of Support Vector Machines to both explain very well why they work. The Neural Networks description could have been a bit more thorough, but I still understand them much better now than I did before.

Personally I find that Professor Ng manages to instill a lot of energy into the ml-class.org lecture videos.

I partially agree. Complete live sessions are easier to watch when the professor is good and prepared. At the same time, Stanford's format had definite advantages in that it's natively interactive and indexed. If I want to review a concept, I can simply click a link to the appropriate part of the lecture. They also have better sound and image quality.

The ML class is good when watched at 1.5x.

I think the rationale is that - at least as Prof Ng articulated in another ML course he taught this quarter - videos following a lecturer as he teaches on stage are actually _less effective_ than videos which just follow slides w/ audio narration. Another researched case was video lectures with the teacher's face in the corner vs. video lectures with only narration. In which they found that student's information retention was even better without _any_ on-screen presence at all. (Surprising, I know . . . I wish I had a citation, but I'm really just coming in on defense of the department's pedagogical choices here.)

It would be interesting if Stanford suggested some path through these courses.

I'm in the middle of AI class (loving it), and I wonder what the overlap is with the PGM or game theory classes - would they have been better to have done first (given a time machine)?

And I wonder, how much more than the probability I've picked up from the ai-class is required as a prerequisite for the infotheory class?

I am taking AI and ML. I would like to take game theory or pgm. But, I think I will start with analysis and design of algorithms. In my mind that should help me take the other classes later on. What do other people think?

Algorithms are core CS and I would definitely recommend taking them prior to other courses. The best way to know what to take first is to take a look at course prerequisites and the program tracks:

http://cs.stanford.edu/degrees/undergrad/ProgramSheets.shtml

I've sat in on the GT lectures at Stanford and I would definitely make sure I had a solid background in logic and statistics. Other than that, you can start from scratch there.

Its an extremely interesting course btw.

Is there some website that tries to keep track of these online classes across universities? Seems like it would be useful.

It would also be really nice for such a site to have a place for collecting tips, comments, feedback, like what I am reading here on HN.

I've learned a lot in just the past five minutes that will be useful for picking which (if any) courses to take in January.

It would be nice to see all these comments in one place, especially for people who don't read HN.

There is a Reddit group to discuss these classes:

http://www.reddit.com/r/OnlineEducation

The courses from different universities vary drastically in quality. I don't mean the professors are not good, I mean the online versions are often include incomplete materials and I haven't seen other classes that take advantage of the medium like Stanford: with integrated quizzes and automatically scored programming assignments.

I've seen slides from MIT's OCW that say "This slide is removed for copyright reasons" in half the slides of a particular lesson - they are just not helpful.

While we're on the subject, I found this a very good source of material on the topic, if you don't want to wait 'til March: http://www.cl.cam.ac.uk/teaching/0910/InfoTheory/

No video lectures, though..

http://www.cl.cam.ac.uk/teaching/1112/InfoTheory/ is a bit more recent and should be pretty similiar :)

I was hoping this would be led by Thomas Cover. I took his class back in Stanford many years ago, and found it to be one of the most enlightening.

He wrote the classic text on information theory -- a very well written textbook that actually keeps you interested: http://www.amazon.com/Elements-Information-Theory-Thomas-Cov...

Tsachy's an excellent researcher and an excellent teacher. A long time ago in a universe far away, we used to sit together taking classes (perhaps even Information Theory was one of them - I don't really remember)

I haven't seen him teach in 13 or 14 years now, but I'm sure he only got better at it, and highly recommend taking this course if you are interested in Information Theory.

(come back and downvote me in 4 months if you disagree)

Wow, it has started. Please please please more EE courses. I was begging for a signals & system course. This is close.

Reply on news.ycombinator.com