Comment by FabHK | Hacker News Reader

Comment by FabHK | Hacker News Reader

Very neat. They write:

> In this course, we’ll be looking for the following trifecta: (i) ideas that are non-obvious, even to the well-trained computer scientist, so that we’re not wasting your time; (ii) conceptually simple — realistically, these are the only ideas that you might remember a year or more from now, when you’re a start-up founder, senior software engineer, PhD student, etc. (iii) fundamental, meaning that there is some chance that the idea will prove useful to you in the future.

In the first lesson, they discuss consistent hashing, and they seem to have achieved their goals.

ETA: Must include this anecdote on the history of the algorithm:

1. 1997: The implementation of consistent hashing given in this lecture first appeared in a research paper in STOC (“Symposium on the Theory of Computing”) [...] Ironically, the paper had previously been rejected from a theoretical computer science conference because at least one reviewer felt that “it had no hope of being practical.”

2. 1998: Akamai is founded.

3. March 31, 1999: A trailer for “Star Wars: The Phantom Menace” is released online, with Apple the exclusive official distributor. apple.com goes down almost immediately due to the overwhelming number of download requests. For a good part of the day, the only place to watch (an unauthorized copy?) of the trailer is via Akamai’s Web caches. This put Akamai on the map.

4. April 1, 1999: Steve Jobs, having noticed Akamai’s performance the day before, calls Akamai’s President Paul Sagan to talk. Sagan hangs up on Jobs, thinking it’s an April Fool’s prank by one of the co-founders, Danny Lewin or Tom Leighton.

Here’s one explanation of Consistent Hashing by a popular author that just started a systems design YouTube channel:

https://m.youtube.com/watch?v=UF9Iqmg94tk

I love this channel, I've been watching their videos for a few weeks. The guy has a great ability to break things down and explain complex topics with ease.

I'm often fascinated by the fact that, 80% of mentors failed at simplifying in their explanation of concepts.

Haven’t looked at his books yet, but they appear to be very popular:

https://www.amazon.com/Alex-Xu/e/B08BNMFT7P/

He also has a cheap online course ($60 a year) that covers everything that’s in the books:

https://bytebytego.com/pricing

* note: not affiliated with him, YouTube suggested his videos and I looked into him.

I don’t understand the connection between 1 and 2.

What happened to the relationship between Apple and Akamai?

Sorry, I left something out. The authors mention earlier in the lecture that

> [The algorithm (Consistent Hashing)] has real applications [and] gave birth to Akamai, which to this day is a major player in the Internet. [...] (Quantitatively, Akamai serves 10-30% of all internet traffic, and has a market cap ≈ $19B.)

This makes the rejection notice so funny.

The rejection notice is funny now, but to researchers whose careers depend on publications, such unfounded rejections are anything but.

Consistent hashing is needed for cashing/CDN.

> (i) ideas that are non-obvious, even to the well-trained computer scientist

> In the first lesson, they discuss consistent hashing, and they seem to have achieved their goals.

I was really excited when I read your comment here before clicking the link, but having had a look at the rest of the curriculum, I'm slightly underwhelmed.

- Generalization ... Empirical risk minimization.

- lossy compression

- Similarity Search. (Dis)similarity metrics: Jaccard, Euclidean, Lp....

- Regularization. The polynomial embedding and random projection, L2 regularization, and L1 regularization as a computationally tractable surrogate for L0 regularization.

- Understanding Principal Component Analysis (PCA). ... The simple geometry of "diagonals in disguise." The power iteration algorithm.

- Low-rank matrix approximations. The singular value decomposition (SVD), applications to matrix compression, de-noising, and matrix completion (i.e. recovering missing entries).

- Graphs as matrices and the Laplacian of a graph. Interpretations of the largest and smallest eigenvectors/eigenvalues of the Laplacian. Spectral embeddings. Interpretations of the second eigenvalue

- Markov Chains, stationary distributions. Markov Chain Monte Carlo (MCMC)

- Fourier methods

- Compressive sensing

- Linear and convex programming. Matrix completion

- Differential privacy

I was thinking that the vast majority of those topics should be pretty much standard knowledge for a mathematically trained computer scientist (not being an expert in each of them, but knowing the basics of what they do, how they work and where they're applied). What level is this course at?

Guessing from the number (168), probably second year? So not too basic if that's what you're implying

Ok, for a second-year course that's pretty impressive. I don't know why, I was expecting a postgrad-level course

The Stanford CS curriculum only distinguishes on the first digit: 1XX for undergrads and 2XX and above for graduate students. (Undergrads often take one or more graduate courses as they progress.)

Most taking this course are advanced 2nd year or 3rd or 4th year students who have taken the introduction to algorithms course. Most/many CS majors never take this class at all.

> I was thinking that the vast majority of those topics should be pretty much standard knowledge for a mathematically trained computer scientist

Well yes, this is literally that mathematical training.

yeah, I guess I was expecting postgrad-level stuff on things that are curious to people who finished a CS education

We like to keep abreast of undergrad classes here, it's useful for knowing what incoming software engineers have been learning as well as to fill any gaps in our own education.

fair enough. also useful for (both sides of) job interviews, I guess. do you keep up to date about relevant research as well? I see a lot of Stable Diffusion posts, but that's arguably not because of the paper but because the published model is hackable. I guess keeping abreast of the papers leading up to Stable Diffusion would have been more important if you wanted to be the entrepreneur who created the tool?

I doubt even a bare majority of HN readers have finished a CS education.

An informal CS education counts.

No disrespect to the parent, but I would strongly discourage any aspiring computer scientist or software engineer from concluding that the above list of topics is in any way remedial or “lower division”.

Google exists in no small part because just over 20 years ago people who could also code and raise money connected just a few of the dots on the deep themes connecting those topics: https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.p...

Just over a decade ago Netflix paid out a million bucks (that was real money then) for a modest improvement over their existing recommender based on an SVD-like gradient descent for a low-rank approximate factorization: https://en.m.wikipedia.org/wiki/Netflix_Prize. Many of the competing teams had exemplary academic and industrial pedigree.

Many (if not most) ML practitioners would profit from thinking more about the information-theoretic links between lossy compression, regularization via norm, and channel capacity to this day.

If I correctly gather that this is a syllabus for elite undergraduates, they’ll be grading on a curve.

fully agree. I remember at one point I studied the Perron-Frobenius theorem because I was analyzing a problem in a different domain but mathematically not completely dissimilar to PageRank, and it was crazy to think that a trillion-dollar company had come out of that bit of math.

> Many (if not most) ML practitioners would profit from thinking more about the information-theoretic links between lossy compression, regularization via norm, and channel capacity to this day.

yes. don't forget compressed sensing there too

All those topics are unimpressive? I’ve spent quite a bit of time trying to teach myself them.

I’ve seen several Stanford courses (e.g CS229) so the low course number of 168 would seem to indicate that it’s taken within the first couple years of college.

At Stanford, 100 level courses are intended for undergraduates while 200 level courses are intended for mixed undergrad/grad. 100 doesn't necessarily indicate freshman/sophomore.

they are not; this is classic hn posturing

at the veryleast this would belong to a second or third class of algorithms for ug( after the usual meat and potatoes: big,small o notation, searching and sorting, dp, greedy algorithms and so on) depending on the difficulty of the assignments this could be easily a graduate course

remember that no matter how difficult, obscure or new a technical field is you will have people here claiming they took a class about it as a freshman 10 years ago and they wrote some papers about it. sometimes it may be true, but most of the time it isnt

The course description states that this is intended for senior as a capstone course, hopefully that help clarifies.

I am a mathematically trained computer scientist (by qualification, not by current practice) who has kept up an interest in both fields for many many years, and I don’t know most of the stuff in that list. I’ll happily accept that I’m just not all that good, but I offer myself as evidence that there might be a market for this course ;)

> I was thinking that the vast majority of those topics should be pretty much standard knowledge for a mathematically trained computer scientist

spoilers: this course is intended to mathematically train computer scientists.

Reply on news.ycombinator.com