Hacker News Reader

Comment by 317070 | Hacker News Reader

The field of meta-learning is still very immature though. I can see why you would already want to start a scholarly discourse on the topic, but I am not sure how useful these techniques are for the students involved. They are still very ad hoc and often unprincipled.

This is a good article on the topic: https://arxiv.org/abs/1902.03477

View on HN

This is Stanford, they aren't teaching the next generation of applied ML practicioners -- they are teaching the next generation of theorists. This is a perfect class for that and getting their students ahead of everyone else. I'm jealous of them.

Assuming 330 is undergraduate level, they're teaching the full gambit of practitioners, theorists, and future drop outs.

300 series are advanced graduate classes. As an undergrad the sight of a 300 is horrifying.

It's always amusing how different institutions number their classes. At the Claremont Colleges, I took a class Math 103 - Fundamentals of Mathematics which was an upper division course geared towards preparing students for analysis, abstract algebra, etc. Because I started college having completed my math coursework through linear algebra and differential equations, this was the lowest-numbered math course on my transcript from Claremont and when I started grad school for a teaching credential a couple decades later, the program director thought that it was a remedial math course.

Recent Pomona College alum here checking in to say that the course numbering system has not changed (though Math 103 is now Intro to Combinatorics), and anecdotally it's still a point of confusion for those who go on to the grad schools the Colleges feed into.

I've never before paid course numbers too much mind, but it does surprise me there's not yet some widespread standard of to help graduate admissions officers, graduate advisors, and grad students themselves when determining prerequisite eligibilty.

Yeah, I was looking for the class to see what the number was and it doesn't appear to exist anymore. I remember being amused that at Mudd, Calculus as Math 1a/b back in my day. It appears to have been renumbered a bit higher since then and now they only offer one semester of calculus (back in the 80s it was radical that Mudd did the Calculus sequence in two rather than three semesters, although I noticed that our local high school offers a third-year high school calculus class covering multivariable calculus).

It's surprising to me - 300+ level courses were part of my undergrad required coursework - of course I wasn't studying at Stanford. Are course numbers standardized across academia or unique to an institution?

Course numbers are not standardized, although there are common numbering schemes. There are some uncommon ones such as MIT's, which uses a number and a dot instead of a subject name. And I've never known what institution the typical "CS 101" numbering scheme applies to.

Afaik they're unique—and not monotonously increasing in difficulty either. Here's Stanford's numbering system for reference: https://cs.stanford.edu/academics/courses

This class is available on SCPD, so it's open to (almost) anyone willing to pay >$5k.

Odd way to spell Philosophy Grad. To each their own I suppose.

Undergrad courses are 100-level, graduate courses are 200-level, advanced graduate courses are 300-level.

Another paper on the related topic of metric learning arguing that metric learning hasn't actually made any progress and is piggybacking on progress elsewhere: https://arxiv.org/abs/2003.08505

This is a pretty good paper, & they bring up many reasonable points, but I think it's important to distinguish deep metric learning from more traditionally formal ml methods for metric learning - there is plenty of progress being made in the context of scalable & provable metric learning algorithms that are robust to noise/corruption & missing data.

Recommend work & talks by Anna Gilbert for anyone interested. Entertaining & good at distilling technical content. Here is her most recent one, but there are other good ones on youtube. https://www.youtube.com/watch?v=Sb1ZhtsZjyM

>This is a good article on the topic: https://arxiv.org/abs/1902.03477

Not to nitpick but that article is a year old and the field is moving at lightspeed

There have not really been a lot of major breakthroughs in meta-learning in the last year, as far as I am aware. The paper is basically saying there was not a lot of progress in the 3 years before that either.

All in all, nobody really has a clue on how to do meta-learning right (or I am not aware of their work). There is progress being made on benchmarks, but some argue that progress is not really tackling the real issue at hand, i.e. learning to learn. Moreover, the current common benchmarks are not really good at untangling the progress in deep meta-learning from the progress in deep learning in general.

Isn't GPT-3 exactly the kind of meta learner you're thinking of?

I would say it is exactly the opposite. :)

It is showing how you can get drastically better at deep meta-learning by being better at deep learning. But it does not really show how you can be better at deep meta-learning outside of the improvements in deep learning.

You can take any deep meta-learning algorithm, take the deep part in it, apply the improvements in deep learning from the last year and claim that you have improved on the deep meta-learning problem this year. Well yes, but actually also no.

It's like trying to find a new antibiotic, and the solution is throwing more existing antibiotics into the same pill. Well yes, it works, but it is also not exactly the problem.

Don't get me wrong though, GPT-3 is amazing work.

Reply on news.ycombinator.com