Udacity's Sebastian Thrun, Godfather Of Free Online Education, Changes Course
fastcompany.com
fastcompany.com
The self-delusion is so strong that people are denying what Thrun himself said: Udacity was (is?) a lousy product [1]. It doesn't do what it was supposed to do, which is educate more people for less money.
The world doesn't need another way to spoon-feed infotainment to well-off people with good academic backgrounds (or at least, maybe the world wants that, but that isn't what Thrun wants, and I admire his integrity -- a lot of folks in his shoes would just be desperately trying to keep the party going, rather than admitting early failure). If MOOCs are nothing more than a place where half-interested techies go to pretend to learn about AI, then they're pretty useless.
[1] http://www.npr.org/2013/12/31/258420151/the-online-education...
Course completion is a valid goal, and an important number. The point that I'm making is that a 5% completion rate is still a success, when multiplied by the massive number of people enrolling.
For an individual, if you watched the first three videos of linear algebra and stopped, you didn't learn the subject and can't honestly claim you took the course. If you finished it past-deadline but did all the exercises, you're probably ready to take the next course.
Udacity and Coursera are a step forward. Perfect? Far from it. Ready to replace Stanford? No, not even close. Is the world better with them than it would be without them? Absolutely.
Education, in technology, is there mainly to give you a root set and confidence. If you actually want to get into machine learning, you have to start reading papers anyway. That goes whether you're a graduate student or a DIYer who's been in industry for almost a decade like me.
They can't expect valid results until the utility of the product is valuable.
Same comment as michaelo.
why does the completion rate matter? If 100,000 people sign up, and 5% complete the course, that's 5k people.
The US is graduating around 12k CS majors per year, so 5k seems like a material number of students to educate.
The combination of lecture, graded exercise, guided reading (usually) and time constraints that Coursera and EdX courses have is, IMO/IME, a very good combination for learning that makes them better than many other alternatives for the purposes of learning (Udacity is, comparatively, fairly bad at some parts of this.)
> Meanwhile, it's hard to take the courses seriously when only 5% of the enrollment finishes the course.
I think that completion is a bad metric. One of the benefits of MOOCs is that it lowers the sunk cost of entry which encourages broader enrollment, particularly enrollment by people who are experimenting with the content or less certain that outside commitments will be able to be contained in a way that allows completion.
The idea that % complete is a measure of how many people are deriving an educational benefit is wrong.
Actually, that is exactly what the world needs, and the comments here further emphasize that the demand is there. Before these MOOCs appeared on the scene, the resources were almost non-existent. I don't care about school, I just want to learn. A service like Netflix, but with MOOC-like content would be perfect.
It saddens me that Thrun, who I have found to be exceptionally good at the job, is more interested in schooling than learning – but I get it from a business aspect. These results dash the hopes of what Udacity hinted would become their business model.
It emphasizes that lots of people on the internet want something for nothing, which has a surprise quotient of exactly zero.
The disconnect is that you probably don't need to learn as badly as someone impoverished and living in the more closed societies that Thrun is trying to reach.
For them to become competitive in the world market they need to have skills and knowledge that you and most of the people with access to computers and the internet take for granted. So I agree with the parent comment that the world does not need another way for someone with a BS to get marginally smarter on an esoteric topic (for 99% of humanity) like AI. We need a platform for those without access to education to learn how to do basic coding, or learn other skills that will be needed in the coming decades.
4 courses at my local state university: $5000
-
Hell lets be a little bit more equitable:
4 courses at coursera with the signature track: $200
4 courses at my local community college: $400
So MOOC can potentially amortize over many more students and thus lower the cost to serve each student. In addition at least for instruction purpose the cost of real estate: MOOC negligible Stanford owner's capitalized rent equivalent for the classroom
The second kind of cost is variable cost that is associated with each student: MOOC: video streaming cost * 10 (to account for 10% completion rate; being very conservative here since students who don't complete likely will also not consume the allotted video bandwidth) Stanford: transportation cost and student's time to get to and from the classroom; additional room and board living on or near campus vs living at home etc
For example, imagine that someone like Richard Feynman, rather than giving his famous lecture series at Caltech for his regular Caltech salary, were doing it in a future era of well-developed MOOCs. He announces he's going to do one final series of physics lectures, but he hasn't decided which MOOC company he's going to do it for. I bet he could negotiate a huge fee for that, much more than even a top-end professor is currently paid. Moving back to the modern day, what if, say, Elon Musk offered to teach a MOOC? You could go for a cheaper, less well-known lecturer, but then you risk looking like the 2nd-rate MOOC company compared to your competitors.
If that's how the market develops, MOOCs might have to pay fees closer to what the public-speaking circuit currently pays prominent lecturers, if they want to attract the top speakers. And those fees are orders of magnitudes higher than what a professor currently makes.
1998 University tuition fees: 1000 GBP/yr
2004 University tuition fees: 3000 GBP/yr
2012 University tuition fees: 9000 GBP/yr
Thanks Tony Blair.
So we could reasonably say the number of Udacity student-courses complete is similar to a year of Stanford's.
Call it $40,000 for a Stanford undergrad for a year. A total of $280,000,000 for those 7000 undergrads.
Udacity has raised $22 million.
So an entire order of magnitude less than Stanford tuition costs for a similar number of student-courses complete.
Of course Stanford tuition doesn't just go to classroom operations, but that's kind of the point, isn't it?
1. Stanford sticker-price tuition is ~$44k, but a very small percentage of students pay full price. For example anyone whose family earns less than $100k/year (!) pays no tuition at all. A ballpark guess at average tuition actually paid is ~$20k, though I can't find an official number. Plus Stanford is probably not really the right peer; Udacity is competing to a large extent (in terms of who they might conceivably displace) with cheaper education providers, like community colleges or even the Cal State system.
2. In addition to the $22m, Udacity is currently getting some $millions in donated labor from universities, which are lending out their professors to develop Udacity curricula and teach courses as part of experimental partnership agreements. Long-term it seems that Udacity will need to pay for this labor themselves, if it's to be revenue-neutral rather than cross-subsidized by universities.
2. I picked Udacity since their courses are custom developed for the platform. Coursera courses are often very similar to the courses the instructors developed at their "day job" univeristies, the Udacity courses aren't really. They're new beasts.
I'm not sure how you measure "real" learning, but course completion doesn't seem to be a good way of doing it.
A well-designed course isn't a random bag of stuff. It has a starting point and an ending point, and skipping out before the end can certainly be regarded as a failure, at least by the instructor.
Here's the thing: you can't tell.
Why is there a final exam? Because otherwise, you can't tell. It's an exit survey.
This is one of the clearest indicators that Udacity and other MOOCs aren't actually "revolutionizing" anything in education; they're just making an economic gamble on an iteration.
Does that mean the class is a failure? Not at all. I found the format and interactive quizzes to be superior to the college classes I took. In addition, you got to skip around to take what you needed, and work on what you like. It doesn't fit credentialism, but as a resource it was unmatched.
I didn't do any of the assignments for CS101 or CS253, which taught python and building web applications, but from there I was able to build a webapp far more complicated than their assignments required (basically I chose my own assignment).
I'd never have been able to find the time and patience to sit through a university class just to build a side project, not to mention cash investment. I can honestly say that without Udacity this wouldn't have happened. For the woman quoted knocking the lack of human contact: sorry this wasn't a social club for you.
The numerator is the important metric here, not the denominator.
And the other 95 million will get more exposure to knowledge than they had before, even if they don't take it to completion.
It's hard for me to find the failure here.
It might better fit adult learning process as, at least I, get distracted more easily (less focus power, more things to handle, different brain). Actually that may be why Thrun is going into corporate education...
The people who are really educationally disadvantaged are often that way because of some other difficult circumstances in their lives (violence, social breakdown, poor parenting, and the associated effects on motivation).
I'm not referring to (typical) hacker news types here (myself included). You could give us all the extra educational access in the world and you wouldn't make a dent in educational inequalities, whether in the US, or globally.
The people who consume the MOOCs treat them as they would _books_ -- to be picked up, sampled, read for the good parts, saved for future reference.
The "courses" offered by MOOCs are in fact, _multimedia books_ with an optional social component. If they were viewed this way, a different business model might emerge.
I wish that was possible with Coursera. I often need way more time then the structured course time. I don't interact with people on the forums so I don't care about that but locking me out of the videos means I don't get to see a lot of things I would like too.
However, you can download the videos once the course is finished.
Of my 12 completed classes, only one does not allow access to the videos. Some of these classes are 2 years old. The one that doesn't have the old videos sent out an email saying they were removing them when they re-ran the class with new videos a year later.
MOOC is about as silly as "horseless carriage". The old model of education similar to a mathematical proof, with technical lemmas built up progressively until there is a macroscopically interesting result. That has a lot of value. Linear algebra is useful in so many fields, and learning it when you have the time, because you have to, gets it out of the way so you can progress to the fields you actually care about (machine learning, signal processing, physical sciences).
I think the new model will be graph-based and open like Wikipedia, but it will have to be aggressively curated for correctness, quality, and coherence. In 10 years, one should be able to take a query like "I want to be employable as a data scientist in 3 years" and turn that into a spanning tree of the relevant subgraph.
There'll also be a need for an offline/online hybrid approach like Meetup.com and dating sites-- some interaction offline, some in person-- and that part of it won't fulfill the dream of squashing geographic inequality. Perhaps education will be the first killer app for true "Metaverse" technology (but it won't be the last or biggest).
MOOC's can't really provide that. I've toyed with doing a startup in this regard. Perhaps something like gympact but targetted at MOOC's. I.e., you pledge a certain amount of money that you'll finish a course and lose it if you don't.
Even better if you pledge with a group of people and the whole group loses their money unless the whole group finishes. That way you could harness some major peer pressure.
As such, the unspoken "secret" about college courses is that most people do the least they feel they have to do. The majority of students are simply marking time.
That's deadly in a MOOC because, well, there's no consequence to failing. The least you have to do is... nothing. Combine that with the fact that there's no tangible reward in the way that there is with a degree from a college and you get a lot of people doing nothing.
People, as a group, really aren't interested in learning any more than the bare minimum they need to get what they want. That's what MOOCs are really up against.
As long as there's not honest discussion about that, MOOCs are going to have a hard time of it. High-minded idealistic the-people-want-to-learn approaches ignore reality.
Any feedback on my startup idea? Or do you think there might be better ways to fix MOOC's?
I would imagine most people doing these courses are looking to either improve their work or get into new work. If all the major companies started to somehow make these MOOC courses similar to requiring a degree, the completion rate would likely skyrocket overnight. If there was a way for a company to say "if you have done these MOOC courses we will give you preferential treatment in your job application", it may make a big difference.
I also think the approach that Udacity is now taking could be very effective. Instead of going on offsite training for a week, employees can be rewarded for taking and completing a MOOC course over the span of 6-9 months.
Having taken a few MMOC's over the past couple of years, the addition of a Meetup was, for me at least, a key motivator to keep me involved in the class. Based on others in the group, I think they feel the same.
You should also be able to invert that, by saying, "I've found all these subjects interesting; what might I be employable as?"
This seems like the kind of thing a recruiting company ought to start working on now. You'd be able to spot likely recruits really early on, brick-and-mortar or MOOC or whatever.
That's still thousands (tens or hundreds) of students who passed. So perhaps the 'click-through' rate for MOOC is the same as it is for ads?
I feel Professor Thrun's work has had a positive impact on my work, even though I'm one of the students who didn't pass on time. I gradually worked my way through the material while riding the Caltrain and have a greater understanding of ML and AI topics. So it's not a total wash.
Also, by admitting he may not have been right publicly he disarms critics overall. Smart.
1) Access the videos/materials for all time (even when the course is no longer open).
2) Try out different courses / professors
3) Learn the basics (i.e., watch only week 1 out of 6); or perhaps watch all videos but ignore the testing materials. After all, if there is no real credential involved, I might only care about gaining an understanding of the subject, rather than prove I understand it through testing.
I am a full time student (not in CS), and in addition to my real life course load I am currently taking Networking, Cryptography, and Algorithms. There is no way I will have the time to complete more than 1, or MAYBE 2, of the 3.
In other words, I view my low completion rate as a huge success. Coursera seems to have realized this fact. They have just started doing pre-course surveys that ask you how many videos you intend to watch (none, 25%, 50%, all), and they ask how much of the homework you intend to do.
Of course, offering degrees or college credit, especially in exchange for cash, will obviously increase completion rates, but mainly by dissuading people like me from signing up.
I see plenty of interesting courses on Coursera, but the one page + one video description is not enough to understand how the course will feel like and if I want to commit few hours of my life to follow it.
I always hesitate to sign because I have the feeling that by signing, I take a solemn oath that I will finish the course... (so at least I am not contributing to the bad metrics, but I feel like the metrics is skewed for wrong reasons anyway)
I think the real problem is that MOOCs are not brick and mortar schools, and any attempts to mirror a brick and mortar school will only detract value from the offerings. I don't sign up to do homework or take tests, I sign up to learn something. I will then take what I learned and apply it to something practical in my own life. If successful, it proves to me that I understand the content well enough to accomplish what I set out to achieve. I don't feel the need to prove that to anyone else. A certificate of achievement hanging on my wall means nothing.
Of course, the real issue here is that the business model was based on the idea of selling graduates to interested industry partners. If I don't successfully complete the tests, I'm not valuable to them.
Everyone's too fixated on the completion rates for these courses. The reality is that the people who are checking out these courses are doing it mostly out of intellectual curiosity at this point, so they have no reason to finish certificates, or finish courses, or watch lectures that they're not interested in. These people have no incentive other than to pick and choose, and to idealistically expect that people will put themselves through the downsides of education (HW, exams, watching the boring lectures when they can just pick the interesting lectures) is unrealistic, and certainly not an indication that "MOOCs have failed."
If people merely wanted to get the information out there, and make it possible to self-educate, then they would develop... well... wikis. Websites. YouTube videos explaining concepts. And that'd be it: they'd judge their success on outreach, on page views and maybe comments and emails. You'd have Khan Academy, full stop.
But that's not actually all they want. They want to know that their students understood. And to do that, they need to have the student prove their understanding as feedback. That's what completion rate really means. It is the metric that professors and teachers are accustomed to measuring themselves by. They've been trained to recognize the quality of their work based on the pass/fail rate of their class.
There are two large problems with MOOCs right now that are documented in the article but not well enough supported IMO: 1) There are no transferrable credits to colleges and 2) They are free.
And the two problems go unsurprisingly hand in hand, and they both have to be solved simultaneously if MOOCs are going to work. The ideal scenario would be that each course costs somewhere around $150 and courses can be transferred WITH credits to as many accredited institutions as possible. Why? Well firstly, once you pay for something, the average persons's conscious will naturally have that commitment in the back of their minds and will not want the investment to be a sunk cost. Furthermore, if credits are offered, college kids will be able to take a variety of courses in fields that they may want to try out and the more institutions that allow credits from MOOCs, the more students will feel comfortable taking these courses. It will be a great outlet for students to use in a variety of ways. The SJSU example is extremely poor as the sample is way too limited and as mentioned in the article, the bulk of students who signed up for courses were in limited situations.
In contrast, I'd be happy to pay $5-$10/month for overall access to Coursera or Udacity (probably just Coursera actually given the quality issues I've seen on Udacity). But the minute you try to shoehorn me back into the traditional mode of learning, I'm going back to reading books at my own pace and timing. Can we pretty please finally have the 21st century that should have started sometime during the last decade?
Bonus points for figuring out that I download and save a lot of lecture material for later viewing because some professors force their content to expire and go away. This stuff is gold for passing the time on transcontinental flights.
Negative points for the absolutely godawful official Coursera mobile app in an age when Coursistant is both free and 100x better. Why did they even bother with such an obvious Acqui-hire in plain sight?
Finally, if people pay $X for a fancy certificate and then don't complete the course, sometimes it really should be considered their problem, not yours. You can't save a world stuck in a infinite loop of its own making. What's the old joke? How many therapists does it take to change a light bulb? Only one, but the light bulb has to want to change.
On the other hand, some professors are experimenting with more interactive methods -- not less interactive MOOCs -- and getting three times better results:
Based on the conclusion of this link, it would be interesting to combine MOOC courses for the lecture part, with a technology like knewton, which by monitoring exercises you do , learn about what areas you have problems(and perhaps about misconceptions) , and can guide you to relevant content.
Overall, I think there was a real lack of attention to detail after an excellent job on getting the big picture right.
That said, MOOCs have revolutionized my ability to keep up with technology trends and I spend 1-2 hours daily on Coursera and Udacity. I do not care whatsoever about certificates or completing courses. I just care about raw knowledge acquisition and reviewing previously learned material (i.e. refreshing my calculus, statistics, algorithms, and linear algebra skills).
I personally think the future of this business will involve celebrity teachers like Andrew Ng and Sebastian Thrun putting together more polished versions of their courses and then allowing a cottage industry to grow around tutoring people through them at their own pace rather than shoehorning a freestyle approach to learning into effective semesters and quarters.
If this is infotainment, it sure beats the likes of Fox News and MSNBC.
This is actually fairly similar to my own classroom designs, so I wanted to call it out for emphasis.
But courses with deadlines I spam subscribe the hell out of. The majority are available indefinitely once you subscribe and I may be interested in revisiting them at some point over the next year or so. Even if they delete the content after the class finishes I just use a small script to keep track of the dates and download everything once the final lecture is delivered.
You can try and recreate scarcity online but it will backfire. I only complete about 1-5% of the classes I've signed up for because I want the option of picking one that might turn out to be interesting later. Places like udacity and coursera would be much better off having class material public forever while having recurring deadlines for submissions only. The only people who would sign up to classes then would be those who are interested in submitting materials.
One thing that I did at work was establish a betting pool with two other co-workers on the Coursera C++ course that finished in December - completers would collect from non-completers. That is my one Coursera completion.
I'm pretty sure that attaching betting pools to MOOC sites would make a major difference to completion rates for participants.
http://logicmason.com/2013/self-directed-programming-and-com...
To summarize the problems, the pre-requisites aren't always clear and are sometimes incomplete, the courses don't generally offer enough work that students can get help on, and set-up for the computing classes is often a huge pain. Finally, for me personally, the set pace is a serious detriment. I do much better if I can get the materials all at once and do 25% or more of a course in a weekend.
However, it is sad to see one of the most vocal people toss aside the original goal of providing education to the under-priveleged to help them escape poverty. Just saying that poor people have a different set of challenges that make online classes a "poor fit" for them is the easy way out.
It's also sad to hear him say that the true value proposition of education is employment. The notion that education and employment are hand-in-hand related is the thing that holds back the education system the most. Education is about allowing people to move forward - advancing culture, science, technology, etc. so that we're not all stuck plowing fields with oxen. While capitalism ties these advancements to employment and profit, the connection is based entirely on how things are done now, and not how things could or should be done (or even how they were done in the past).
I passed the original ai-class on schedule with distinction. Then when Udacity started I zipped through intro to CS as a review. When I took Norvig's 'Design of computer programs', it was a kick in the teeth. I knew pretty quick that I wasn't going to finish on schedule, then that I wasn't going to finish at all.
Going back to the comparison with textbooks, there are a good number of people who could teach themselves subjects purely out of textbooks if they were motivated. But even after mastering several subject in this way, your employment prospects and marketability are not significantly increased. Instead, people do their learning (online or in person) under the umbrella of a College, who can offer them "credit" / verification to the rest of the world, which in turn increases employment prospects and marketability. The MOOC product does not offer that benefit upon completion (they give you a certificate, but right now it is not respected at all and therefore does not give the desired effect), so only those interested in the intellectual knowledge have motivation to complete the MOOC courses (not surprisingly, a very low number of people).
But I think if they give it more time they / the market could develop products that provide value for completion of these courses. Maybe some sort of independent service that tests and certifies completing students (i.e. what College Board has done for high school courses with the AP program). They could even track combinations of courses students complete to ensure a rounded knowledge of a subject area.
That is just one idea off the top of my head, but the point remains that this is an economic issue of motivation and benefits for completion, not an issues of resources.
I intend to put that assumption to the test. I suspect that once GRE subject tests start falling, both my prospects and marketability will jump dramatically.
As a matter of fact I do not like Udacity very much. It is too playful and too unorganized compared to a lot of courses on coursera and pretty much all courses on edX. Udacity tries too hard to educate everybody by being very friendly at all times - this either leads to courses which start out all bubbly and funny and all of a sudden turn complex too suddenly - or courses that stay on that trivial level.
MOOC is awesome but what Thrun apparently doesn't get is that this concept still requires dedication and disciplin from the student. And you can design your class whatever way you like - either the student has it or s/he doesn't.
Also he is comparing apples and pears with his ratio #succesfull/#registered.
#registered of course is a marketing driven KPI and sould be as high as possible, so every time somebody clicks "Learn free" it is counted. But you have to "register" to just figure out what the course really is about and whether it is worth taking!
Also he is a business man and by switching to the new course format it is much easier for Udacity to monetarize - what a surprise. Opposed to edX and coursera, Udacity is kind of a one-man-show and he is maybe too ambitious and lacks patience (see biking with somebody else).
An article on my experience: http://www.joyofdata.de/blog/social-network-analysis-lada-ad...
If you don't fall into one of the above categories, it is _very_ difficult to overcome the practical, and motivational challenges that stand in the way of success in these classes.
Examples: a poor inner-city kid with serious challenges at home and at school, an out of work high-school/college drop-out who is years out of any sort of academic setting.
At least in the US, the disadvantaged side of educational inequality spectrum is defined by these types of of people. Without a great deal of in-person support (whether by government via schools, their communities, or their families), it is unlikely that such people will be able to increase their education, MOOCs or otherwise.
EDIT: added some clarifying wording.
I don't think that's an insurmountable problem, with the course notes being better designed and readily available (as compared to a disinterested professor + textbook). On the other side, many students don't bother talking to their professors and TAs anyway.
Most online courses are still under the old mindset of the conservative education model. Why does a course have a start and end date? I learn on my OWN time since I'm constrained by full-time employment. Why does a course have tests/exams? I came here to learn not obtain a badge, degree, or a set of meaningless numbers/grades. Why does a course NOT have a programming/application-centric method of learning? Why does an online course not have GREAT material and resources (i.e. lackluster application tests, labs, programming projects)? All these block me from doing what I want when I join an online course - to learn.
On the other hand, if the goal is to provide an education that is as solid and as valued as a university, which Thrun seems to think his goal was, then Udacity and MOOCs are not succeeding.
Further, if Udacity, which is a company that needs to make a profit, is going to succeed then it needs to change its focus.
For free online courses with certificate have a look at the above link
1.6 million times 5% = 80,000. Still an impressive number.
I've probably started 20 MOOCs and completed 3.5, so I'm part of that "low completion" statistic. I often fall behind the posted deadlines because I have so much on my plate. That's not a knock on MOOCs; if it's a knock on anyone, it's on me. The quality of MOOCs is (IMO) as high as for most big-lecture college courses. (It's not yet as good as for a 20-person course.) However, when you have a full-time job plus side projects plus family, and you don't have the social pressure of having to pass 8-10 courses per year, you have to be really motivated to complete the work, especially within the deadlines. But is it a loss or gain to the world if someone completes a MOOC 6 weeks late? Or gets only 60% of the material? It may not be as much of a gain as the traditional college course provides, but it's a gain in absolute terms.
If those 80,000 students are getting the same quality of course as they'd get at Stanford, that's fucking huge. Not a small accomplishment at all.
I wish people wouldn't write off MOOCs because, yes, they're the crappy first iteration that only early adopters care to power through. They're not ready to replace traditional education, the latter being a gigantic trillion-dollar problem that touches all sorts of deep sociological problems.
The problem is that you have a lot of hucksters in education (cough Knewton cough) who massively overpromise, raise a lot of money, and don't deliver much. They've damaged the reputation of the space immensely. The truth, however, is that online education is a good thing and it's (albeit slowly, with fits and starts) getting better. If there's anything worrying to me, it's that our ability to educate each other is, while improving, not doing so quite as fast as the rate of technological change. But I guess that's a good problem for the world to have.