Online Education's Dirty Secret - Awful Retention

Thankfully, we have a data analytics team now (3 out of our 17 engineers), and they are studying our retention statistics and the factors that affect it. They're also running A/B experiments to see what increases it and getting some interesting results.

We do see some big advantages of the timed model for learning, particularly in classes with peer-to-peer grading and evaluations, but there's obviously a big desire for the self-study mode, which is enabled for a few of our classes currently.

We're also introducing things like Signature Track, which some students sign up for just to encourage themselves to make it to the end (and it seems to work for many of them).

We'll keep experimenting to see what makes students both happy and successful. :-)

For me, the pace of the course is actually a plus. If there were no deadline, I simply would not get around to doing it. I remember when I first found all the MIT courses on-line (several years ago). I really wanted to take some of them, but because there was no schedule and no deadline, I never got around to taking any of them. It's only with Coursera that I have actually taken (and finished) any.

I've written about my experience of all three courses on my blog. The latest was Algorithms: Design and Analysis, part 2 http://henrikwarne.com/2013/02/18/coursera-algorithms-course...

As for courses being too fast: they're college courses. That's a core part of their value proposition. If they were slower, or did not follow a rigid schedule at all (like the "better" examples presented in the article), they would be a fundamentally different product. The author simply doesn't understand the concept behind MOOCs, and probably isn't their intended audience. He would be better served watching lectures on Youtube (I don't mean that sarcastically, there are fantastic courses available there).

Synchronous learning is not, as another poster claims, an anachronism. It simply isn't necessary or valuable for everyone. For some, myself included, it is a significant benefit. That is the market that Coursera and edX are targeting, and one shouldn't criticize a company simply because one isn't a member of their target market.

Unfortunately my free time isn't available in nice predetermined six-week chunks, but even if I am able to catch up three weeks or more in a weekend the courses gave a very negative vibe about continuing to progress as soon as you miss a single one of their deadlines (i.e.- "you missed our deadline for this multiple-choice computer marked test, so your effort no longer counts"). I've 'failed' several coursera sessions in the fourth or fifth week for that reason.

Timetabling seems a very traditional educational view, and it contrasted sharply with codecademy and sites like duolingo where I spent Jan and Feb learning the basics of new languages - computer and human. I finished the courses I took because I did them at my own pace.

I think the majority of MOOC course developers want to run their online courses as closely to their university versions as possible, since that's what they're used to working with. It also provides a handy way for them to go off-duty, in a sense, if the course has a finite end date. Using their current pacing structure (which is incredibly difficult, as the OP points out, for people that aren't full-time students) allows the teachers to do something other than devote themselves solely to the course, assuming they don't want to just post an archive and leave it alone -- which would meet a lot of people's needs, but misses the whole teacher-student interaction, which is pretty much missing from MOOCs anyway. If they want to provide an environment that's like a classroom, with students interacting with the instructors and with each other, you kind of need everybody at the same pace. It would be nice for us if they could slow that pace down, but that would probably increase the workload for the instructors.

We're still early in this game. I'm glad that so many professors have been willing to invest the time into developing the courses, and I understand why they are currently set up to be conveniently structured for them. I think we'll start to see some improvements if/when the money appears in the MOOC game. Once it's no longer basically charity work for the instructors, there will probably be more efforts to work around student schedules.

These completion rates are actually more like conversion rates for free trials. As many of us know these are almost always quite low. How many people actually ever read books they "Look Inside" on Amazon? How many people finish long articles on the web? Now many people sign up for paying accounts after free trials of your new website? Not very many.

They are not the same as dropout rates in high schools or universities despite what some online education haters say[1]. Like any disruptive technology the existing players are threatened so you need to pay attention to the source of the criticism.

Instead completion rates are a metric that can be used to improve the class such as the suggestions here. The developers of online classes should and are using A/B testing to improve completion rates. However like almost all conversion rates they will likely remain low.

[1] http://mitsloan.mit.edu/shared/ods/documents/High-Costs-of-F...

Back awhile ago free online lecture videos (e.g. Berkeley) were simply uploaded to a page and anyone could go watch them without signing up for a class or otherwise jumping through hoops.

This resulted in me watching a few random lectures about subjects I know little or nothing about. Like econ classes, engineering, science, or similar.

This was good because it was short bursts of information without any hassle, commitment, and similar.

These days everyone has locked their content behind sign up/registration pages, and you're expected to commit to a tradition semester of a class in order to get a worthless certificate of completion.

They also dictate the speed at which you learn. No more learning at your own pace, no, you have to do one a week for as many weeks as it goes on for like it was a traditional class.

And why? Why indeed. Why is online education simulating University education when it actually has no relationship to it? You aren't getting a University qualification by completing a "class," there is no reason why a "class" has to be X number of weeks or you should have to complete the rereq's in order to take it.

Places like Khan Academy have got this right. I can go to Khan Academy right now, click on a video series, watch a few bites of information and then stop when I'm ready/bored. Coursera's class system is just pointless, stressful, and annoying.

If Coursera was transferable then it would absolutely make sense to do. But that doesn't look likely and while that isn't the case they're just making access to learning material more difficult with no obvious benefit to anyone.

On the other hand, I mostly completed the first algorithms course and Model Thinking, and while Daphne's class kicked my ass the first time, I'm already reviewing to tackle it again and try to finish it this time.

It is true though that the time commitment is horrendous, especially for Daphne's class, which estimates 15-20 hours per week! If I fall behind, I may finish slower than their schedule and miss the homeworks and final exam, in which case I'll officially be an incomplete but I'll still have gotten through all the material.

In short, retention is an irrelevant metric for online classes, especially for Coursera which has such a strong incentive to sign up "unseriously." My suggestion would be to keep accepting and grading homeworks and exams at any time, and give extra props on the certs for completing on time with the other students.

(I've completed the Codeacademy Javascript track, and a full Coursera class (livin' in the 5%, wooo!)).

You say Codeacademy got it right, but I think you're dead wrong. As other articles today have pointed out, at best you're going to learn syntax on codeacademy, and the most basic programming principles, but you'll be miles away from being a capable programmer. The jump from the online editor with tiny little exercises to setting up your own environment and programming your own project is huge. Many of their lessons give misleading or outright incorrect information, not to mention teaching you bad practices.

What this article says is the huge barrier to entry, is not a barrier to entry to online education, but a barrier to entry of learning. What the author proposes is solutions to hide the barrier of learning, by gamifying the platforms and making them more "fun" with a disregard for the depth of effectiveness, and the consequences of pursuing such techniques. (Obviously I'm delving into opinion land, but I strongly believe gamification cheapens and platform and doesn't produce the deep engagement necessary for learning. This is my assessment from my experiences, I'm not going to tell someone their wrong for feeling differently about this. But I think if you wanted to create a deeper experience from gamifying education, it would take a tremendous effort, far different from the typical "Wow you completed 5 exercises! ZOMG KEEP GOING!" joke of rewards that other websites have.

I don't want to rail on the online education effort either, but this is plain cheap thinking. Coursera and EdX are doing an excellent job, but they certainly have many problems to surmount. They've definitely solved the problem of higher education accessibility, now the problems they have are in the effectiveness of education, and on administration and grading that gets the f#$% out of the way of learning. But making it "easier to learn"? I call utter bulls$%t.

The lower the retention rate, the more the likelihood that they are following lots of courses at the same time or multitasking with something else. This means that people are "exploring" more! I think that "exploratory learning" has always been stifled by classical organized education and is one of the reasons why I hate most academic environments (yeah, they're cool if you're in one of the top 10% unis or in a "privileged" position", but not for the rest of cases...). I believe that, after a certain level of baseline knowledge, in any field, it's actually more important that someone learns "what they want/need/have inclination for/find more interesting" than that they "learn more"! Maybe more time exploring and less time actually uploading things to your mind is better (not "productive", just "better", and I said "fuck productivity" a lot lately because I found that it just doesn't lead to better anything). All the new ways of doing education make exploration easier (as in you can explore a lot without really "wasting" that much time), and we should take advantage of this!

The ideal model is one of personalized education. What's kind of shocking is that online education so far has basically ignored this altogether. There's been zero investment in ways to continually pace and encourage students to learn at a speed that is both challenging and within reason.

This isn't too surprising: it's a hard problem, and it's much easier to have users paying a subscription fee forever (for those services that are paid) and blame themselves when they fall behind.

At Thinkful (http://www.thinkful.com/) we see evidence that our learning model, which pairs experts and students together much more like you'd expect from tutoring, sees 10x the retention and completion as other online course options.

The future me always has more time, is always more dedicated to learning, is always more focused. So I sign up, and when the class starts I'm no longer the better, stronger, faster, future, me, I'm just me.

Still I manage to finish some, and learn a little from the ones that I drop, so I don't see it as a problem.

Meanwhile, the schools that the authors hold up as more successful examples of education have 20-500 successful graduates per session offered, nearly all of whom are idle young wealthy white people living in rich western countries, and who pay up to several thousand dollars per session each for this privilege.

There is some lower bound to the amount of time that it takes to learn something (absent some educational revolution). Many people aren't even willing to pay that lower bound simply because they have other priorities in life. But they do like to sample (which is good), and potentially finish out a course if nothing else gets in the way.

Online courses are great because they lower the cost of sampling, so we shouldn't be surprised that there are more samplers, and fewer people finishing out the courses. If 1000 people complete an online course, then that's great, even if 99000 people signed up and disappeared a week later.

That being said, I'm pleased to see the specific criticisms offered in the article, and I hope they lead to a better balance for more people.

I signed up for the Hinton neural network class when it was almost over, and completed it well after it ended.

Was I "retained"? Did I not count as completing the class?

Just because there's deadlines doesn't mean you have to finish it by the deadline.

All three overlap at the same time!! And they don't give assurances of when they will be offered again, or even if. Argh. I've started Scala but doubt I'll be able to continue if it I attempt PGN again. This happened to me last fall and I ended up completing none of the three.

Plus, it is apparently "known"... somewhere... that certain courses are easier if you take other courses first... but good luck finding that information when you want to refer to it.

What she found was that the course was way more difficult than the description suggested, and proved to be well above what she could understand with her current background. She watched the first video and found that it was going to take way more time than the course description predicted, because she would have to try to fill in a tremendous amount of background knowledge that the teachers presumed. On the plus side, from the course forums she found about resources (videos, etc.) that were more at her current level, and has been working through these.

The comments made elsewhere about the synchronous model for these courses is probably right for her case - she had to wait a number of weeks in order to find out it wasn't the right class for her. This is what you expect when you're in college, at which point you either drop the class and maybe find a different one to take, or tough it out in a class that's not as interesting or useful as you'd hoped. I hope online courses can do better than this.

That said, not all material on Coursera is "amazing". Some of the classes have very high completion rate like Functional Programming with Scala has a 19.2% completion rate [1]. Similarly, the class taken by Andrew Ng on Machine Learning is fantastic. However, many of my peers had bad reviews on Daphne Koller's Probabilistic Graphical Models class. Last year, I myself registered for one of these MOOC's and found that half of the course was good while the other half was quite bad - both halves had different professors.

At some point of time, universities would have to realize that great researchers do not make great teachers. Some excel in both - researching & teaching while some in just one of those two fields.

PS - Other problems on reengagement do stand though.

[0] https://www.udacity.com/ [1] http://www.katyjordan.com/MOOCproject.html

Or we could ask whether it is a problem. I'm not sure that it is.

I don't think the numbers are surprising or unnatural at all. The thing to compare them against is not traditional education but rather the patterns of all people who take the autodidactic route.. every person who has ever picked up a book and not finished it. Even then, the comparison isn't perfect.

In fact, we could argue that low retention numbers are actually a good thing for non-credit courses. It means that the content is challenging and focused, and that people are able to discover early whether it is or isn't for them or cherry-pick the content that they need and move on.

If there are usability problems, fix them. But if participant retention got dramatically higher, I'd worry.

Codecademy and Udacity allow you to freely dip in and out of lectures and assessments, whereas Coursera courses demand that you deliver quizzes/assignments every week. It's hard to recommend one approach over the other -- with weekly deadlines I find I have more of an incentive to engage in the lectures and course materials, but the added failure conditions can push students to abandon the course.

For my part I sign up for more classes than I could possibly complete. Sometimes I just want to take a peak at the material to see if it grabs me. And sometimes I just want to get familiar with a subject without going all-in, picking and choosing the lectures that are most relevant to me.

People shouldn't feel like they're hurting Coursera's/EdX's/Udacity's stats by dipping their toes.

I hope there will be more short and tightly focused online courses in the future instead of monolithic traditional-length courses we have today. I think that'd go a long way to improving the apparent retention rate.

Why? Three reasons:

First, a 'dirty little secret' of the US software industry is how much of the learning from the beginning of electronic computing to the present has been just from individuals teaching themselves from books, e.g., K&R on C, Lippman on C++, Ullman on database, Sedgewick on algorithms, and on-line, e.g., Microsoft's MSDN site, StackOverflow, etc., essentially independently without courses, lectures, problem sessions, credits, homework, tests, etc.

Or since just K&R, ..., StackOverflow, etc. have been responsible for so much learning so far, then 'the bar is low' and on-line courses should do even better.

Second, in my experience in technical subjects, pure and applied math, mathematical physics, some topics in electronic engineering, e.g., surrounding the fast Fourier transform (FFT), and software, with a class or not, nearly all the learning (in my case) took place from study, alone, in a quiet room, from good materials just on paper. My 'educated guess' is that on-line learning can't replace such learning but can help stimulate more of it.

Third, for a researcher in applied math and software, and also likely some other fields, one of the main 'work items' is to take recent books and papers and work through them much as working through advanced course materials in such subjects.

Back to my case, in the fields I worked hardest on and did the best in, math, physics, and software, starting in the ninth grade, through my Ph.D., and in my career to the present, I did nearly all the work with relatively little contribution from teachers. E.g., in plane geometry my teacher was the most offensive person I've ever known in education, and I slept in her class and refused to admit doing her assigned homework. Instead, I worked every non-trivial problem in the book including the more advanced supplementary problems in the back where she never made any assignments. Then, after working all those problems, no wonder, on the state test in the subject, I did fine: I came in second best in the class; the guy who beat me also beat me by a few points on the Math SAT -- we were 1, 2 in the school. Net, my approach to learning plane geometry worked fine.

I never took freshman calculus. The college I started at wanted me to take some math that really was just a review of what I'd covered in four years of math in high school. So, a girl in the class let me know when the tests were, and I showed for those. The teacher said I was the best math student he'd ever had. Meanwhile, I got a good calculus book and started in, worked hard, and did well. For my sophomore year I went to a much better college and started on their sophomore calculus and did fine. Yup, never took freshman calculus.

When I went to graduate school, I took a problem with me and had an intuitive solution. My first year had some good courses (one was just terrific, from a star student of E. Cinlar now long at Princeton) and gave me what I needed to turn my intuitive solution into a solid math solution; I did that in my first summer, independently; and that was the research for my Ph.D.

I continued that way in my career: E.g., in a software house working for the US DoD, I saw a problem in a specification, got Blackman and Tukey, 'The Measurement of Power Spectra', and read it carefully enough to see what was wrong with the specification and how to fix it. Right: without courses, lectures, problem sessions, ....

As far as I can tell, nearly all the technical content on HN, StackOverflow and other Internet fora is from people who taught themselves in similar ways with little or nothing in courses, lectures, problem sessions, .... And that's part of what researchers have to do and is just part of getting tenure as a research professor.

So, since so much work is being done by essentially independent study now, just by not making things worse on-line courses should be able to look successful.

But I see some problems with the on-line materials I tried:

(1) The video quality just sucked. I couldn't read the board. That meant I couldn't copy what was on the board and study it. Bummer.

(2) The sound quality was not good enough.

(3) The course materials, e.g., on paper or in PDF files, were from not good enough down to just missing.

(4) Sadly the quality of the course content was too low; apparently the main reason was the desire to make the course more 'appropriate', that is, 'easier', for 'the common man in the street'. But, omitting material 'waters down' the course content and, really, for a good student, requires that they fill in the gaps for themselves -- bummer.

E.g., I looked at the course by Stanford professor Ng on 'Machine Learning'. What I saw were weaknesses (1)-(4) above. For more, (A) a lot that he was doing was maximum likelihood estimation but with far too little explanation and justification; so, I would have had to have run off and studied maximum likelihood estimation on my own. So, again I was on my own to do some independent work, trusting Professor Ng that somehow maximum likelihood estimation was better than it has long seemed in the statistics community. (B) He mentioned the 'maximization' to be done via following gradients, and that is an overly simplistic and not very promising approach to maximization -- the standard, first problem is that spend nearly all the computing time moving in directions nearly orthogonal to the direction really should be moving in.

So, from (A) and (B), I concluded that for a good course I would have had to have taken his lectures just as topics to be investigated, gone to good materials elsewhere, collected good details, and written my own text. That's his job as a professor, not my job as a student. His field, 'machine learning', didn't look worth that much work for me now. I've done some serious work in several cases of applied math that could be called 'machine learning' as much as his material, and I'm left without much respect for his material.

I looked at the course 'Probabilistic Graphical Models' taught by Daphne Koller. Since I very much liked a course by a star student of E. Cinlar, maybe I should like Kollar's course. Sorry, I didn't -- the quality looked too low. Better quality from Stanford? Sure, K. Chung, H. Royden, D. Luenberger (his 'Optimization by Vector Space Methods' is a beautifully, even elegantly, done one mile long applied math dessert buffet), D. Knuth.

For courses in 'how to code', that is, introductory material in software, gotta be kidding! 'Coding' alone is easy; it's just, pick a language, learn the basic syntax, and write if-the-else, do-while, call-return, allocate-free, etc. It's easy but doesn't take one very far. So, don't get very far with thousands of Web pages of documentation at MSDN on .NET, ASP.NET, ADO.NET, administration of SQL Server, IIS, Windows Server, etc. or the equivalent in the Linux world.

I have two broad conclusions:

First, traditionally in academic material in technical subjects, the author and the student 'reached' to each other, and they connected at a well written textbook. So, the author went far enough toward the student to prepare a good text -- and the best texts are terrific. And the student reached far enough toward the author to make do with little or no more than a good text. It's how I learned plane geometry, freshman calculus, theoretical, applied, and numerical linear algebra, everything I learned about statistics, most of what I learned about advanced calculus, signal processing and the FFT, stochastic optimal control, artificial intelligence, ..., and everything I learned about software. E.g., it's heavily what worked for me.

The on-line community will have to face the fact of this 'reaching'. In effect, on-line learning requires the professor to do more work of a kind that promises not to be well rewarded by tenure and promotion committees. So, net, so far a student should still reach mostly for one of the best texts, on paper or PDF.

Second, the situation will 'settle out': The professors will come to understand what minimum quality is needed, and the students will realize that the learning is not just a spectator sport, is not like watching a movie, and still requires nearly all the traditional work from a book or PDF file. Then the courses will get better; the students just looking to watch a movie won't sign up; and the course completion rates will increase.

Before I read the fine article (which I recommend you read too), I was going to react just to the headline. And that reaction is still there. The reason we (correctly) criticize lack of retention in online education is that online education makes apparent metrics that show the lack of retention. But I would suggest, for learned discussion by all the participants here, that most forms of classroom learning, and especially compulsory secondary schooling in most of the developed world, suffers from a very bad lack of engagement on the part of the students. (Some of the comments posted here before this comment tell stories about that.) A high school student may attend classes often enough to avoid being expelled and gain a diploma, but the student may operate in "regurgitate and forget" mode the whole time he is in high school, not really remembering anything he supposedly was taught. The one kind of deep learning many high school students engage in is learning how to pretend to be studying. If online education develops better demonstrations of learner acquisition of knowledge and skill than mere seat time in a classroom, it may be a powerful force for exposing the dirty secret of classroom education.

If you have issues with the class, that's a personal problem and you simply need to work on your willpower.

For the Coursera classes I did sign up for, I did it because I am interested in the subject area, know almost nothing currently and wanted to gain some functional understanding. I would say of the classes I signed up for, and probably 50% of those I have seen all the lectures. But then I go off and incorporate into side projects to test out / make sure I understand the materials. I have never turned in an assignment, (almost) never posted on a message board and never opened a final exam. Why should I? I am going straight to the application of the content into my projects, I've already achieved my goal of understanding and expanding my capabilities.

What reasons would someone have for wanting official completion of the course? On most courses, you do not gain any officially recognized credit, the certificate has limited (if any) commercial marketability for someone... so again, why would I want to go through the hassle of officially completing the course?

Two takeaways from these thoughts are the following: 1) How many people like myself have utilized the content of the course but are not seen as retained due to the fact they do not partake in the academic completion side? 2) What potential motivators can exist in the future to drive completion of the academic side of the coures? (college credit is the first one that comes to mind, but still does not help motivating many users who are above that age). What other drivers could come into being to incentivize?

The biggest thing I noticed were the clueless forum posts by students who obviously had not read much of the provided material. Two of the classes had material similar to ESR's "How to ask questions the smart way" and this was largely ignored. I think we are seeing the fall out of the distraction of modern society. Sadly, tl;dr doesn't work with complex technical material.

That said, it was a privilege to interact with motivated fellow students. Their input in the forums helped eliminate misconceptions.

By the way, the statistics for our Data Analysis class were similar to those from the OP - from the instructor: "There were approximately 102,000 students enrolled in the course, about 51,000 watched videos, 20,000 did quizzes, and 5,500 did/graded the data analysis assignments."

Coursera and other MOOCs are media properties that monetize viewership (thorugh certificates rather than ads). As media properties, michaelochurch and others are right that the synchronous model isn't really going to work in the long term.

I think ed tech should be about connecting teachers with the content they need to teach (and I'd include self-educators amongst 'teachers' here). Students need synchronous access to teachers, but those teachers don't necessarily have to have created that content just now. This suggests that online education might look more like a 1:100 teacher-student ratio rather than 1:100k. Nothing's more motivating and engaging than a human being.

Personally, my best experiences have come from two sources: self-study from great materials and guided study from great teachers. This suggests that a role for ed tech might be connecting great teacher with great content rather than broadcasting teacher-as-content. Personalization, engagement, motivation, etc all come for free if we empower great teachers rather than seek to replace them.

It's misguided to think that some professor from a great school can best deliver the content to all audiences. Imagine great open content accessible to every teacher for use in their own classrooms. With an inverted model of teaching, the lets them push content out to the students for self-study and then have synchronous classes where they, the teacher, can motivate and customize the delivery of tough concepts for their particular group of students.

1. Videos are usually too long. There isn't a natural break to stop and try things out. (Udacity does a good job with this one.)

2. Information density isn't constant. With text, this isn't an issue: I naturally adjust my reading speed if I hit a sparse or dense area. With video, I have to either live with it or constantly re-adjust the video speed.

I don't plan on giving up, but I'm curious if others have had that experience or if I'm the only person who's figured out how to use Codeacademy without actually learning to code.

His bait is retention .. and that's something HN readers understand from a business POV. But retention isn't his issue. Retention is a problem for Coursera et al.

His problem (and mine) is that we (personally) found it hard to stay the course. And Coursera don't help us.

I've a full time job, two side projects, a small business and a family. I'm about to start the Gamification course and I probably wont finish it because I don't have time. Not because I don't want to, but because the course expects all students to spend 4-6 hours per week studying. Those weeks would be rare for me.

How hard would it be to offer a 2 hour-per-week version? Or whatever commitment I'm able to make? Then, when the inevitable happens, let me hit the 'pause' button.

The only thing I don't particularly like is the inflexibility on turning in assignments. I completed five classes but did not pass one because something happened work wise that caused me to turn it several assignments late and that really knocked down my score. Great class though.

I have also started four other classes that I dropped after 1-3 weeks. I am still glad I was exposed to some of the material for those classes, so still a positive experience.

Accepting the figure that 95% of students drop classes on average, I would like to know if most of those people in general felt their time had been worthwhile spent. That is the important question.

CodeAcademy (also wonderful) is a completely different animal. It is not comparable to Coursera.

What you're calling a bug (aka "dirty secret") of Coursera is actually a huge feature: that we can all, whatever our goals or our budget, sign up and check it out. Quite unlike an class at MIT or Duke, which is only available to a pre-selected few.

In short, the MOOCs, and specifically Coursera, are awesome. We are so lucky to have this opportunity. Quit yer bellyachin'.

IMO for a class with tens of thousands people signed up, 5% retention rate is not really that bad.

Personally, I've signed up for dozens of Coursera courses but I've only completed one (Game Theory). It's not necessarily that the other courses were lacking the ability to keep me enrolled, but rather I never planned to try at them. I sign up to most classes to see the videos and get surface-level understanding of a subject.

The next step for online education is get a real qualification that improves your job prospects through places like Coursera.

Also I think there are a bunch of other hacks such as collaborative motivation (getting a group of members to commit to each other to complete the course and automatically reporting to each other when they have completed each section where everyone in the group has collective responsibility to help each person in the group to complete the course.

For Stanford's database class that just finished we had 64 thousand register, but only 20,000 actually do some work, and then close to 5k actually take the final or get a statement of accomplishment.

My blog response: http://sef.kloninger.com/2013/03/online-ed-retention/

I'm the engineering manager on Class2Go, Stanford's open-source MOOC platform. Check it out: http://class2go.stanford.edu/

I want to know as much as I can about different subjects but I want to immerse myself in them to different levels.

Sometimes I am happy just to stick a few videos on in the background while I do something else, so I get the gist of what something is about and know what to focus on later if I need more information.

I don't necessarily care if I could pass an exam or not.

The guy presenting, Ed Hess, was very skilled and interesting and the material was very relevant. I also enjoyed the discussion of various homework on the course forum.

Having taken online courses from the start my sense is that enrollment/participation is slowing. The bulletin boards of some classes are pretty dead on the first run.

Its not crazy to think that the people most likely to take the classes would participate right away and finding the next batch of students will be harder.

We are talking about a new model, low "retention" seems to be normal for free clases wich have absolutely no barrier of entry.

I have taken 5 courses, only finished 2 "on time". For the other 3 I downloaded the material, got some exercises on sites like MIT OCW, and finished them at my own pace.

The main incentive to power through a Coursera course is personal enrichment. This is not the reason most people go to college.

http://en.wikipedia.org/wiki/Churn_rate

I've tried pretty much all of the "MOOC's" out there, and so far the only two I've stuck with were Codecademy and Udacity. Why? Precisely for the reasons that are mentioned at the article which started this discussion. The format of Edx and Coursera is just all wrong for most people who DO want to learn something but have a busy life that is not conducive to a traditional college class.

Besides which, all too often, Coursera and Edx just throw a college class online (lectures, course notes, homeworks, quizzes, and tests) and call it a day. Then they have arbitrary deadlines--which I acknowledge might help some people but the first time I got a deadline email I decided that it wasn't for me...no matter how interested I was in the class topic. I'm already married and a father who is working part-time while doing a Master's program in Computer Science. So I don't have huge chunks of time available for traditional course work (in addition to my Master's coursework). I have little bits of time every day which I use effectively to progress through the Codecademy and Udacity coursework. Slowly but steadily I'm getting through both and making substantial progress.

But, since Coursera and Edx require huge blocks of time committed every week, and a tight timeline to complete the course, I just don't find it's workable. Which is sad since they seem to offer some tremendous material.

In short, I think that Coursera and Edx (and probably colleges in general) should learn a thing or two from the highly modularized approach of Udacity and Codecademy.

I find that my learning retention and interest remains much higher with both Udacity and Codecademy because they present a little material (little bite-size chunks) which slowly build into more and more challenging material and intersperse "challenges" or "quizzes" throughout--rather than the traditional approach which--by necessity--was far less responsive.

The fact is that online learning doesn't just reach potentially numerous people that wouldn't have otherwise been exposed to such educational opportunities at such a low price (currently free), it potentially allows for a radical rethinking of the traditional education model.

I think that it is sheer ignorance to say, as some have in the comments I've read (and as--apparently--Coursera maintains), that those who don't finish a course just aren't interested enough or motivated enough. This is ridiculous. The very same (allegedly "unmotivated" or "undisciplined" or "uninterested") people may be successfully working their way through Codecademy or Udacity. And yes, it may take them a while, but eventually they'll complete a Codecademy "track" or a Udacity course. And they'll do it one bite-size chunk at a time...and sometimes, say on a weekend, they'll plow through a whole bunch of bite-sized chunks.

When Coursera and Edx adopt a more bite-sized format which isn't oriented around arbitrary deadlines, I'll check them out again. Until then I'll stick with Codeacademy and Udacity.

Bottom-line: the standard model is broken. Lectures and homeworks and quizzes and tests are poor ways to educate. For many years they were the best we could do within the technological constraints we faced. But now we can do better. Heck, I'd encourage everyone working the bleeding edge of online education to think "outside the box" and try new approaches. While the best new approaches I've seen have been those utilized by Udacity and Codecademy, that's not to say that there aren't even better methods--methods better at retaining learners (keeping learners engaged), AND methods which excel at increasing retention of material.

Sigh.

* The Dabbler will pick up tons of things, "overjoyed" about the newness that they are learning. Eventually, they hit their first plateau and the 'newness' becomes boring and they'll seek out another activity to experience the same 'newness' high. Maybe they'll come back to it later and see if they can progress again.

* The Obsessive will seek out perfection on this new activity, pouring over every YouTube video or article to figure out how to be better. When they plateau, they'll try harder; hoping for the same results. If the return isn't there, the Obsessive will look for another activity.

* The Hacker will actually ride out the plateau potentially indefinitely. In some cases, this will create progress eventually, but in others, it can become the cap of their learning capability (hence, 'a hack'). They just don't put the extra effort needed to progress further.

While all these types are negative, two-thirds drop the activity for something new, something they might be a natural at. From a business perspective, a monthly subscription favors Hacker-type customers; where a 'per course' pricing scheme favors Dabblers.

Steering back to online courses, I think the issue isn't 100% the site's fault; but that certain students will join for the excitement, but fan out eventually (Leonard goes on later to discuss how to better approach each of these types as the Teacher). I've signed up to all of the sites in the article, along with Udacity (where I'm currently taking the Web Dev course). For Udacity and edX, I've completed the courses I've signed up for; however at Duolingo, I stopped after the Basics 1 in Spanish. I want to learn Spanish, but the drive and time isn't there for me.

Jumping back to 'Mastery', Leonard talks about 5 keys that can help promote mastery and can be used to retain students: Instruction, Practice, Surrender, Intentionality, and The Edge.

* Instruction is obvious: have great teachers(not just through credentials, but are generally good AT teaching), something these sites offer tenfold.

* Practice is where I felt edX's course fell through. It was a lot of lecture, but the Finger Exercises were more the tradition 'regurgitation' of knowledge from traditional classes. Duolingo and Udacity (at least the Web course) seem to have this concept taken care of. Duolingo offers different ways to practice and at least with Web Development, you can to see the fruits of your labor with each class.

* Surrender is more for the student than the instructor, but talks about accepting you aren't a Master as something and so sometimes you need to take two steps back to take one step forward (in Aikido, we refer to this a 'Kihon Waza', or practicing the basics). So many people have the mindset of "If you're not doing it this way, you're doing it the wrong way", that this can stifle learning. As an instructor, thinking up new and creative ways to practice the basics can help alleviate this.

* Intentionality talks about visualizing the action, and while focused more on physical activities, could be utilized online. Give the students a view at the end results/expected return on their time. Sure, its dangling the proverbial 'carrot on a stick' in front of them, but it can help them sees where the fruits of their labor. If you can mix this with some of the Practice key, you could get better results (IMO).

* The Edge tip-toes the line between endless practice and pushing the envelope (though Leonard says it isn't). For a new student, this can be a little harder, considering they are still learning; but creation of 'outside the box' milestone/projects could benefit from this. The student's amassed this bit of knowledge; make them use it in a less than cookie cutter way. This would be a great undertaking by online courses, due to how do you 'personally' grade someone's submission, especially overtime? Maybe setting it up in a 'there is no right/wrong answer' way, namely because if a student takes a risk, they shouldn't be penalized for it.

Like I said, I'd recommend the book. I teach community college currently, and while most this is anecdotal opinion, I feel like it's helped me maintain engaged students.

Synchronous learning, IMO, is an anachronism. We have video recording technology, practically infinite bandwidth, and near speed of light communication. The idea of synchronous learning, where you have five thousand teachers teaching the same Calculus I class every year, is predicated on a world where these technologies don't exist.

As an aside, for this reason I think the collegiate model is going to completely change in the near future as people start realizing how absurd it is. The real value professors provide is having someone to answer questions (and in a lot of cases, professors don't even take questions during lecture). And even that functionality can and is (in my experience) mostly replaced by other students in the class or students who have already taken the class.

Put the material up, and let communities form around it. I don't need the professor to answer my questions, all I need is the relevant forum (in the original sense) to ask.

It's not a "dirty secret", though. One should know and expect that. If anything, low retention is a good thing insofar as it means that the courses are demanding and people who aren't dedicated trickle out.

I feel like we have three problems to solve in online education. I'm sure there are plenty more, but 4 stand out right now:

(1) Hidden node discovery. You're 23. You just learned Python. You want to be a Data Scientist in 3 years. How do you get there? "Data Scientist job" is one node, and there are a bunch of prerequisites to those (hidden nodes) and prerequisites to those. How do you navigate this network? The 23-year-old programmer doesn't know where those hidden nodes are. In other words, the Google for Learning and Development.

(2) Forward learning. Recommendations. Things that would be interesting to a person that she doesn't know she wants to know, because she doesn't know that it exists. Since there's a lot of investment here (you're not just buying a book and possibly reading it, but anticipating putting 50+ cognitively intense hours into a course) it would be nice if the service gave indications as to why it was making those rec's.

(3) Interactivity and (buzzword warning) "gameification". When you haul out an 800-word machine learning textbook, you often have to go for a long time (hours) without the "kick". There's a flat array of 50 exercises of which 25 are easy, and 25 are really hard and will take a long time (they might be worth doing, but they aren't quick) and it's hard to pick which ones to focus on. The programming exercises in texts often don't get your creative juices flowing. Not a lot of people can get through long spells without feedback, and the skill of "making your own feedback" seems to be losing ground in our distracted culture.

(4) "Social". Online study groups. This is Big and I don't know how it's going to evolve. How do we keep quality control in place and make sure that our automated expert discovery mechanisms work?