Show HN: We measure classroom confusion | Hacker News Reader

Show HN: We measure classroom confusion

blog.classmetric.com

133 pointspkrein32 comments

Just to point out -- whether it's relevant or not -- I remember reading recently that confusion is often a good sign, i.e. confusion often means that one mental model is being replaced by another, which can be an indicator of learning. For example, in the process of replacing a common-sense model of physics with a Newtonian one, there has to be a "confusion" stage somewhere in-between, and an absence of such stage could indicate a failure to acquire the Newtonian model.

Certainly, although you have to differentiate between confusion, which can be beneficial as you say, and being completely lost, which hampers learning.

The subject matter being obvious and well understood can look a lot like the subject matter having no meaning at all because the learners are completely lost.

Particularly, asking 'any questions?' and then moving on (often rapidly) when no questions are forthcoming is another sign. It's hard to tell if it is because there is full understanding or almost no understanding. Instead, try giving more time for people to think of questions, and ask 'what questions do you have?'.

I've not had a chance to try, but in a similar vein, having an IRC chat running along side a lecture could provide similar useful feedback, and an avenue for questions without making learners feel stupid.

I like the direction they are taking with this tool. Good, honest, and especially timely feedback is one of the difficult challenges of education. Without it you are flying blind.

How do you reconcile this with the observation that the most confused people are the ones who understand the most? https://news.ycombinator.com/item?id=2931518

The crucial thing is that students can now give some feedback - if it turns out that having students more confused improves their learning, then professors can purposefully drive the red line higher. As PG has said, you improve what you measure. Professors have a powerful way to measure confusion now.

As a fairly recent graduate (just over a year now) I'd like to say, please stop giving us things to distract us in lectures.

That's actually my lecture that ClassMetric broke down in the blog post. From a (first time) instructor's perspective, I don't see the tool as a distraction to students. I'm not punishing students for using it, and students have another way to communicate to me that they don't understand something in lecture. I only lectured for one hour a week, the TAs spent the majority of the time with the students. In my case, Class Metric allowed me to touch on the big ideas of the week during my lecture, measure confusion on topics, and figure out how TAs should structure their discussions and labs to be the most useful to the most students. The message posting system also allowed TAs to participate directly during lecture as well, answering questions and addressing confusion as it appeared so students could better keep up with the lecture material. Yes, it may be distracting to have knowledge coming from two channels, but students can still refer back to that channel later when they are reviewing the whole lecture material.

If you think about it, the way we teach now is really primitive. So much feedback could be gathered and incorporated so easily, and greatly increase the effectiveness of the whole process. Imagine something like this incorporated with the Kahn Academy. Also, imagine the feedback being used not only to help the teacher improve the lecture, but also in a "stack overflow" sense to cause the best lecture videos to "rise to the top". I am very excited about what is going to happen in the field of education.

This looks pretty interesting. I actually would be more interested in using it for curriculum design, not really for real-time classes. I think I'd get a group of students and a BETA version of one of my books, teach a class with it, then use this to review which exercises are working and which need to be redone.

If it also included metrics on improvement in understanding over time and across instructors it'd be even more powerful.

It would be great to have you give it a try - email us at friends@classmetric.com and we can get you hooked up with the alpha version.

ClassMetric actually started as Bookxor, where the focus was on document analytics and discussions. We showed teachers which pages students were reading and how long they spent on each page. The system also let students post questions and have discussions in the margins of the document.

Over the next couple months we'll be bringing some of these ideas back to ClassMetric. Maybe these document-centric features would be more useful for you?

I think this is ok, but not nearly as good as: < https://learningcatalytics.com/ >.

Hey p4wnc6, welcome to Hacker News. I've downvoted your post - here's why.

The comment isn't particularly insightful. It espouses your opinion (totally OK) without any reason or rationale (not OK). Why is it better? What could this company learn from Learning Catalytic's product? Is this just observation, or are you uniquely qualified to judge these two products?

The link is completely superfluous. It goes to a login page and doesn't tell me anything about the company or product. Also, a note on style, links are typically just left inline http://google.com or in parens (http://google.com) but that's orthogonal to what matters - insightful comments.

I hope that's helpful!

(a) I am a grad student who works with Mazur and the post-doc who founded Learning Catalytics, so I think I am in a position to make a useful comparison. (b) I only intended to offer my opinion because, as the other commenter mentioned, some people feel Learning Catalytics is very different and I don't. I specifically wanted to avoid a lengthy back-and-forth about the nuances between the two. My two cents is that anyone who would use this product would be better served by Learning Catalytics. (c) The web link was just to alert you to the the existence of Learning Catalytics. If you want details, search harder. (d) I don't care about link style, especially not in a one-line comment. My comment wasn't hard to read or obtrusive, so why are you splitting hairs about the formatting?

Learning Catalytics is cool but different - it's the work of Eric Mazur, a famous Harvard physics professor who created the practice of "peer instruction." It seems to be a very effective method, but it requires a huge amount of work for the professor.

ClassMetric is intended to require very little additional effort from the professor, but still give them good feedback. ClassMetric's discussion system is very much in the spirit of Mazur's peer instruction. Students often answer each other's questions on ClassMetric.

LearningCatalytics and ClassMetric are different, but we'll definitely learn from Mazur's research. Thanks for the link.

Just an idea:

http://www.opticsinfobase.org/abstract.cfm?uri=oe-18-10-1076... + http://www.khanacademy.org/ + www.classmetric.com

As a university instructor, I want to find out more about it, but your page makes it harder than it should be. The "home" icon links to the blog, the one on the right does too. No links for more information, status of the project, or cost involved.

Woops - our apologies. We're currently testing at MIT and Boston University, but we'd love to talk to you more. Shoot us an email at friends@classmetric.com or head over to http://classmetric.com

As someone who've taught before (I was a TA at Berkeley during grad school). It's awesome to monitor the classroom and see if what you are saying even makes sense to the class or not. Kudos!

Anything that requires extra work from students is no-go.

Very cool! Helps turn a one-way learning experience into a dynamic one.

Guys, please don't do this. You submitted your startup. The submission got popular, and will now be discussed. Do we really need obvious sock puppet accounts created 10 minutes after the initial submission, commenting with enthusiasm and marketing speech?

Sadly, this is the only way to have your submission not drop off the 'new' page. The rate of submissions to HN is so high, that within the hour new submissions disappear from the first page. This is why people use PSUNs.

A PSUN is a Personal Sockpuppet Upvoting Network, which is a bunch of Amazon EC instances masquerading as human HN users, programmed to immediately upvote submissions and/or comments made by the owner of the PSUN. They're getting more clever at it too, by posting vapid marketing one-liners to bump the comment count. Some PSUNs are really elaborate with precisely emulating a human traffic pattern, reading comments, up/downvoting here and there (triggered by textual analysis of comments), and therefore impossible to distinguish from a real human user, even with social graph analysis.

Like Windows can't do with a Anti-Virus program, HNers who don't want to see every submission they make slide into oblivion, are required to deploy a PSUN.

Antonio, don't be so quick to cry foul.

I am one of classmetrics founder's brother, and run my own company out in Philadelphia area called LeanServer.

If you need my papers, I will be more than happy to fax them to you.

Best, Mike

> I am one of classmetrics founder's brother

In which case, although you may not be a sockpuppet in the literal sense, what you did is frankly not much better. Sorry to be the bearer of disagreeable news, but that's how it is.

It's not just that you'll get downvotes for this. I don't know about anyone else, but my opinion of the likely value of ClassMetrics went down when I saw your comment, and further down when it was confirmed that you were indeed basically spamming HN. That's probably not the outcome you were looking for.

[EDITED to fix a typo.]

On Wikipedia, the term for accounts like that is "meatpuppet".

Hi Mike,

no need for papers, I'm not the Internet police. :-P

It's just that your first comment came across as very suspicious, aside from not adding much to the discussion.

Even having established that it's not a sock puppet account per se, it is still convention around here to disclose affiliation (anonymously praising your brother's company qualifies for this).

Anyway, welcome to HN and best of luck with your startups to both of you.

Actually, with that knowledge, I find your initial comment dishonest. You have a personal interest in seeing the project succeed. That's different than someone with no personal interest in the project thinking it's cool.

Perhaps I was quick on the draw too, but your comment added nothing of substance - it's a "Me Too!" comment which is heavily discouraged here.

The attitude here is "Add value, or get downvoted."

To be honest, I've seen that sometimes people get downvoted anyway, but that's life.

Added in edit: And who's Ilya?

My first comment on HN, but thanks - I'll keep that in mind for future.

Always good to read are the guidelines: http://ycombinator.com/newsguidelines.html

Guys, do you have better things to do than comment on the appropriateness of other's comments? Spend that energy on discussing the actual subject matter of this post. From guidelines: http://ycombinator.com/newswelcome.html << Essentially there are two rules here: don't post or upvote crap links, and don't be rude or dumb in comment threads. ... Empty comments can be ok if they're positive. There's nothing wrong with submitting a comment saying just "Thanks." What we especially discourage are comments that are empty and negative—comments that are mere name-calling. >> That said, my participation in this thread is closed, have a good one.

We like it here. We try to preserve what it is we like about it. Sometimes that means explaining our culture to new users.

I assume Ilya Volodarsky, a cofounder: http://www.linkedin.com/in/ilyavolodarsky

Reply on news.ycombinator.com