When A Game Designer teaches a College Course: No Grading, just Levelling Up | Hacker News Reader

When A Game Designer teaches a College Course: No Grading, just Levelling Up

gamingtheclassroom.wordpress.com

142 pointsbiggitybones49 comments

All the teacher did was change the names of things; the underlying structure of the class is disappointingly standard.

It may seem like each assignment grants the maximum possible XP (which would be a different, though probably broken, grading model), but not if you read this: "Grading is rigorous. Spelling, grammar and punctuation must be proofed. Points will be deducted otherwise."

But the main difference is the perception of the students. It may be the same underlying structure, but it's from a progression standpoint as opposed to the traditional back and forth of grading.

This TEDxUniPittsburgh talk speaks to this concept: http://www.youtube.com/watch?v=0tg55pdNMxw&feature=relat...

Quote: "This has a number of powerful psychological motivators. One of them is no matter what you do in the class, anything you do is forward motion. As opposed to a normal class, when you get a bad grade or something, it takes you back a step which is very demoralizing. Game designers would never put that in a game, because they know people hate it."

"...it takes you back a step which is very demoralizing. Game designers would never put that in a game, because they know people hate it."

I agree that people hate this (I know I do, at least), but game designers put that in games all of the time, regardless:

You just died ten minutes into the level with the finish line in sight--start over again at the beginning of the level.

You are only allowed to save at particular save points. These are spaced out just far enough that you are likely to lose significant amounts of progress every time you die.

The type is the same, but the implementation is wildly different these days.

Save points in particular are far more numerous and well-placed in today's games. Notably, challenges are now intentionally separated by save points, so that a single skill-type is in focus during each unit.

That way someone who has trouble with, say, a vehicle section won't fail and be faced with five to ten minutes of run-and-gun before they can even attempt (and thus practice) the vehicle portion again.

Essentially, that's precisely what the OP's approach to education is: adding more save points and breaking up the challenges better by type: solo vs group, oral presentation vs written proposal, etc.

Further, failure penalties are only really used in games where the player has an open-ended amount of time to defeat the challenge, precisely to encourage learning and/or mastering of a technique or skill. In a setting where the player has a time limit (such as in education) it would be unnecessary and inappropriate to additionally penalize failure as "repetition + luck" wouldn't be a valid strategy in the first place.

Remember that save points / levels were often a technical limitation

Mass Effect 2 auto-saves seamlessly before every battle, before most non-battle challenges, and at regular intervals when navigating or collecting resources. It even saves at different stages of boss battles. You hardly ever lose progress--yet it's a runaway success, both critically and commercially.

In my experience as a TA, the worst obstacle in teaching someone something is when they think they already know it. There's nothing that can replace a bad grade and the consequent soul searching in these situations. A disagreeable experience for the moment can sometimes benefit you much more in the long run.

Ah but that assumes that grades being awarded correlates to knowledge or understanding. All through uni I would do really well in assignments as I would actually love learning and doing the stuff. Come exams and it was one huge cram session. Pretty silly to see your entire grade hinge so much one pointless and worse out dated activity.

After a while we just realised uni was just another "tickbox" for the next step - work. Only after you get a job would you realise the total discontinuity between uni (or the different modes of learning) and reality.

Do you have any arguments why exams would be pointless and dated out? From meeting lots of students, I found out that their understanding of the subject correlated much better with exam than assignment grades. The way I see it, there are two main proofs that you've really grokked something:

1. You can recall and use it under pressure, without the luxury of consulting books and friends.

2. You can use it in a creative manner, to solve new problems.

Only exams can furnish such proofs. The exam system is not perfect, but the best we have (and incomparably less corrupt than the assignment system). The proper way to see things is that assignments are rehearsals for the exam; and the student who never turns in a homework but aces the exam well deserves his/her A-.

I agree with you until your last sentence. Especially in upper division courses, the material is more complicated and problems that really test understanding are too ambitious for an exam. This means that if you want to see the student solve interesting problems, they have to complete problem sets or a project. (The corruption of the assignment system means that you're still stuck with exams until the students reach a certain point, though.)

Of course, my experience is only with math courses, and your mileage may vary.

Sorry I should have qualified that with some kind of research. When I say my experience, I mean myself and a few dozen fellow students I had spoken to. I understand that that does not constitute any way close to a majority.

Our experiences were that with the assignments we had undertaken we felt enriched doing the research and learning about the things that needed to be done. Exam time was inevitably cram time. Pure short term memory work (coupled with going over past exams - which coincidentally were templates for the current exams).

Now both your arguments about using something under pressure and doing so creatively would be fair, if it was not the case that most of our exams were based on previous years' ones.

I do agree with you about exams being less corrupt. But I am talking about students who actually enjoy doing their assignments and the whole research process (and students who learns for the sake of learning rather than doing well for exams).

I absolutely agree with #2; using knowledge in a creative manner to solve new problems is a perfect demonstration that you have grokked something. #1, however, just proves the ability to cram, which is frequently the same as cram-and-forget.

The forward motion idea is very interesting. I wonder how it plays out in a student's mind. A main issue with regular grading is that a lot of students end up caring more about the grade and their performance rather than what they are learning. It sounds like the forward-motion model is a better representation of the learning process. Dropping from an A to a B doesn't cause you to lose what you have learned, while being up 10,000 XP points (B) instead of 12,000 XP (A) is still up 10,000. Is this differentiator enough to take focus off the grade and cause more students to focus on learning?

I guess an important question could be: What would the feedback from a teacher mean to a 10,000 XP student who is desiring an extra 2,000 XP?

Does anyone know of a good resource to read on game design? I'm not interested in the programming aspects, but on the psychological mechanics game designers use to make games fun (or, if you're Zynga, addictive).

I don't know any specifically, but the syllabus has required/suggested reading that would probably be a good place to start:

Required Text

Designing Virtual Worlds. Richard Bartle.

Character Development and Storytelling for Games. Lee Sheldon.

Suggested Reading

Developing Online Games. Mulligan and Petrovsky.

Massively Multiplayer Game Development. Thor Alexander et al.

Synthetic Worlds. Edward Castronova.

Community Building on the Web. Amy Jo Kim

My Tiny Life: Crime and Passion in a Virtual World. Julian Dibbell

A Theory of Fun. Raph Koster

Flow: The Psychology of Optimal Experience. Mihaly Csikszentmihalyi.

Good idea, thank you.

I took a course in high school that used that grading model before (minus the gaming terms ofc). It was an online, at-your-own-pace class in one of those subjects I find interesting but would absolutely hate to learn about in a traditional classroom environment. The grading scheme was indeed awesome and motivating. Completing the next exam/assignment ALWAYS meant your grade went UP, so let's do it now and see what grade I have then! IIRC I finished the course far before the end of the term, which was an additional reward as it allowed me lots of goof-off time in the computer labs.

Of course, it can be a bit awkward to explain the F on your first interim report card, but if this kind of grading became standard that wouldn't be a problem. :)

Oh, I thought I should also mention that the class started at 0% and you could only reach the top grade if you got 100% on all quizzes and assignments, so it wasn't quite the hippie "everyone's a winner" deal some people might be thinking. Your current displayed grade was simply whatever you would get if you didn't do any more work until the end of the term. Also, there were many exams instead of just one big one at the end. I did do quite well in the course, so I wasn't just motivated to rush through the material, just motivated not to laze around instead of doing work.

Calculating the grades in that way just seems to make the interim grades seem more meaningful. When you have 35%, that means you have mastered 35% of the course material. Traditionally, if you have 73% at some point in the term... what does that mean in concrete terms? It doesn't mean you've mastered 73% of the course material, not even "so far", because chances are good that you've only received dinky assignments, labs, quizzes, etc., for a large portion of the material you're supposed to have learned so far. You won't be truly tested for mastery on chapter 1 until the midterm or sometimes even the final.

>All the teacher did was change the names of things; the underlying structure of the class is disappointingly standard.

The name of a thing is an integral part of the thing. Often, solving a problem means reformulating the problem in different terms.

The name we use for things, how it's displayed, is often very critical. While the underlying structure might be standard, it's how the underlying structure is changed and displayed to the end user.

Take any FPS, and it can be broken down to a simple point and click. The underlying actions are essentially move the cursor to a spot and click. It's everything else that makes it a game.

So while I agree, that he didn't change the underlying structure too much, how that structure is displayed is what makes it special.

(I got three responses of this ilk, so I hope no one will be offended if I just respond to the topmost one.)

I agree very much with your point in general. However, in this case, if I were a student in the class, I don't think this scheme would really do anything for me. I think I'd have the same reaction I expressed in my comment - saying "XP" instead of "points" or "level" instead of "grade" doesn't change the work I'm going to have to do or the effort I'm going to put into it.

> if I were a student in the class, I don't think this scheme would really do anything for me.

Maybe it would. Maybe it wouldn't. Whether it does or not is irrelevant. It's whether it get's more people to put effort into a system they normally wouldn't (something that is proven time and time again). Branding something as X might not be important to you, but for the vast majority of people, that brand categorizes it in their mind.

It's easy to dismiss this, but I think if you look at what choices you make in your life, you'll find areas where the brand something is given impacts your choices, even in the slightest ways.

Yeah, I came in expecting something like a discussion of a grading system with options A, B, Incomplete and what effect it's had on students' learning/progress.

One difference is that you get a small amount of points for a large amount of things, rather than just the final.

I've had plenty of classes that worked that way. CS228 at Stanford has weekly quizzes; CS148 has weekly projects (and two exams). 154 has weekly problem sets (as well as two exams). 221 has 4 projects and 4 problem sets and a midterm. And so on.

I teach computer science at the high school level. Inspired by Dr. Sheldon, I switched my classes to use points instead of traditional grades, but I went further than he has.

I'm only halfway through the year, but so far things are very positive. Students seem to be learning more and covering more material on average. (I'll write up a blog post about the experience this summer.)

Teach, would love to hear the details!

Okay, here are the basics. Sorry for the delayed response; I'm used to reddit with its handy orangered envelope....

For the past dozen years, I've taught in a very traditional way. I'd give a lecture on a programming concept, then assign several small programming assignments of increasing difficulty to practice the new concept. Once most of the class is fine with the new thing, move on. Rinse, repeat.

This year from day one I had all 130 assignments up on a class web page, each worth anywhere from 5 points to 400 points (most in the 30-100 range, though). I spent a couple of days teaching them a "hello world" Java program and how to compile with javac on the command-line. Then I said, "By the end of the first grading cycle (13 class days, each 90 minutes) you need 250 points to earn an "S rank" (100%) on your report card. It's 190 points for an A (90%), 120 points for a B, and 60 points for a C. Below 60 points is failing. Do any assignments you want in any order to earn the points, although doing them all in order is probably smartest. Go."

Students turn in the completed assignments into a digital dropbox, and I grade them daily, giving each assignment between 0 points and its max. If a kid makes less than about 95% of the possible points on any given assignment, I conference with him to explain what he's missing, and he'll redo it and turn it in for full credit.

I added in some LearnPythonTheHardWay-style assignments as the first couple of assignments of each new topic, so they can type in and mess with some already-working code that demonstrates the new thing.

I only give class-wide lectures when there are several students all stuck in the same place. Otherwise I just help the kids one-on-one or refer them to my slide decks or Google.

Points are cumulative, so for the current grading period, students need to have a TOTAL of 3300 points for an S-rank, 2600 points for an A, 1800 for a B and 1200 points to pass.

Some students have 4000+ points already and are on pace to complete two years' worth of curriculum in their first year (and sit for the Advanced Placement test a year early). Others will be fortunate to understand if statements, loops and functions by May.

Everybody's in a different place and it's fairly chaos in here all day, but every student is learning something and I love it.

I've been thinking about this idea a lot lately - I think gamifying a single course could produce a modest boost in performance, but it seems like this particular implementation, as endtime pointed out, simply changes some names.

I think this would be a way cool system around which you could base secondary education. But instead of the generic game parallels, you could treat the four years of high school as a sort of mash of character creation and early game exploration. If you had 5 lines of progression (Math, Science, Social Sciences, Art, Language) that went from level 1-10, each requiring passing exams, preparing presentations, or completing projects, you could allow students to naturally find the work that pushes them personally.

The one big thing I think is necessary to really see the benefit of a game-like system in education is the ability to try stuff multiple times.

Consider an alternate version of angry birds. In this alternate version, there are 150 levels, many of which mirror the fun and challenging levels we know and love in the original. The 150 levels are in 15 worlds, each with 5 practice levels and 5 "test" levels. After playing the 5 practice levels as much as you want, you can play the 5 test levels all in a row, precisely once. However you do, that's how you did. That's your score.

Not a whole lot of replayability there. While it can be necessary (especially at the university level) to distinguish between those who are and are not competant in a given field, I think game mechanics can teach us a whole lot more about how to make people want to achieve.

Additional thoughts:

- Unlockables could be really exciting i.e. you have silent study hall until you hit level 4 in any subject, at which point you gain access to the study lounge; if you're level 10 in at least 1 area, you can apply for off-campus lunch

- Incentives for tutoring would be cool. (Think "Prestige Levels")

- DATA! Tracking these things would give unprecedented amounts of data that could provide a lot of insight into the way students are progressing through the material.

- For this to really work, you would have to develop a new system for scheduling courses in high school that more resembled a college. This is not a bad thing. I think giving high school kids a bit more autonomy in deciding how and what they go about learning with their time will produce more well-adjusted, mature adults.

- The specifics of how to implement a lot of these systems are less difficult than you think.

This system and our current system face the same fundamental issue: how do you determine that a student has 'leveled up'? If what gives you 'experience' is flawed, it won't really matter whether a student is getting a level or a letter grade.

Also, if you want to encourage students to try stuff multiple times, you'd have to make sure 'not leveling up' doesn't equate to 'staying back a grade'. Students won't be encouraged to really learn what they need to, but just to get through things as quickly as possible, because Harvard only accepts people who are level 10 in math and science.

Obviously admission requirements will have to change.

My understanding of levels is like the Euler challenge. http://projecteuler.net/index.php?section=problems&page=...

For example, leveling up in Calculus would be about solving problems, putting calculus into practice, writing a project using learnt concepts.

Determining if a student has leveled up is easy, they just do. They solve the problem they're given, they complete their projects.

-------------

The idea of customized education, choosing levels and perks and having a clear organization of how things are connected e.g how many levels do I have to complete before I can do project X or understand concept Y

Levels can help understand associations between subjects. Student would build mental models of how things are connected instead of applying formulas on equations like calculators.

I never understood why some subjects were more important than others. You could live your entire life without understanding physics or economics.

When it comes to education, I would rather take choice and exploration over enforcement and standardization.

PS: Reply to your comment is the first part. The rest is just my plug

> Obviously admission requirements will have to change.

This is where the focus needs to be. Big colleges get an enormous amount of applications. They do not have the man power to deeply analyze each candidate. So, they use metrics to filter candidates, such as required course work and a minimum GPA. The current high school curriculum is largely dictated by this, and I don't see that changing with your system because the incentives are the same, whether you call it 'passing an AP calculus test' or 'becoming a level 9 Math student'.

> For example, leveling up in Calculus would be about solving problems, putting calculus into practice, writing a project using learnt concepts.

This isn't that different from how schools work today. How was your grade decided in Calculus? By solving calculus problems. You don't necessarily put it into practice, or write a project, but teachers could certainly assign those things to you. Many science classes give you open ended science projects.

I agree with your intent, and I like you way of thinking, I just don't see it doing away with enforcement and standardization. Yet.

The change in perspective will probably be enough to provide some positive psychological benefits, but I think doing some actual restructuring would help, as well.

In most games where you level up, you get as many chances as you need. If you fail to defeat a mob the first time around, you don't forever lose the possibility of reaching the highest level. You just have to try again.

But, in this grading system, as in every traditional one, you only get one shot at each task, and if you score less than optimally on any one, your maximum possible level decreases.

That's a big departure from the usual mechanics of games, and one that I think could have a significant impact on morale.

To sum up:

In games like WoW, you can win through sheer playtime and repetition.

In real life, such as school, you must apply intelligence.

There are many guilds in WoW who beat a boss by mindlessly throwing themselves at it over and over until, by luck, they win.

This approach is not desired by the university.

> In games like WoW, you can win through sheer playtime and repetition. In real life, such as school, you must apply intelligence.

Effortless recall is one of the foundations for reasoning fluently about concepts. And one of the most efficient ways to build that recall is repetition.

I'm not dissing repetition as a method to facilitate memory.

In WoW, people repeat the same boss over and over without learning anything or changing the approach until through luck they win. This is not possible with a school exam because you only get one chance, and even in the event that you can repeat it, the exam will be changed.

Not so in videogames.

So like most analogies, it leaks.

Actually the WoW approach has more to do with a different form of learning -- variable interval reinforcement ratios -- which has been canvassed at HN before.

I like the approach for that class, although it looks like a terrible experience curve for a game; you have to grind for weeks to reach Level 2, but then most of the subsequent levels are achieved relatively rapidly. Also you could hit a ceiling where it's no longer possible to achieve Level 12 because you missed out on XP earlier. Fun, but the metaphor doesn't fit quite well enough for me.

Intermediate work is used as a way to get students to perform work as the semester progresses, rather than trying to do it all at the end.

There's lots of reasons why. For example:

* Memory formation works best with spaced repetition.

* Students will just cram for an exam if you let them, which won't form long term recall.

* Spreads the assessment workload over the semester.

Actually, spaced repetition with game mechanics might work best. I was looking at a spaced repetition idea for a thesis ... a badges/levelling up thing might make it easier to pitch to students.

I like the idea implicitly communicated that there's almost a shopping list of assignments, and each one carries an XP value; if you're not doing so well, then with that kind of a system you can correct that by working harder to learn the subject matter. Obviously, there's problems, such as enabling a brute force strategy for getting good grades, but I'd be interested to see how well a quest list approach to grading would work in comparison to a sequential assignment style.

I woke up from a dream the other morning and could still see a mini-map of my house in the upper right corner of my vision. There were little blue exclamation points for all my daily quests. Brush teeth, take shower, make breakfast, etc. I was curious who gives out the quests, what kind of rep I'm farming and what the full quest text said, but then I was fully awake and it was gone. It's good to see that a college course can be similarly rewritten.

Maybe it's time to start looking for better quests than "brush your teeth".

My high school sociology teacher implemented a point-based system that everyone but a few complainers loved. (And so he was forced for the next year to axe it or quit.) It was simple, clear, harmful to procrastination yet beneficial to laziness at the same time. Everyone starts at 0, an F, you optionally do various assignments throughout the term and gain points, and if at the end you have enough points you can cruise. Oh, and points over 100 rolled over to the next term.

It is only a matter of time before the first powerleveling services become available for this class.

This is the guys Jesse Schell talks about in his Dice10 talk

http://g4tv.com/videos/44277/dice-2010-design-outside-the-bo...

Best part:

Leeroy Jenkins 0pts.

LOL

[ref] https://secure.wikimedia.org/wikipedia/en/wiki/Leeroy_Jenkin...

Sweet that you shared that link using secure.wikimedia.org. I'm glad that others are using HTTPS where they can!

Too bad that I was apparently downvoted for the rest of the post.

I still like the basic premise -- leveling up etc.

Reply on news.ycombinator.com