Hacker News Reader

Comment by waqf | Hacker News Reader

The article doesn't seem to even consider the possibility of assessing students in some other way than through standardized homeworks which are easily copied.

For example, individual projects where everyone in the class is working on something different; or at the other extreme, proctored exams.

(Of course, neither of these systems is entirely free from cheating, but the barrier is higher.)

View on HN

For intro-level CS courses, neither one works all that well. You can do that in more advanced (but still undergraduate) courses, but for beginner-level you IMHO need the feedback loop of regular homework. It doesn't have to be strictly graded, but it basically has to be there and at least some enforcement for it actually being done (e.g. handing in homework is required, but it's not a problem if it is doesn't actually work).

Coming up with hundreds of different small projects to e.g. get people to understand pointers isn't very realistic, and if you only test them in exams you're missing critical feedback both for the teachers and the students before it is too late.

Assess the homework, give them a grade on it but don't have it factor into their final grade at all. Just let the exams / big projects take care of that.

If you're regularly failing your homework, it probably means you won't be ready for the exam that actually counts. Which should be enough feedback

I think me editing overlapped your comment: yes, that's an option, and I guess then you do not have to care about plagiarism so much. There probably still should be some effort against it though.

having been both a TA and a student, i think this idea is entirely too optimistic. "optional" problem sets can work well in graduate and advance undergraduate courses, but for the bulk of students in the intro/intermediate courses, homework that doesn't get graded for accuracy may as well not exist. i certainly didn't do it, and when the CS department decided to grade homework on completion only, i saw student attendance of my office hours immediately drop to zero.

ultimately, you cannot just expect college students to make reasoned decisions based on the way you set up the course. you need to arrange the course in a way that incentivizes them to keep up at every step.

That's the conventional wisdom in America, certainly. And yet my undergraduate degree (maths at Cambridge) was 100% assessed by end-of-year exams and it worked fine.^

Did I stay up with the classes? Well, not always — some of them I caught up during breaks. (That's the advantage of end-of-year not end-of-semester exams, too.) But did I know the material by exam time? Yes.

But look, more generally, you say you have to "incentivize" students to keep up but I think we agree that it's in their long-term interest to keep up with or without graded homework, so we're talking about behaviourist incentives not rationalist incentives here. And with that admitted you have to consider that there's broad scope for other ways to incentivize than through GPA consequences. (I mean, hand out candy, or pizza, or Pokémon, or porn, or whatever this year's students are into, you can probably think of a much better ideas than these.)

^Unless you were that one kid who had mono during finals week. But that's a different issue.

as a grader, it is already hard enough to keep up with the work that gets generated by thirty students attempting the same weekly homework assignment. i don't know how i could possibly grade thirty different projects in a week, let alone in any fair or reasonable way.

i've always wondered why they don't do something like select a random number of students and ask them to explain their work in an interview with the professor or TA.

i figure, even if they cheated, if they can give a satisfactory explanation, they probably learned the damn thing anyway. plus, you don't have to select even half the students before word gets around that they better understand the work they are turning in.

Add a grading section to each assignment with something like this: --------- Program is expected to maintain this interface <link to interface here> over stdin/stdout (or sockets, etc).

Students will pass when their program passes the grading/test suite. Test suite can be check ahead of time by uploading binary/zip/tarball to this location using their student id: <location here>. Detailed instructions <here>.

IIRC, this is what my university did 15 years ago. Not everyone had unique homework, but often the homework assignments were not all the same. I guess how unique they are depends on if you have some parameters in your homework generation or if you have a bunch of misc grad students to generate a few sets of homework that you can build up over time.

How might you grade style? IMO Don't. If you must, then save it for interactive review/lab sessions/office hours or style-specific spot checks (e.g. "At least once per semester your assignment will be additionally graded on program style according <insert guidelines here>. This will be added as additional points to your semester total.")

as a TA, i love when professors set up grading scripts for me, and as a student, i love knowing what grade i will get before turning in the assignment. unfortunately, you need the students to have the basic level of competency where they can actually write code to a testable interface or correctly use print statements to produce good enough output for a script to grade by parsing. in the course i currently TA, neither of these prerequisites are met. many students turn in programs that will not compile and i am expected to give generous partial credit if i can determine that they are "most of the way there".

i also have pedagogical concerns about providing the test script ahead of time. i suspect that it biases low effort students even more in the direction of "guess and check" rather than the ideal (imo) "reason and predict".

all that said, the approach you outlined is probably the best feasible approach for most cs courses.

The issue of incentivizing students to guess-and-check when providing the test scripts upfront is, IMO, fixed by making the students write the tests themselves. This paper explains it pretty well:

https://www.cs.tufts.edu/~nr/cs257/archive/stephen-edwards/a...

Essentially the students would write a test suite and the grading framework would grade based on

1) Code coverage when running the student's test cases against an instructor's reference solution

2) Correctness of output: running student's test cases on student's code and comparing with output from running those test cases on the reference solution

3) Number of test cases passed in student's test suite

Also from the paper:

"All three measures are taken on a 0%–100% scale, and the three components are simply multiplied together. As a result, the score in each dimension becomes a “cap” for the overall score—it is not possible for a student to do poorly in one dimension but do well overall. Also, the effect of the multiplication is that a student cannot accept so-so scores across the board. Instead, near-perfect performance in at least two dimensions should become the expected norm for students."

Students still get the benefit of knowing their grade when they submit, and as an added bonus students get more hands-on experience with test-driven development. Having the students write the tests themselves also increases the cost of mutating code until it just works.

this sounds like a great solution. i particularly like that it encourages careful thought about test cases.

It sounds like students who can't program need to be offered/forced into a 099-level class on whatever language or languages are being used for pedagogical purposes prior to actual CS classes. We don't let students who can't handle Algebra and Geometry take Calculus classes and we shouldn't suffer students who can't generate compiling code in CS classes. I made most of my money in college because such a class didn't exist.

Standardized homeworks as a simple check on learning with an individual or group project at the end has been my favorite way of learning.

Sure! But the method of learning doesn't have to be the method of assessment. In fact, as pointed out upthread, it shouldn't be.

When students are doing homework you want them to form collaborative study groups or freely consult any other source, if it helps them learn. Fear of plagiarism is antithetical to that.

I think a related problem is that many university educators have a pretty abismal understanding of education itself. For some of the, the notion that leaning tools shouldn't necessarily be assesment tools is rather suprising.

My recollection of CS-50/51 from decades ago is that the final project with the most value was an individual one. It seems infeasible to make all projects individualized, if you have a student/TA ratio greater than 3.

Reply on news.ycombinator.com