acbart

Haven't posted much lately, but that's my blog. Mostly CS Education stuff.

I suppose it is vaguely terrifying.

> A suite is valid if it accepts (i.e., its assertions pass) all correct implementations... In order for a suite to be valid for all implementations of median, it must not include any assertions involving empty input lists. We can accurately identify such assertions as invalid by checking them against two correct implementations (henceforth wheats [24])... If a student asserts that implementations should produce an error on empty inputs, their suite will reject the wheat that produces 0 (and visa versa). Provided that the set of wheats completely exercises the space of underspecified behaviors permitted by the specification, accepting all wheats guarantees that a suite is valid and will accept all correct implementations.

> A suite is thorough if it rejects (i.e., its assertions do not pass) buggy implementations. We assess the thoroughness of a suite by running it against a curated set of buggy implementations (henceforth chaffs [24]). The thoroughness of a suite is measured as the proportion of chaffs it rejects. To assess test suites, the set of chaffs should include subtly buggy implementations. To assess examples, we take a different perspective: the set of chaffs should exercise logical misunderstandings that students are likely to make. For instance, to assess the thoroughness of examples for median, the set of chaffs could include implementations of mean and mode.

I want to see this used in more curricula and tools. I need to see if there's been any follow-up on this research and learn how it's gone.

[1]: https://github.com/blockpy-edu/skulpt/blob/master/src/lib/ma...

That said, the HtDP curriculum that they were involved in, and led to Bootstrap and a host of their other projects, was not grounded in any kind of data collection or treated as an empirical research project. Smart people with good intentions built something and threw it into a classroom. Intentionally so, by design [1].

Their subsequent work with Pyret is much better grounded and well-informed, and I've been absolutely fascinated by it. Their data science curriculum delights me, and I have often thought that if I had more time I'd translate it to Python. I think their more recent proposal about a Table Abstract Data Type is just inspired. But we are still VERY far from ever proving things like, "Racket is easier for students to learn". I'm fairly sure it's not even worth doing.

I appreciate the original article was just one student's perspective. It's a perspective I don't entirely disagree with. Nonetheless, they make interesting claims not backed by any kind of published, empirical data. They hypothesize that Racket is virtuous to learn because of how it presents recursion, and because of its simplistic syntax. Cool theory, but not proven using Randomized Controlled Studies. I mean, most things in CS Ed aren't, so I don't think it's a surprising thing to point out. But that doesn't mean we should just sit back and accept these claims.

Personally, I have seen the HtDP curriculum do damage. I have also seen it firsthand do a lot of good. My Racket relationship is complicated [2]. The author makes a great point of the potential value of these models, and also mentions how this whole debate is sort of pointless. I believe that at the end of the day, the CS1 language is simultaneously very important and yet somehow unimportant - it's all just the psychology of the people involved. That, too, is an unsubstantiated claim.

But please, tell me more about how I have not done my due diligence. Would you like to cite some actual RCT papers that back up the claims founding all this stuff? Or do you want to keep citing textbooks like that somehow proves something?

[1] https://felleisen.org/matthias/Thoughts/Measuring_education.... [2] https://github.com/acbart/myracketrelationshipiscomplicated....

What the hell kind of academic culture is this? Maybe in graduate research, if I expected a research assistant to know how to do a task... but certainly not true in an undergraduate course! If they can't do the assignment, that is the perfect time for them to come to me and learn from me how to do the assignment. That's like, the whole point of my job as a teacher is to help people learn. Yes, I make assignments too, and people underestimate how difficult that part of the job is, but helping people figure things out is like my #1 job. It's sad to hear that your experiences (and apparently many other commenters) are something very different?

Also, don't gatekeep the field, unless you have a citation that proves that modern teaching techniques have really led to a decrease in quality of software engineers. I really doubt you have any concrete measure you can point to that won't be anecdotal, but if you have one I am interested in hearing why you think you can strongly believe that folks have gotten wore over time because we got better at teaching.

[1] https://www.amazon.com/Dont-Shoot-Dog-Teaching-Training/dp/0...