I think this was a good middle ground that forced you to think beyond the rigid parameters of the public tests, and discouraged things like “switch” statements.
Quality of code was also a component, as determined by a TA.
I think this was a good middle ground that forced you to think beyond the rigid parameters of the public tests, and discouraged things like “switch” statements.
Quality of code was also a component, as determined by a TA.
The system used to pull this all off is pretty interesting, I think. It's all open source: https://github.com/redkyn
I was also never brave enough to have my autograders submit grades directly to Canvas after the first time I did it. I confidently had it directly submit grades the first week during the first year I implemented autograders. Half the class forgot a shebang on their shell script, and the autograder happily gave them a 0.
For bonus fun times: Gitlab's runner timeout is pretty much nonexistent if you're using the shell runner. It just won't kill them if they take too long, so I had to write our own management system for runners. Silly university IT not support containers :(