Availability of cookies affects evaluation of teaching (2018)
pubmed.ncbi.nlm.nih.gov
pubmed.ncbi.nlm.nih.gov
I've known teachers who bring cookies on evaluation day. Another rule of thumb is that evaluations tend to favor teachers who give out easy grades.
The evaluations that my spouse received contained threats of violence.
> I've known ...
That is common. I had a friend who left a picture of their family on the desk.
I am sorry to hear about your spouse's evals. That stinks.
I at least read “later” here as “subsequent, following” classes, not classes which take place later in the day.
I work in education and this has been my long held belief; it is interesting to see good evidence for this effect.
Student evals measure their happiness; it is trivial to increase them by limiting their frustration (ie., the difficulty & depth of content).
The goal of education isn't short-term comfort, it is long-term competence.
For example, in my profession, I typically only have a few days with a group to rapidly onboard them in complex areas. It was my approach to end my session by considering all the issues the learners did not understand; and to review them again.
However, this causes them to be aware of the limitations of the learning, and to frustrate them again with whatever was too complex. And thereby evaluate me lower.
Now I am forced into a 'review' which convinces them they have learnt something (true), only because that feeling of accomplishment is better for the eval. It isnt better for them.
(1) You're better able to teach if you've learnt it recently (say, within 5 years). This is because you remember what it was like not knowing it, so can better communicate with someone who doesn't know it. e.g. "demonstrators" (often last year's students) often communicate better than lecturers.
(2) The paradox of excellent teaching is that students get less practice in struggling, working it out for themselves, finding other resources (peers, texts). They learn the material, but not how to learn.
That is, spoon feeding is a form of "disabling help", where doing something for someone that they could do for themselves deprives them of developing that capability.
Perhaps... extremely knowledgable but hopeless teachers are best.
I disagree with this conclusion and note that the cookies were provided during the course, not during the evaluation.
It seems obvious to me (and should be to anyone who has been in a class) that a course with cookies is a better experience that may indeed result in more knowledge transfer. So this would validate the SET.
I learn better when im not hungry.
There is also some research that suggests chewing helps learning https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.3467
The full article also seems to indicate cookies had no effect on test scores; see this Twitter thread, which is where I found this link.
Your experience of the same teaching materials could be substantially different in a class with cookies - in a more informal atmosphere, students are more likely to discuss the purpose of the materials and therefore have a better insight into their value.
(The atmosphere this creates is also likely to persist over multiple sessions).
> cookies are provided only in the session of the course where the evaluation is performed (note the phrasing "course session", not "course").
I agree that would be relevant, but the Methods as it is written contradicts this. It says cookies are provided during a "course session" and the evaluation happens "after the course" (the implication is that this is not during a course session)
https://pubmed.ncbi.nlm.nih.gov/29956364/
> The cookie groups were provided with 500 g of cookies at the beginning of the session; this corresponded to an over‐supply of cookies to avoid limiting students to taking one cookie only. The teachers were instructed to comment on the availability of the cookies only by saying: ‘I've brought some chocolate biscuits.’ Teachers were allowed to eat one chocolate cookie themselves to break any reservations about taking the first cookie. The control groups received no chocolate cookies.
> If students brought cookies or other sweets for their group, the teachers were instructed to report this to the study group.
> At the end of the session, all students were asked to fill in the anonymous end‐of‐session evaluation form. While the students answered the questions, the teachers were instructed to leave the room. The forms were collected in a ballot box. After all students had left the room, cookie consumption was determined by subtracting the weight of the remaining cookies from 500 g.
Maybe. Maybe it just gains favor and students don't worry about getting value from learning since they get value from the socializing.
ie., yes, every lesson must be a performance. but it must also be a challenging one.
learning is no a michael-bay movie, it is a stanely-kubric: a wonderful light show, yes -- but challenging, stressful, opaque -- and requiring your active participation.
What happens when you use student evals as guides to effectiveness is that learning becomes less challenging. (as evidenced here: http://faculty.econ.ucdavis.edu/faculty/scarrell/profqual2.p...)
I work in high-end commercial education, where this effect -- to me -- is quite clear. But its less obvious how to deal with it. Customer satisfaction is more important than education, to selling education.
Also I wonder if someone could draw the same conclusion that the environment being taught in (i.e. has cookies) is an important factor along with the actual material being taught.
In Table 3, they report the 95% confidence interval for the treatment condition (i.e., receiving cookies) as 0.67–11.62, so it seems that the difference between the two conditions is significant at the 5% level. Interestingly, the reported p-value here is "just" 0.02 instead of the <0.001 that they include in the abstract.
It would be interesting to have tested this during the time of fidget spinners or some other tchotchke, to remove the food element but that aside, I think a weakly confirming result would be confirming what we expect: signs of care and emotional engagement beyond the immediately neccessary improve mood, and so improve some outcomes.
There is a saying that positive reinforcement beats negative reinforcement but it comes with an implicit corollary: negative reinforcement beats nothing: That negative pupil teacher interaction reports exist with higher scores should not be surprising, because it is possible SOME people achieved higher scores, despite hating (and negatively rating) their teacher.
That's a separate claim not a corollary. In fact, all else unknown, positive beats negative is suggestive that negative is worse than neutral, which might be better than positive.
Does this finding "question validity of" recruitment process?
Or does it just (re)discover that recruitment process is imprecise as any other process that is based on human judgement?
The effect size here, 0.68, is pretty insane for a trivial change of this kind. (if accurate)
I think it is pretty clear, from independent research, that student evals capture their eval-time happiness and little else. This is just another piece of evidence to that effect.
An eval of this kind cannot really capture anything that useful. Esp. at a university, it would be better to ask people to eval their 101 prof.s at the end of their 201 courses -- ie., with sufficient distance to appreciate their impact on their learning.
A better survey would measure sleep, nutrition, and exercise levels of students across many campuses against performance of students.
It would also be a great thing to fund research of since ideally you'd want optimal schools.