One of the great challenges facing us in this 21st century is that we're going to have to come face-to-face with some of our casual assumptions about the world, now that we've got machines that can manifest them, and learn how to start clearly distinguishing between goals and methods.

Usually I'm talking about this in the context of laws, one of the easiest examples of which is the speed limit. The purpose of the speed limit is not to ensure that all cars are going 65 miles per hour or less. That is, on its own, a pointless goal. The purpose of a speed limit is to keep people safe, and to give police a mechanism to exert pressures to keep people safe. That's why if the speed limit is 65 and you're driving 40 on ice and slip off the road, you still may get a speed citation even though you're "25 under"; because the law is not really about speed. It's why you might cruise by a cop when everyone around you is doing 75, and the cop doesn't simply leap out and grab the first one they see, because the law is not about speed so much as it is about safety.

In this case, consider classroom expectations. Obviously, if we have a machine that can perfectly gauge how much attention a student is paying to the teacher, bigger numbers are better than smaller numbers, right? The student who is 100% engaged is clearly better than the one 95% engaged, who is better than the student 60% engaged, who is better than the student 5% engaged. But what are you going to do with those numbers? 100% almost certainly means "this student has figured out how to game the system somehow". 95% is probably still a bit high, but they may be a student who is just at the limit of their ability and learning. Meanwhile, 60% may be the one who mastered the material last week and is currently bored.

In theory one could hope that these numbers would be examined holistically, and compared to a control group against the final outcomes of the education process. In reality, ha! You've got these bright shiny concrete numbers, and you've got them now, and they will overshadow everything else. And in general, numbers should be larger rather than smaller, and the whole system can't hardly help but set all the goals too high. And you've got teachers looking at the numbers, and principles looking at the numbers for the school, and the school superintendent looking at the numbers across schools, and the regional government looking at the superintendant's numbers, at every step of the way the goals will creep up because Numbers should be Bigger.

Buckle in for some decades of this sort of thing before we finally become wise enough to figure out how to use these systems, or, perhaps, learn not to use these systems.