gchallen

> Of course there are lots of rationalizations for why every program needs to be unique and special, but does it really benefit the students?

Of course not. What would benefit the students would be having a lot more standardization so that we compare ideas and approaches and determine what works. But the problem with standardized evaluation is that half of the programs suddenly discover that they aren't in the top half—as most of them had previously thought. This seems like more or less what happened to standardized testing in K–12 education.

But, in the context of a specific curriculum, having no idea what is happening and therefore no way to improve your bespoke curriculum is even worse than just deciding to do things your own way.

And it's worse than medicine, because at least they have some common metrics for what it means to be healthy. Whereas, faculty get to assign grades however they want! Imagine you ran a diet study where you both controlled the meals and got to reposition the numbers on the scale at will.

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place.

For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly standard and important indicator of program health. Next, break this down by various cohorts: What is our retention rate among women? And so on. Heck, frequently we can't even answer questions about the current gender ratio within our program—and this is something that has been a focus of our diversity efforts recently.

I've had people say with a straight face that we _cannot_ calculate retention because we don't know when students leave our program. But of course someone knows this! And I've been able to produce rough estimates even given the limited data that I have access to. But a lot of educational data is fairly siloed, and frequently the people assigned to perform these tasks don't have much training and tend to give up quickly.

I suspect that many departments just don't have anyone assigned to do even basic educational data analysis on a regular basis, and with access to enough data to run interesting reports. My department is in the process of creating a faculty leadership role around academic data analytics, but my sense is that this will be a very unusual position. (And don't worry—it'll be filled by a faculty member, and not a new administrator.)

And don't even get me started about student evaluations of teaching. Yes, we give a survey at the end of every semester and ask students whether they liked a particular course and professor. No, those answers have very little to do with how much they actually learned. Yes, we could measure learning in other better ways—success in downstream courses, for example. No, people don't tend to do that.

There's a lot of room for improvement here, just working with the data we already have. No need for additional "telemetric signals".

I suspect headcount actually makes the situation look better than it actually is. A lot of administrators also get paid a pretty large amount—more than many faculty, sometimes a lot more. So administrative spend may be at ratios even higher than the 3:1 administrator-to-faculty ratio quoted in the article.

FWIW, if you're interested in doing some data analysis of salaries from a large public R1 institution, I've parsed and published publicly-available academic professional salary data for the University of Illinois going back 15 years: https://github.com/gchallen/graybooker

Very cool idea! We actually built something very similar for the CS course I teach at Illinois. Example: https://cs125.cs.illinois.edu/lessons/asserting/#switch. We've built this on top of Ace, which allows us to deliver lessons that have a mix of text, playgrounds, and these interactive walkthroughs.

Overall I think that this is a great direction. Glad to see others working on it!

I used RabbitMQ to distribute messages between components of a distributed grading service that I wrote in Kotlin and deployed on Kubernetes.

My experiences were pretty mixed. Overall I found it to be more difficult than I would have wanted to get simple things to work. Part of this seems to be a problem with the Java library, which is not great. For example, IIRC you have to be really careful not to create the same queue twice, even with identical configurations, since the second time something blows up. At the end of the day just a simple fan-out configuration ends up involving a lot of somewhat-intricate code. It definitely does not Just Work (TM).

And then there was the bizarre hangs that I would experience during testing. I set up a Docker Compose configuration so that I could test the various parts of the system independently. It included one container running RabbitMQ to simulate the cluster we have running on our cloud.

Usually tests ran fine. But then, from time to time, the client would just hang trying to send a message through RabbitMQ. Unfortunately, again, the code you need to just run a basic configuration using RabbitMQ is complex enough that at first I was pretty sure that I had done something wrong. But after a few hours of increasing frustration I finally broke down and discovered that a simple test case that just sent a single message using code torn right out of the docs would hang. Forever. (Or, long enough that I gave up waiting.)

After a lot of digging I found the culprit. RabbitMQ will just take its ball and go home if the broker doesn't have enough disk space. Given that I use Docker heavily for a lot of projects, the amount available to new containers would vary a lot depending on what other data sets I had loaded or how recently I had run docker system prune.

I filed an issue about this, asking to have a better error message displayed when an attempt to send a message was made. The response was: there's already an error message, printed during startup. You didn't see it? No. I must have missed it among the hundreds of other lines of output that RabbitMQ spews when it starts.

Overall my favorite part of this story is that RabbitMQ chooses to start but refuse to send messages when low on disk space, when just crashing would be much more useful and make it much easier to pinpoint what was going on.

Anyway, I'm in the market for a simpler alternative that's Kotlin friendly.