Comment by orangeoxidation | Hacker News Reader

Comment by orangeoxidation | Hacker News Reader

orangeoxidation

Do they not? "At scale" is somehow often synonymous with "with less effort and care per instance", but that doesn't have to be the case. Things work worse if you scale only one part of the equation (the "customers"), but do not scale the other side appropriately (the "service provider"). That should be obvious.

> Text campaigns are most likely to be effective when they:

    Are sent by a school and even staff member who students know and trust
    Add meaningful and novel value above and beyond existing outreach and support efforts like giving students real-time information on the status of their financial aid applications
    Provide content that is personalized to the student and where they are in the college application and financial aid process

Having a new org sending out mass messages to all students is not scaling The described. It is something different.

Scaling it would be convincing more school staff and schools to make a similar effort.

The positive effect was not caused by "getting a message", it was caused by "getting personal advice from someone you know". They scaled the wrong thing.

Things also "work" if you are able to scale down a cost (employment of human labor such as teachers and counselors) while maintaining or scaling up a source of revenue (student enrollment.)

I worked for a company that offered this service (among other data-y things.) It employed lots of bright-eyed young people who were extremely happy they had found a tech job where they could do good, helping struggling students succeed at college. It was a dream job. However, the information we heard trickling in from the customers (public universities and community colleges) was pretty demotivational. They ones that were shrinking were panicked about their future existence, and the ones that were growing were basically like, yeah, hiring a proportional number of front-line staff to interact with students like in the past would just be silly. Nobody is going to do that. Run along, idealistic tech people, save these poor kids with analytics and text messages, because we're not paying to do it the old-fashioned way.

This really gets at the heart of why I have very little allegiance and essentially no desire to donate to my alma mater: Why would I care about or donate to a business that really doesn't care about me individually besides $$$ signs? Sure, I patronized them for four years of my life, but I've done the same with In & Out for longer and you don't see me sending them a check for nothing. Much like In & Out, the positive memories of my alma mater were more so about the people I was with than the location or business proposition itself. Ok, I'm ending this metaphor before it becomes too contrived...

Better to donate to Khan Academy. Can even enjoy an In N Out burger while doing so.

Khan academy is absolutely fantastic. However, it's only a step above giving students free textbooks or other self-learning resources: It mostly work for students that are already self-motivating. This tends to be a minority of students, and most require some degree of a human support network to succeed.

Realistically you donate so that the school has a bigger budget and more opportunity to gain prestige making your diploma more valuable. That's really only relevant to a small subset of the population and rapidly shrinking. Four decades ago degrees had meaning but now they are largely participation trophies that come at great expense.

Things also "work" if you are able to scale down a cost (employment of human labor such as teachers and counselors) while maintaining or scaling up a source of revenue (student enrollment.)

This is an unfortunate truth. Schools grow, don't scale human labor to accommodate additional students, overall student performance & dropouts increase, and their answer is to recruit more students in a downward spiral. Never mind the fact that it costs more money to do this than simply hiring 2 or 3 more advisors to do real human outreach. I saw this cycle over the course of 4 or 5 years where I work. After providing hard analysis of this pattern each year, I, working with colleagues on the academic side of things, managed to push for those advising resources and the results have more than paid for it.

The problem is that many of the idealistic tech people are only looking at the problem from the tech point of view. I have resisted efforts over the years to assimilate me into a central IT division because the viewpoint I get through being embedded in an operational division strongly informs my analytical efforts. I see the business cycle, I talk to advisors, I construct & analyze surveys that mix quantitative and qualitative factors-- there is no detail of the student experience that is too small for me to ignore. It hammers home the point that analysis & tech can only help focus the efforts of human labor, not replace it.

Some problems can be improved with a simple contact point.

I have a colleague who works in this space in a research capacity. They were looking at use of different systems to measure behavior and identify behaviors that are correlated with poor performance.

In one scenario, they surveyed a cohort who was leaving and entering the dorm after 1AM, resulting in poor AM class attendance. 40% of the responses were essentially: “I went to meet my [girl/boy]friend and have sex.”

Getting a text message to tell that kid to get his beauty rest isn’t going to work. Just treat people like adults, imo.

Some problems can be improved with a simple contact point

Absolutely. Most schools don't employ enough advisors to have sit-down appointments with every student every semester. However, I did see a school implement a policy where 1st semester students-- a higher risk population-- were required to have a sit down like that before they were allowed to register for the next semester. That single effort resulted in a 15% decrease in 1st-year dropouts.

Makes sense.

I went to a big public university where they had two freshman tracks, a "college within the university" where, if you knew to sign up, you'd be in a cohort of students to meet some general education requirements and would get access to faculty motivated to help you. The problem was that the kids who needed the least help tended to discover this program.

The "normal" path was being dumped into the University bureaucracy blind, and you had to learn to sink or swim. It was tough -- the registrar in those days was an experience like DMV. I'm pretty sure the system was devised as a funnel to disqualify people for popular majors like Business, Computer Science, etc.

the kids who needed the least help tended to discover this program

Heck yeah did you hit the nail on the head. I've analyzed the data over a period of years for which students utilize tutoring services: They very much tend to be students that were already doing alright before they started getting tutoring.

Conscientious self-motivated students are precisely the ones most likely to seek out help on their own. As a result, for students that go on academic probation we now require them to attend a certain amount of tutoring hours each semester, and the software we use to manage advising caseloads allows faculty to raise a flag for students performing poorly so that advisors can pro-actively reach out and direct them to tutoring resources.

This has led to a huge increase in tutoring resource utilization, but unfortunately they are now at the limits of their capacity. Our next hurdle is to analyze the academic outcomes of students using those resources, which will hopefully show increased performance & allow us to petition for more support services.

Fortunately our leadership is not completely short-sighted on these issues, but as obvious as "more one-on-one advising & support" would seem, they (not too unreasonably) still demand very concrete evidence to justify additional expenditures. However it would be nice if they viewed customer service of this sort as the profit center it is, rather than a cost center. The problem is that things like the benefits of quality tutoring of freshmen may not yield tangible results for a few years, so things move slowly.

It might help to send a text at 9pm reminding them to go attend to their affairs earlier so they can get beauty rest.

Excellent point. It’s a problem with operational definitions which plagues a lot of research.

But “scale” in this context implies economic efficiency. You can’t give every kid a full time counselor. You can write a script to send sms messages.

One question for the tech community, to what extent can advances in ai and other technology impact this question of scale/efficiency. If the student doesn’t know his counselor is a bot, maybe the nudges become more impactful.

Reply on news.ycombinator.com