I am one of the two PhD students that conducted this research with Boston Public Schools. I'll be glad to answer any question you may have.
Real-world problems are so much messier and more frustrating than the clean abstractions many of us choose to work with. But this is really meaningful work that should have concrete social benefit. We should all hope to do good as you have.
Do you think more user and stakeholder engagement would have helped the public accept your research? I have no idea if BPS has the bandwidth for supporting this type of engagement, but I imagine a lot more fears would be assuaged if parents could have some role in "co-creation" of the algorithm, rather than just accepting results.
A lot of research from the fields of citizen science, public understanding of technology, and user experience show that participatory design techniques can enhance acceptance. E.g., https://www.bmj.com/content/319/7212/774.short
I am sure more engagement is always a good thing! One of the tricky tasks of BPS is to make sure that this engagement is "balanced", i.e., everyone's voice is equally heard. And you can imagine how hard that may be in practice. For example, Boston Public Schools had this survey asking people for their preferences on the start times. But most of the answers from these survey came from schools with less economically disadvantaged families. One way to do it is to call the families directly or have a lot of local meetings. But it is still a complicated problem.
I think this project was useful because it started this debate and brought more information to the table. For example, I found that a lot of people did not know how much the start times affected the district costs (tens of millions of dollars every year), or how the schools times seemed to be unequitable.
And this Boston Globe article allows even more people to read about this issue and make their own opinion. Ultimately, a lot of people in the US want later start times for high schoolers. This work can provide some information about the ways to implement such a change, as well as the consequences.
That's a ~9 month notice?
Yes. This is what separates the pros from the not-pros in major systems design and operation. The transition plan is the major key to plan, not the algorithm (which plays the role of Shiny Object in this case).
It is lots harder, lots, to get smoothly from A to B than to simply big-bang B. It takes enormous attention to detail and thinking through of every angle, and when done well is a thing of wonder.
I think in this case your team was not well served by how it was tasked.
Thanks again for responding, and for your team's efforts in the design.
Can you talk about how you made this tradeoff? There's research that suggests high schoolers do better with later start times. But is there any research that backs up doing it at the expense of younger students? ie, is it possible that all students benefit from a later start time?
> Research findings suggest that changes occur in the “biological clock” during adolescence. As a result, teenagers have a natural tendency to fall asleep later and to wake up later. This is referred to as sleep phase delay.
> A correlation was found between subjects' melatonin secretion and their stage of development. The results indicated that melatonin onset occurs later in adolescents, making it difficult for them to go to sleep earlier at night. At the same time, the hormone “turns off” later in the morning, making it harder for them to wake up early (Carskadon et al., 1998, 1999).
It's like things are happening to the body during puberty.
Presumably, you are already doing that, by limiting the start/end times from 7am-10am, or something similar. In this case, instead of a global time restraint, there would be a local time change restraint +-1 hour for each school.
Also, it is an algorithmic challenge to find a "path" of good solutions that go step by step to the good final start times.
In my mind, this alleviates a lot of the “would have to change jobs this year” concerns, because the people making changes would ALREADY be making bigger changes (starting to take their kids to school), and the most extreme schedule changes would be deferred until the children on the existing schedule had aged out of that grade’s current schedule.
(Done enough optimization modeling to know this isn’t an easy path... but seems like it has some potential.)
Similarly, (maybe this was just the article dumbing down reality a bit, but it sounded like the transit expenditure vs parent happiness units were somewhat fungible. Any possibility of offsetting a bit of upfront expenditure with future savings of the more efficient (say from year N/2 onward) future state?
Another way to ask this question is: were there volunteer transit, ride share, or pre/post-school care incentives included in your algorithm, to balance the potential negative impacts to parents?
I guess volunteer transit is always a positive thing for the parents and the districts. The algorithm was just trying to fulfil the objectives given by the district (costs, later students, fairness...). The compensations are what should happen after the algorithm: what do you do with the saved money, how do you help the most impacted parents? These negative effects were expected to disappear with the years as new parents start to choose the schools after the start times have changed (and therefore choose schools that can accomodate their schedules).
[1] http://web.stanford.edu/~niederle/SFUSDBoardPresentationFeb....
Now I am sure we have done some communications mistakes. For example the algorithm does not "decide" the start times. You just give it a (long) list of preferences and objectives (e.g., less HS students starting early, more fairness across the population...), and then the algorithm generates thousands of solution that try to satisfy these objectives. Solutions are carefully reviewed, and the objective updated if we think we can do better.
So in the end, this research is just a tool, it does not solve all the problems, and in the end people still need to find a way to agree. Hopefully they have a little bit more information to do so!
So, was the real motivation here one of efficiency and cost-savings then?
That is a clear thorn they might have wanted to solve.
Is the added sleep beneficial to students, all other things equal? I'm sure it is, but was the real motivation a cost savings and the sleep deprivation a nice cover for the real motivation?
There has been a lot of research on high school students lack of sleep, and it seems that the school times do have a important impact on the students :
BPS is pretty forward-thinking, and lots of research has shown that high school students are better off with later start times. Don't dismiss this as the actual motivation.
Maybe some will remember this if money for student transportation comes up to ballot.
http://mitsloan.mit.edu/actionlearning/labs/a-lab.php
My company had pitched the students of the A-Lab a couple years ago, but unfortunately we didn't get picked. BPS was one of the others pitching the students at the same event.
In the meantime, there is some available information that is listed on my website: https://sebmart.github.io/media.html
I would argue that although it could have been more transparent, it is way more transparent than the status quo, where each individual school can try to pressure the district to get the time that they want, without considering the "big picture" and the impact on other schools/neighborhood/budget.
All decisions were made after a lot of group conversations with parents/staff/principals... But I am sure some things could have been better, in the end we were just the grad students trying to help :)