728 karma · joined November 27, 2010
I think reproducibility can be tackled--at least some journals (shameless plug--I'm a lowly associate editor on science advance) are strongly encouraging people include data/code with publications. I have reviewed papers in Nature Comput. Materials where people have included data/jupyter notebooks (not perfect, but a very good start). It would be great if funding agencies started adding more teeth to requirements on data sharing. However, many more groups are putting their code on Github.
There are groups that are using graph neural networks to understand statistical mechanics and microscopy. There are also a number of groups working on trying to automate synthesis (most of it is Gaussian process based, a handful of us are trying reinforcement learning--it's painful). On the theory side, there is work speeding up simulation efforts (ex. DFT functionals) as well as determining if models and experiment agree (Eun Ah Kim rocks!).
Outside of my field, there has been a push with Lagrangian/Hamiltonian NNs that is really cool in that you get interpretability for "free" when you encode physics into the structure of the network. Back to my field, Patrick Riley (Google) has played with this in the context of encoding symmetries in a material into the structure of NNs.
There are of course challenges. In some fields, there is a huge amount of data--in others, we have relatively small data, but rich models. There are questions on what are the correct representations to use. Not to mention the usual issues of trust/interpretability. There's also a question of talent given opportunities in industry.
If anyone is interested, we have one on data science in industry coming up: https://attendee.gotowebinar.com/register/604483936035643777...
You mentioned the question of what the A means. I wonder if just the act of working to get the A tells us something and that something may be more predictive of college success than the SAT/ACT.
Or, another thing to imagine looking at would be say AP exam scores. I am biased, but I would imagine these would be more predictive than SAT/ACT and probably more informative than GPA.
The question of change in major is interesting--but I think even the coarse question of say graduates/doesn't graduate would be interesting to see how it correlates with high school GPA (as compared to SAT/ACT) before delving into the more difficult cases that you rase.
Has anyone else found other studies? I would be shocked if there isn't some kind of meta-analysis. Before getting into any discussion about social issues, let's see whether the tests are actually predictive. Or, if it's just a cargo cult.
My intuition was that these tests (ACT/SAT) would help to normalize for different levels of schools, but I will go with data over intuition.
In some sense, this move by Harvard will allow for a natural experiment to see if there are actually changes in outcomes when GPAs are/are not considered.