0d9eooo

I think this is kind of the tip of the iceberg, a general reflection on biomedical research and academics. Not that this is how most researchers act, but in the sense that it's not unusual for the field as a whole to encounter this. I mean, if this happens in the Lancet and with the WHO with something with worldwide significant consequences, what do people think is happening with stuff with less scrutiny?

It also is a twist on all the calls for the importance of rigorous peer-reviewed studies, as if the preprint flood is just an example of why we need traditional academic structures. Here's a case where we needed the traditional academic structure and it failed miserably, even when prestigious institutions.

The irony is that this is being caught, but outside the confines of the traditional peer review. That is, there is peer review, but in the general public of scientists. So which is better? It seems like open publication and transparency is key, more so than the traditional structures per se.

Unfortunately, I sympathize with the anti-mask requirement sentiment, even though I wear one in indoor public spaces like grocers. I don't think people should be shamed or that they should be mandated.

When I read into it it's not always clear masks work well to limit egress of virus, and I do think the scientific community is to blame to some extent for presenting things as "masks are scientific and lack of masks is not." This approach might only worsen things relative to a message of "we don't really know but it's a good idea to be prudent."

Linked scientific articles advocating masks I find are often misleading. They often are based on other viruses that potentially have other transmission properties, or make questionable assumptions, like that the mask is an n95 mask or is wet cloth. When I have found actual studies of SARS-CoV-2 with dry fabric of the sort most masks are made of, the masks work, but not nearly as well as you might think.

The problems with masks have really been underscored for me practically too. I've tried a couple of masks now, fit to my face, and I've run into a telling problem: when I breathe in mildly colder air, my breath shoots out from underneath my mask and fogs my glasses immensely. Not only is it evident in a very visible way that when I'm wearing the mask it's not actually blocking airflow very well, but then I have to constantly adjust things because I can't see, and sometimes have to resort to removing the mask entirely.

I still wear the mask and the fogging has gotten better as it's warmed, but my overall impression is that there are good reasons people might not wear masks, and I think the evidence in favor of them is much weaker and less rigorous than is often acknowledged. I think if the scientific community were being honest, they'd say they don't work great, but it's something, and every bit can count. That message seems to be enough; it would go a long way.

Or maybe you just don't want to play the game anymore.

The hospitals losing money part is complicated because most of them are losing money because of lost elective procedures. In many hospitals, especially private ones, those elective procedures are the main source of profit and underwrite the acute/ICU cases. So you could have a hospital that's at occupancy with intensive/acute care cases, and still losing a ton of money if electives are being canceled either to quarantine orders, or due to patients canceling them out of fear.

There's also essentially a U-shaped curve between intensive care occupancy and profitability. As ICU/acute care cases decrease below some point, profits decrease, and as they increase they increase, but at some point the cases become overwhelming and the hospital loses money again.

Beds are not all equal when it comes to profit/financial solvency at hospitals.

It should go a bit further than what you're suggesting even. Any half-serious upload of a document or image now, less high stakes than a CB test, generally displays the file you upload for you to verify. Ie, "here's what we have from what you uploaded. Is this ok?"

The college board should have had a list of accepted formats, should have included instructions that they put in the tweet in the upload UI, and should have had an uploaded document verification process. This is solely the responsibility of the CB, and is deceitful at some level, by leading students to believe they were complying when they were not.

This stuff is very interesting and informative, but I had several reactions:

1. What happens if you extend the dates back further?

2. It's interesting to see China's dominance decrease and rise again over a long time period. I knew this but it's interesting to see quantitatively. It leads me to wonder if a simple rise-fall pattern is necessarily "natural" or if more variation exists.

3. It seems like in certain cases, the fall of prominence isn't the same. I don't think of the Netherlands as ever falling into irrelevance or destitution, just maybe declining in power as the major economic and military power. This seems different from more dramatic boom-bust cycles. Maybe I'm wrong about this, as I know relatively little about Dutch history.

4. It's tempting to predict major from transitions but they don't seem to correlate as well with the rise and fall curves, possibly because in actuality there's so many and lots of the major conflicts involve coordinated interests of smaller powers, rather than single interests of single powers.

5. I wonder if some of this is oversimplified because of changes in what constitutes a nation-state. That is, what we recognize as "China" wasn't always as such, and many European states may become better understood as part of the EU in the future.

The problem is that the objective criteria aren't much better. What do we mean? GPA? Test score?

Those correlate maybe .4 with first year GPA in school, and even less with later criteria. So, putting aside variability from one day to the next in when you take the test, inequities in students' preparedness on the day of the exam, etc and so forth, you're left empirically with a fairly weak correlation with one very immediate criterion, that gets even weaker with time.

I don't disagree with your concerns at some level, but think the alternative has its problems too. We discuss "merit" as if people can't change, can't improve, and that, more importantly, all the proxies we assess are equal to what we're really looking for. And then we use it as some statement about individual ability as an inherent, perfectly assessed metric.

Imagine if we measured people's weights at age 18 with a scale that correlated only .40 with true weight, and then used that as an index of physical fitness that then determined important trajectories in their lives. It sounds absurd because it is.

Meritocracy is broken because we've become used to treating proxy metrics as equal to the real attribute of interest, and assuming that attribute can't change and is fixed with time. I think a lot of the problems we have in the US all come back to this, whether it be inequality, resentment, distrust, or dysfunction: there's very much a class divide built on sand, and a sort of daily cognitive dissonance we're asked to just accept.

I'm not saying these purely holistic strategies are better or worse; I have no problems with "objective" criteria, if used appropriately, but now they're not used appropriately.

I think raising standards or changing things is different from being an asshole. Having dealt with this in a big way recently, it's very much on my mind.

It's entirely possible to say "we need to work harder, or take a risk, or be bold" while understanding where your colleagues are at in terms of what they need to do that, or what has or hasn't been communicated to them about how that will happen, or how things have been done in the past to change them.

Too often bad management or administration is justified by saying "well things needed change" or "things needed to be shaken up." There's a way to implement that while bringing others on board, giving credit where things are due, etc.

Put another way, it's easy to ruthlessly level an organization and rebuild it in the way you want to if you have the power to do that, but it doesn't mean the fairy tale you tell yourself about it having been necessary is actually true. More often than not you're just rationalizing away being inhumane and destructive, and probably even losing out on even better possibilities in the process.

I agree with your point re: the depth of information in a field making it difficult to maintain competence in multiple fields.

On the other hand, I think society increasingly projects an expectation of this, that someone cannot possess skills in multiple areas, where areas are increasingly narrowly defined. I think in part this plays a role in stress over higher education, in that a degree is seen as a skill certificate (that is, a statement about what someone can do) rather than an opportunity to learn (that is, a statement about what someone has done).

One thing that struck me at an exhibit of Leonardo's notebooks is how totally wrong he was about so many things. It wasn't mentioned in the exhibit but for every one thing he was correct about, it seemed like there were many many, dozens perhaps, that he was completely incorrect about. Some of the ideas seem preposterous now, although it's difficult to view them appropriately as they would have been at the time.

This could be seen positively or negatively. One way of spinning it is that he was persistent in exploring ideas. Another way, though, is that something -- mythmaking, hype, his art, whatever it is -- has allowed history to ignore the fact that he seemed to have been wrong more often than he was right, along the lines of a broken clock being right twice a day.

In either case, I think there's something to be said for some kind of cultural and social context playing a strong role in how all of this is interpreted. A different person in the wrong place or wrong time might have been interpreted as a crackpot.

These things always have a grain of truth to them, in that they describe some places some of the time, but I also think there's so many things going on in any given place it's hard to generalize.

A well-run organization will hire the right people, and then foster their development in such a way that they are productive, happy, and grow, and will compensate them sufficiently. If any of those things break down, you can run into any number of problems, and they won't necessarily take the form that seems obvious at first.

I've seen organizations take very competent people and ruin their morale and shunt them into projects that were really detrimental to them getting out, in ways that aren't obvious at the time. Or weird organizational dynamics cause changes over time that make someone look bad in ways that aren't their own fault.

There's just weird things that happen that don't simplify to simple models of "talent that is completely independent of environment and perfectly visible."

I'm wondering about the different types of tobacco being used, such as pipe vs cigarette, as those have different patterns of use. One of the studies I read collected that data but didn't report those comparisons.

Those comments weren't at a math journal, but a more applied biomedical journal.

I also have started refusing requests from that journal.

The instructions I referred to werent available until well after accepting the review request, for what it's worth.

I've only used each of those a little.

Turing is a bit like Stan, JAGS, or BUGS, it's [closer to] a general probabilistic programming system with a Bayesian emphasis, although maybe less Bayesian in emphasis than those other DSLs. PyMC would be another comparison.

Edward is (or when I used it) coming more from a very general latent variable modeling framework, encompassing hidden variable models, and is less focused on Bayesian modeling per se, maybe more variational inferential approaches. It seems to be broadening in scope over time.

Pyro I think is further down the deep learning/NN path than Edward.

It's hard for me to ennumerate strengths/weaknesses, as they have different foci and are parts of different language ecosystems. It depends a bit on the use case. My own experience with each is such that I might use Turing, or move to something like TensorFlow; things like Edward or Pyro seem to occupy this intermediate ground that was difficult for me to utilize in the way I thought I might.

I've been excited by Turing, just to see a probabilistic programming framework like that in Julia. I think the expressiveness of Julia and it being native to that framework will be helpful.

The balance is gone, at least in the US, because of several reasons, some easily quantifiable, others more subtle or difficult to quantify.

Most of it is due to the prevalence of indirect fund systems on grants in the US. The way this is budgeted, universities bring in far more money than their actual costs, meaning there's huge incentives to prioritize grant funding over actual research. Because of huge competition and the way grants are evaluated, you have grants only weakly being related to impact or progress, and instead being targeted to what is most likely to be popular.

Another contributing factor is reliance on bibliometric indices like h indices etc. Although this seems reasonable, what it leads to is huge pressure to publish what's popular over other considerations, with lots of co-authors. I've reviewed for high-status journals have said explicitly in instructions to reviewers (and I'm only weakly paraphrasing) "don't evaluate the methodological integrity or integrity of the conclusions, only evaluate how popular this is likely to be."

What happens is you're incentivizing groupthink very heavily. Combine this with eroding of tenure (either through elimination of tenure-track lines, or protections of intellectual exploration), especially in places like biomedicine where the financial conflicts of interest are huge, it leads to big problems.

I have mixed feelings about peer review. I do think it's overrated, broken, and not transparent enough. Having done it for over a decade, I'm still being surprised by ethically questionable things that happen (ironically in my experience it seems like this actually happens the most in fields that are supposed to be more "objective", like statistics or math). On the other hand, my emerging experience with the unbridled internet is that it just reinforces the TED-ization of academics, in that things like twitter attention get mixed in with more formal evaluation of research.

Given the traditional peer review system versus some more open free-for-all, I'd choose the latter for multiple reasons, but they both have issues. In today's dysincentivised climate, the more open approach does have downsides, but maybe if the culture was fixed in other ways those concerns would be alleviated.

To address your question about what to do: there have been lots of proposals but they don't seem to get traction.

Open publication, archives, etc. seem to be moving in the right direction, but many of the grant solutions (eliminating indirect funds or making them line-item justified, lottery systems, funding based on publications through awards) haven't gotten traction.

This is absurd. If anything they should be looking for people in good health who gave already had it.

I agree this is an important discussion, but don't believe in speed limits as it applies to any of this really, depending on what's meant.

Many of the problematic developments regarding SARS-CoV-2 were corrected fairly rapidly. In this regard I see a lot of recent research and its coverage as being typical, but sped up, and with more attention.

What I think has been happening is people -- citizens, journalists, scientists -- just seeing what actually happens in science, and not being able to turn away or sweep things under the rug because of the urgency of things. So when claims start being made on shoddy research motivated by questionable motives, and then enshrined in policy with weak justification, everyone notices instead of it just being ignored like it usually is.

My sense is that the problem isn't necessarily with preprints or anything -- I suspect outlets like that will only increase in use and significance over time -- it's in the way everything is interpreted. There's a tendency to interpret something peer-reviewed as "The Truth" by many journalists, scientists, and clinicians, whereas in reality science is much more messy, gradual, and variable. It's more of a directed stochastic walk than a straight line.

What I'd like to see is everyone treat papers, even peer-reviewed ones, as potentially very fallible minute bits of information, and to recognize the distorted incentive structures that exist in academics today. University press releases should be looked at with as much skepticism as press releases from businesses, for similar reasons.

I think maybe you misunderstood what I was saying / I wasn't being clear.

I agree with you actually... what I'm suggesting is that the current situation with online teaching isn't really generalizable to long-term experiences of it, or the situation where something is planned with online teaching in mind.

I know things will vary by institution for sure, but for a lot of schools going forward they're going to try to avoid the problems you're mentioning.

My guess is even when students return, there will be things like substituting large lectures with online material, etc. Colleagues of mine at other institutions are being told that going forward training and skills in online instruction is being expected of everyone. This sounds obvious but what it means in practice is that the university probably won't find what happened this past semester acceptable.

There's a lot of issues for sure, but I don't think that the long-term experiences of online instructions can be generalized from this past semester.

I've not taught online, but I have a close colleague who was doing it for years pre-pandemic, and another colleague who did a bit of it and my impressions are the same.

My sense is that major universities doing online stuff have video production crews (from Communications etc) and everything is heavily edited and redone. It's more like a television show than anything else. Much more preplanning, and interactions with the students are much more structured.

It's not like some zoom meeting off-the-cuff with your professor who had to switch gears on the spot along with you, while you're uprooted like a minor refugee and living who knows where that you didn't plan to.

Students will be back in the fall. The risks of returning do not approach the risks of not returning for most undergrad-age students. This is especially true that universities will be prepared this fall, and were not in late winter/early spring.

My sense is that a lot of the schadenfreude and resentment here on HN and elsewhere is due to a general sense that something is really very wrong with higher education at the moment, at least in the US, and wanting to see it radically reformed. I agree, but I think the problems are not really related to the value of in-person degree education vs what you can get online. The real issues have to do with the cost of the education in general, how degrees are viewed in vocational settings, and costs and spending within higher educational institutions. Until those things change, it will take a lot more than SARS-CoV-2 to change things.

Speaking as a college professor, this is true. I started noticing really quickly that there was an implicit expectation that faculty be evaluated on their suitability and/or desire to move into administrative positions. This is fine, but it got mingled with professional administrators brought in from other institutions trained in business administration etc. Our president actually had no experience in higher ed prior to their appointment, it all being in large corporate business, and it was seen as a good thing somehow.

Everything has become very hierarchical, run as a corporation, focused on profit maximization, with those profits going progressively more and more to those higher and higher up the administrative chain.

If you can figure out the reasons for university price increases, good luck. It's a combination of a variety of things.

One is that increased demand for employers for "certification" of ability pushes demand for degrees among students. Every recession there's articles showing that those with a college degree fare better than those without. We can argue about why that is, but that's what the average 17 yo sees, along with their parents, and they don't want to be left behind, regardless of the details.

Another is that loans basically prop that up and facilitate it. So schools don't lose money on people who say "I can't afford it" because they find a way.

Another is the increasing administrative bloat at universities, which are now largely run like for-profit institutions even when they are not on paper. I think this is similar to a lot of fields, but my sense is that (based on personal experience) things are much more hierarchical now, with more of a focus on exploiting the university for personal gain than anything else. So administrative salary costs, and administrative costs in general, drive up the need for costs.

The public colleges have to compete for private colleges for faculty, administrators, etc. which then increases their costs, which then drives them to bigger need for more tuition.

At the same time, states have been cutting their funding. If they cut funding and unis have all these fixed costs, where else are they supposed to get money from?

A major dynamic between public universities in the US and political circles in the last couple of decades has been an increasing pressure for them to operate as revenue generating, profit-making institutions in every aspect. At the state level, the idea is that their role is to bring in money, at less cost (an alternative is to see them as providing a service to the state and country -- can you imagine wanting K-12 schools to be treated as profit-making ??? ). As a result, what we've ended up with are institutions with a lot of profit-seeking administrators, pyramid schemes, taking advantage of federal funding loopholes at every chance to bring in money, etc.

Public universities should be able to offer a good value to students, but they're being run as profit centers. If you think of it that way, it's no surprise tuition isn't competitive.

It's more complicated, and a deeper problem than that.

For example, I'm one of those former psych majors, and though those majors are full of people who aren't good at math, there's also a lot of them who know more computer science and math than some of the comp sci majors I've worked with. In fact, a common problem for psych majors and other students is to take the first couple of psych courses, think it's a breeze, and then hit advanced courses with cognitive modeling, neuroscience, epidemiology, and genetics, and realize it is basically STEM.

The problem with this is that everyone who took those first couple of classes, or didn't, has this stereotype of psych as all involving people laying around on couches discussing past lives or something. So you might have a psych major who did a thesis involving neural network models and cognition, maybe with some imaging involving a cutting edge system in the imaging center, working with physicists, and they will get looked down on by employers. Meanwhile the comp sci undergrad who barely gets python and did their group final project on some kind of toy webapp is qualified.

I have family who are making significant money in web development who are actually English majors, who just happened to have the right connections at the right time. When I was in undergrad, comp sci was filled with a glut of unmotivated students who just wanted a degree for a job, and we were all being told that there wasn't going to be demand for comp sci degrees because there would be too many grads in it and you could always learn that stuff through different means. My father, who is now in medical administration with an MD, talked me out of biochem and biology degrees because he couldn't find employment with those degrees and had to go on to grad school.

So what do we make of a degree? Does my dad's biochem degree suddenly become more valuable when it's a stepping stone to an MD program? Did that family member with a "worthless" liberal arts degree waste their money when they're making a comfortable income in the tech sector?

The deeper problem is that people are seen one-dimensionally, as equal to some degree, rather than their experiences, like college is some glorified vocational certification program.

Whenever we talk about "worthwhile" versus "worthless" degrees at some level what we're accepting is this idea that someone's skillset is defined by the degree, rather than the degree being an indicator of part of a skillset.

Society used to see things in the latter way, and at some point it transitioned to the former.