Comment by Reelin | Hacker News Reader

Comment by Reelin | Hacker News Reader

Parent Full thread

I appreciate what's going on at UW regarding this and realize that it must have been a huge amount of work to say the least. However, I'd like to point out that most of what you've described is related to the bureaucracy and paperwork of certified medical procedures. Under ordinary circumstances that's all well and good, but given the severity of the situation it seems like it would have been better to eliminate much of that at least a month ago. Put another way, combat medics don't let someone die just because they're short some official FDA approved widget.

Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork, whoever ordered the test can deal with that.

The supplies for RNA extraction are common and readily available (I think all the raw components are mass produced?), but only specific kits are FDA approved.

Only specific swabs are FDA approved. (https://khn.org/news/as-coronavirus-testing-gears-up-special...)

> the government is considering expanding its recommended testing material options to allow for more general nasal swabs to keep up with the increased testing demand

Standard qPCR machines are incredibly common in both academia and industry and have 96 wells at minimum. The protocol published by the CDC has an 80 minute runtime. That's 1000 tests per day for a _single_ low-end machine.

The US response has been an absolute shitshow of bureaucratic dysfunction as far as I'm concerned, and people will likely end up dead as a direct result.

> Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork, whoever ordered the test can deal with that.

Detailed records and redundancies ensure you don't mix up any of the tens of thousands of samples coming in. These systems are crucial for being able to scale up as they have without complete chaos breaking out in the lab.

You are right that barcodes are far more efficient, after implementation. But implementation takes months, and requires diverting resources that would otherwise be used to run more samples. The lab would face significant downtime putting a barcode setup in place and retraining. (See nkrumm's comment about labs not having a "test" environment.)

> Standard qPCR machines are incredibly common in both academia and industry and have 96 wells at minimum. The protocol published by the CDC has an 80 minute runtime. That's 1000 tests per day for a _single_ low-end machine.

That's only if we ignore the sample preparation time (orders of magnitude longer than analysis time), and the many control wells necessary to ensure the technique was successful. While I suspect these will also be dismissed as bureaucratic inefficiency, they're needed to make sure your newly-installed instrument running 24/7 hasn't broken down, and that the high throughput sample prep wasn't botched for a given prep batch.

The stakes for these tests are high enough that cutting corners on QA/QC isn't acceptable, because it means more lives lost. Doing a mediocre job for testing at this scale will have a very real impact on our population.

My calculation (1000 tests per day) left space for 12 controls per batch - is that not sufficient under the circumstances? Alternatively, at 500 tests per day (again, this is _per_ low-end machine!) you could run each sample twice in addition to the 12 controls. (And I would _never_ dismiss controls as an inefficiency, even the most trivial exploratory experiment is highly suspect without one.)

The point I was trying to make with those numbers was just how readily accessible the necessary instrumentation is.

Regarding sample preparation, I'm well aware that it's a time sink (I've done DNA and RNA extractions before). But it's fairly trivial and can be done fully in parallel by multiple people; the primary bottleneck is likely to be available bench space. (Unless the sample prep has been fully mechanized, in which case I'm really not seeing the issue.)

My point being that multiple, parallel sample prep pipelines can feed a single qPCR machine to keep it running just about 24/7.

Regarding barcodes, I didn't mean it had to be fully automated. Just drop the excess data entry and switch to a serial number scheme with built-in redundancy (or at least error detection). Many academic labs employ such schemes by hand.

Given just how dire the circumstances are, I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC (as compared to standard medical diagnostic testing) would be unacceptable. None of what I described is at all out of place for handling research samples, and in my experience those are quite reliable.

I used the combat medic analogy in my previous post for a reason - a significant number of people are likely to end up dead due to our having blindly stuck to the rules. A bit of pragmatism could have saved them, and I view that as a tragic systemic failure on the part of the US government.

You're saying a lot of things here that don't make any sense.

> I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC [...] would be unacceptable.

> I would _never_ dismiss controls as an inefficiency

So, the QC in QA/QC stands for Quality Controls. So, you're strenuously insisting they're unnecessary, yet vehement that you would never dismiss the thing you just said wasn't necessary? Which one is it?

> But [RNA extraction is] fairly trivial and can be done fully in parallel by multiple people

It takes longer than the instrument analysis was my point. That's the typical bottleneck in running samples, not your instrument sample throughput like you claim. So even if you can run 1000 samples on the instrument, if it takes longer to prepare them (which it always does), then that's your rate limiting step.

Unless, of course, you're working in this magic lab of yours where new, fully-trained techs suddenly materialize any time there's a spike in sample volume (whereas new benchspace doesn't? Might wanna take that up with your deity lab manager).

> Regarding barcodes, I didn't mean it had to be fully automated. Just drop the excess data entry and switch to a serial number scheme with built-in redundancy (or at least error detection)

With all hands on deck who do you think will have time to do this? Barcode migrations are a lot of work. Adding the word "just" to what I've described doesn't make something trivially easy.

Though apparently this whole process is trivial, from the sample check-in to the RNA extraction to the instrument analysis. Why not just fire the UW resident and all the staff, so that you can swoop in save the world, since it's all so terribly easy for you? I look forward to touring this fully-automated magic lab of yours.

I'm fairly certain that everything I said makes sense, so I'll try to clarify a bit.

First though, it seems a misunderstanding has developed. I was not calling the competence of the UW lab into question! They are bound by various state and federal regulations and I'm certain are doing the absolute best that they can under the circumstances. The examples I provided were to illustrate just how absurd the current state of affairs is, and to make it clear that bureaucratic ineptitude is to blame.

We obviously both agree that QA/QC is necessary; note that my back-of-the-envelope calculations included controls. I don't pretend to know precisely what requirements FDA regulations place on various diagnostic tests. What I do know, and attempted to illustrate in concrete terms, is that it is physically possible to scale testing in a sufficiently reliable manner using common instrumentation and bulk reagents. Any genuine attempt to explain US testing shortages _must_ account for this fact.

Yes, parallelizing sample prep requires a sufficient number of properly trained technicians. Presumably the US government is capable of locating and making use of qualified personnel in an emergency situation such as this? (To start with, literally any graduate student whose research includes running molecular biology protocols is likely over qualified for sample prep.)

Regarding serial numbers and labeling, it is indeed trivial. You can literally keep a list on paper and label tubes by hand (I did this for years in an academic lab). This step is fast compared to overall sample prep time.

I have no idea where you got the idea of a barcode migration from. Remember, I'm not describing a clinical lab operating according to FDA regulations; I'm describing a sufficiently reliable setup that closely resembles an academic research lab and would maximize throughput in an emergency situation. The only goal is to receive a package, label a corresponding tube, run a test, and report the result.

(BTW, I claimed that bench space for sample prep, not instrument throughput, was likely to be the bottleneck. In fact, instrument throughput not being the bottleneck was one of my primary points.)

If you're basing your sample throughput calculations on instrument throughput (1000/day), you're implicitly assuming that instrument run time is the limiting factor, and the rest of the operations can match it.

I'm also confused by your apparent assumption that bench space is a universal constant that cannot be changed, even under a global pandemic, while lab personnel can be scaled up effortlessly.

> Regarding serial numbers and labeling, it is indeed trivial. You can literally keep a list on paper and label tubes by hand (I did this for years in an academic lab). This step is fast compared to overall sample prep time.

Scientists don't keep records on scraps paper for a reason, and I'm shocked your PI didn't read you the Riot Act for shoddy record-keeping.

With thousands of samples coming in, this is how you end up completely losing track of your samples. Once again: it's not trivial, particularly at this scale.

> I have no idea where you got the idea of a barcode migration from.

> Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork

I was attempting to illustrate the absurdity of the past month and a half by performing a back-of-the-envelope calculation, not provide a detailed business plan for launching a biotech startup. FFS

I never claimed that bench space was a universal constant, nor that hiring personnel is effortless. Rather, I attempted to make it clear that from both scientific and logistical perspectives the barrier to effective testing is quite low.

The point of the instrument throughput calculation was to make it clear that instrument availability can't possibly be the limiting factor because a single low-end one that many academic labs already have on hand can facilitate an incredible number of tests each day. That's important because you can't depend on being able to procure new instrumentation in a timely manner during a crisis so any estimates ought to assume that you're stuck with whatever you already have.

We see reagent shortages reported in the news, but that can't possibly be the limiting factor either because (as far as I know) only common chemicals (plus primers and probes) are required. (Actually you could probably dispense with the probes if you were willing to run an awful lot of gels, but that would further complicate the pipeline and require more labor so better not.)

So then we might suppose that personnel and training is the bottleneck, but that can't be it either because the government has deep pockets and just about any practicing molecular biologist is over qualified to perform such a trivial procedure.

Looked at this way, it should be abundantly clear that US policy is entirely to blame.

I also never said to use scraps of paper in lieu of proper record keeping. The obvious interpretation of what I wrote is to go line by line in a lab notebook, which is something I have in fact done before (obviously not on the scale described though). It works quite well in the long term provided that your serial numbers incorporate either a day or a page number and are strictly sequential. It's hardly the only way to go about it; I only mentioned it because the original post that I responded to explicitly called out record keeping as a necessarily time consuming and complicated part of the process. I was illustrating that once again, that is only true when complying with existing US regulations.

What was described above wasn't "US Administrative Bureaucracy" so much as "this is how busy labs are run." It can be challenging to understand that if you're coming from a small research lab where QA/QC boils down to recalibrating when you feel like it, and sample records are maintained by transcribing loose paper into Excel.

The bureaucratic aspects are to ensure you don't mix up the large volume of samples, and the quantitative QA/QC metrics are needed to objectively esablish what level of error is considered acceptable. Good record-keeping and QA/QC are all necessary when you're dealing with healthcare, particularly largescale healthcare.

That doesn't mean US policy or the administration's handling of this crisis was anything close to acceptable, but the issues there are pretty far removed from routine laboratory sample check-in and record-keeping.

Reply on news.ycombinator.com