timy2shoes

Or you have the classic case where the model trains on the wrong things (e.g. the classic husky/wolf and snow case). One example I heard was in heart attack risk prediction from x-rays, the model was focusing on the present of stents in the x-rays. Which is a trivial prediction because the doctor already knows the patient has a stent, so the model is not adding anything of value.

It's not about revealing information, it's about how you model the problem. Say you're using logistic regression. If you upsample the minority class (never downsample the majority class because that's statistically inadmissible), then your classifier might get better at modeling the decision boundary. However, that comes at the cost of losing the best part of logistic regression: probability prediction and calibration. Sometimes the trade-off is worth it, sometimes not. Depends on the type of problem.

For more details I suggest reading Frank Harrell: https://www.fharrell.com/post/classification/

I can tell you from extensive experience that domain knowledge is more critical to understanding such issues than this algorithm. Fundamentally, such causes can not be identified if they don't exist in the data and the algorithm doesn't "see" them. Like the previous example where the identity of the object being hurled is hidden and all the data shows is the distance. And the causes change a lot from field to field.

> Suppose we throw 1,000 baseballs, shoot one rocket (45 degree angle), record the distance each one flew, and present the 1,001 data points to someone for analysis.

But I would argue that's not really a case of an outlier, that's a mixture distribution because the underlying identifying feature (baseball or rocket) is missing.

Not in my experience. The reason is because of funding. NIH grants are focused on biological discoveries and not so much around infrastructure. So you can get funding to build a tool, but not so much to maintain it. The downstream effect is that tools and websites are stuck in the era in which they were created, databases are not updated, and tools are broken because the grad student left and now there’s no one to respond to issues.

> Two, all the functional annotation is based off transcription profiles. They essentially looked for clusters of genes with correlated expression, and assigned function based on genes with previous annotations

This is an important point, because if you've ever worked with single cell data you'll know that the transcriptional profile is extremely noisy and your transcriptional profile to cell type map has many researcher degrees of freedom. I heard a story about a paper early in the single cell work that started with 53 cell types and after review ended up with 37 cell types. Are those true cell types? Did the experimenters validate that those cell types all performed different functions? Well, of course not. That's way too much work.

Then add on technological biases, which make mapping between technologies difficult. I say this because they used a new sequencing technology that appears to have homopolymer bias (https://twitter.com/lpachter/status/1533875723995185153), which will bias the gene quantification.

> this does not sound to me like we know the "function" of these genes, only that they're nessary for each phenotype

But they're not even measuring the phenotype. They're using the transciptional signature as a substitute for phenotype/cell function (i.e. the bag of RNA model). This is a poor substitute if you try to apply this to practical applications such as cell engineering. Let's say I perturb a cell to match it's transcriptional signature to that of a neuron. Does that make it a neuron? Not if it doesn't function like a neuron.

The bidding up is exactly why Zillow lost money. Algorithmic house buying suffers from the winners curse. When Zillow thinks they are getting a good deal, there are usually extenuating circumstances that are apparent to local buyers which cause them to not bid up the price. That and the iBuy program was expanded way beyond the original scope.

Starting material is a huge problem. You need at least 50ng of DNA for an Illumina sequencing run, usually more if you want to sequence ultra deep, which you will need for whole genome sequencing. For PCR you need at least 0.1ng (usually higher) of DNA. But a single cell has about 80 pg of DNA. So you need to amplify the DNA several orders of magnitude before you can apply standard PCR. Several approaches can solve this problem, DOP-PCR, MDA, MALBAC. But each results in biased amplification, with differing biases with each method.

Source: I worked on the computational modeling of this problem in grad school, https://academic.oup.com/bioinformatics/article/30/22/3159/2...

I suspect it's because kvogt is an active member of HN: https://news.ycombinator.com/threads?id=kvogt. His (long-form) insight into this will be valuable to hear.

You don't get it in the USA.

I prefer to go to the original source, specifically The Transformers (1984-87).

The game: https://en.wikipedia.org/wiki/The_Game_(mind_game)

If he wanted to work on computer vision instead of a vegan restaurant (or something similar, I forget exactly).

The evidence definitely points the other way, see [0]. The vaccines dramatically reduce the rate of severe infection and hospitalization. The vaccines do not prevent infection, they prime your immune system to fight the infection. That is why the endpoint for the clinical trials is not infection, which would have to be determined by testing, but rather symptomatic infection.

[0] https://www.covid-datascience.com/post/israeli-data-how-can-...

> Any ownership required the Judge to recluse himself.

Where does this requirement come from? Citation please.

Even the murder stats can be juked, like McNulty's arc in season 5 or the bodies in the vacants.

Fantastic point. Another thing about academic ml models is that they're commonly evaluated on in-sample data. For example, medical AI devices are commonly evaluated on data coming from the same source/hospital that the training data comes from (see https://www-cdn.law.stanford.edu/wp-content/uploads/2021/04/...). This leads to over-confidence in the performance because of overfitting of batch effects.

I'm not excluding the lab hypothesis, I'm just pointing our that the natural hypothesis is also valid. Honestly, I don't think we're ever gonna figure out definitively which one it was (I don't think there's a 3rd competing theory). Even if it was a lab leak, I suspect that the evidence for that is gone.

Yes, the jumping of viruses from animals to humans due to keeping them in close proximity is so unlikely that it's happened hundreds of times before (e.g. SARS, MERS, Spanish flu, bubonic plague).

> But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Then, months later and after “post-processing,” another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera.

That's falsification of evidence. Will there be any consequences? Who watches the watchmen?

> My naive understanding is that "smaller p-value" == "more likely result is true".

I think you're making the classic Prosecutor's fallacy: https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy. In my experience, smaller p-value tends to be more of a measure of sample size than anything else, or an overly restrictive null distribution that is almost certain to be rejected.

My personal opinion: I respect the opinion of a medicinal chemist working in the field of drug discovery more than I do a journalist. The insider who works in the field understands the issues and problems of getting drugs to market more than an outsider.

> if the rainbow colormap is so awful, why is it still so common?

Because it's the default for a lot of matplotlib visualizations. Per Goethe: "Misunderstandings and lethargy perhaps produce more wrong in the world than deceit and malice do."

Exactly! This reminds of the recent case of the AMA increasing the suggested age of regular mammograms. More testing means earlier detection of breast cancer. But a false positive means an invasive biopsy. These are the two sides that need to be balanced. The AMA decided that since the base rate of cancer is so low in women under 50, the posterior probability that you have cancer given a positive mammogram for the under 50 age group is considerably low. Therefore they decided that the increased costs (monetary and emotional) do not outweigh the benefits.

From https://www.ama-assn.org/sites/ama-assn.org/files/corp/media...

"Screening mammography reduces mortality from breast cancer, including in women younger than age 50 years. However, screening mammography carries harms such as false positive results that can lead to additional imaging and invasive biopsy procedures, and overdiagnosis that could lead to treatment in patients who may not benefit from it. The USPSTF considered the balance of benefits and harms using a commissioned targeted systematic evidence review of randomized clinical trials and a decision analysis that compared the expected health outcomes of starting and ending mammography at different ages and using annual and biennial screening strategies; it concluded (in part) that routine screening should begin at age 50 years and continue biennially until age 74 years."

> let's remember that the AlphaFold team at Google solved the protein folding problem with a relatively small team in a relatively small number of years after extremely large, well funded companies whose primary business was drug development failed to do so for decades

Did they though? They did extremely well at CASP14, and much better than competing groups. But does this solve protein folding? Deepmind's marketing department would have you think so, but for those of us that work in the field we know that this is not the case.

Furthermore, does protein folding solve the relevant problems of drug design? It solve the forward problem, given an amino acid sequence predict its 3d structure. But for drug design we need the inverse problem, given a specified structure predict an amino acid sequence that produces that structure.

I think regulatory capture applies here (https://en.wikipedia.org/wiki/Regulatory_capture)

> Larry Page greenlit Google Health (a terrible idea)

I'm curious why you think Google Health was a terrible idea. I'm not disagreeing, just wondering your opinions on the matter from a more inside point of view.

Thank you.

The other way around is definitely easier. RNA-seq analysis is a mostly solved problem with DESeq2 (or edgeR/limma). The tutorials are very detailed. The most difficult part is experimental design, which you probably know already.

Deep learning, on the other hand, is so fraught with pitfalls and traps. Even if you can code up a model successfully, it's very easy to trick yourself that you're doing very well (see a previous discussion at https://news.ycombinator.com/item?id=27376839). In my opinion, most of the work should be spent on making sure that you're not tricking yourself.

Although it's not a review article, but maybe Bork's The Antitrust Paradox, which the Amazon Antitrust Paradox works hard to refute.