apathy

This (and dendritic cell vaccines, and inactivated poliovirus, and...) all can be traced back to an observation that the only people with good outcomes from glioblastoma were those with listeriosis in the past.

Steve Forman at City of Hope has reported some excellent results using CAR-T cells in GBM (http://www.nejm.org/doi/full/10.1056/NEJMoa1610497) and Linda Liau at UCLA has seen good results combining checkpoint inhibitors with dendritic cell vaccines (https://insight.jci.org/articles/view/87059).

The fundamental problem with aggressive brain cancers (the most commonly diagnosed kind in adults, unfortunately) is that "excellent" means "the patients survived an average of two years instead of one!"

It's nasty stuff. I work on acute leukemias, which aren't much better, but at least we see some durable cures now and then.

Google wasn't evil, at least not before the IPO (which is when I worked there). People took the slogan seriously and based decisions on it. I don't know what happened.

RF is appallingly difficult to re-use for inference, though. At least with a DNN or CNN you can pop open the hood and see what the model is doing at various points.

Tradeoffs, tradeoffs everywhere. It's almost like traditional mathematical statistics has something to offer them fancy machine learners. (Breiman was a professor of statistics, after all... ahead of his time, but no less a statistician.)

Logistic won't do anything useful for text, to be sure, although an HMM often will (or if you have continuous-valued sequences, a Kalman filter often will do the same). Logistic or multinomial can be tremendously handy for picking up interactions between measurements that can be followed up on and/or expanded in the limited-data case.

I think that the nonlinearity is what really sets apart problems better handled by NNs (not just nonlinearity, but nonlinearity that resists any sort of linearizing transformation), even for lowish-dimensional data. If you look at a linear fit plus a relU, you're just tacking a hinge loss onto a linear/logistic fit. Stack a bunch of these on top of each other and you have a universal function approximator, for which the goodness of fit is limited by the data. If you don't have a crapton of data, the fit isn't likely to be a lot better than linear or transformed linear. If you do have a crapton of data with nonlinear relationships, the implicit structure can be better captured by the flexibility of an NN. But of course you can also spend a lot of time training and debugging the fit, when it might be possible to quickly fit and diagnose a low-dimensional linear or additive model and put it into production. For a long, long time, the most popular "machine learning" method in the valley was logistic regression :-)

For image classification the way people expect it to be done, you are absolutely right (CNNs are incredibly good at this when given enough labeled data). E.g. for relating histological images to genetics or other markers, there's almost no point in not using a CNN with or without a denoising autoencoder in front of it. For low-detail or sparse-and-low-rank mixtures, often you can use compressive approaches to get a lot faster training. But I'll not argue against CNNs for the general image recognition case.

Linear or logistic is typically a great start, and as you note, it's very general. If after trying the simplest thing that can possibly work (linear or logistic), you need better performance, or the linear models can't give you useful answers, ratcheting up the complexity is a reasonable response. You do need a good deal of data to make the latter step worthwhile in most cases. I see a lot of people skipping the first step or ignoring the need for lots of data, and these are the people who get in trouble.

Yeah, sorry about that. By the time I realized the way the tone had come off, I had managed to "noprocrast" myself off the site. I need to write better hot takes

Sorry, I should have used the phrase "inverse cdf". If you can map a pile of data onto 0 to 1 (i.e. by ranking it) then you can invert it onto the values that occupy those quantiles for a Gaussian. This is nice because you can use quasi-parametric assumptions (very nice when multi-dimensional relationships are to be explored) without the data itself needing to be distributed appropriately. Other alternatives are to use nonparametric smoothers and the like but I kind of hate going to all that trouble when it's usually pointless. (important exception: when you have interesting correlated structure in high-dimensional data)

It's a hack, to be sure, but especially if you want to pool data (e.g. in mixed hierarchical models) for better predictions, it often pays off. The name is a play on "logit", "expit", "probit", "tobit", etc. since the actual transformation is relatively trivial for data that is already sorted. (For large unsorted data or streams, not so much)

quantile regression is computationally intensive and inverse transformations usually less so. Although you could certainly make the case that, given enough data, quantile regression better captures what we actually want to find, most of the time (i.e. how's this effect diverge towards the extremes).

Normit typically (not sure if universally) has the lovely property of giving you something like a marginal t-test without the assumptions of mixture-of-gaussians errors. You don't jerk around with U-statistics and thus the sample size doesn't make the test statistics so damned granular.

re: normit/inverse transformation: Pet theory? This isn't a theory, it's a simple inverse transformation. It's used all the time. George Box showed how to do a 2D version of it in the 1950s. Normit may be a pet name for it, though. It's just a play on words (probit, logit, expit... normit).

As far as describing data, what's wrong with median + IQR for marginal distributions, or some flavor of energy statistic for joint distributions? You will always need to trade off robustness for sensitivity and bias for variance. That's simply a mathematical feature of the universe. There are plenty of ways to take advantage of this to highlight outliers, for example, which often gets you thinking in terms of "hey, this really looks more like a mixture of two completely different distributions" and seeing if that intuition holds up.

The whole point of describing data with summary statistics is that if the assumptions are met this decouples them from the underlying data. If you use the median as an estimator for the center of your distribution and the MAD as an estimator of its scale, you may choose dimensions along which it's not very good at partitioning your observations. If you want a resistant way to describe the expected center of your data, the median and MAD are very useful. Sometimes it's even more useful to plot everything and point out "our results fall into K obvious clusters" each of which will have their own center & spread.

What I'm saying is that there's no silver bullet. Most of the time we take descriptions of the data, see if there are interesting inferences to be drawn, lather, rinse, repeat. "Get a lot more samples" often thrown into the mix. Are there strong clusters in the data? (Usually a plot will show this, whether via projection or in the raw data) Are there continuums that are interesting in relationship to things we care about? (Usually we'll turn around and model their relationship to said thing-we-care-about, conditioned upon a bunch of other items... multivariate regression, which if you're doing it right, will get you plotting the residuals, themselves descriptive of the model fit)

You simply can't do responsible statistical inference without exploring your data to see what's going on. In order to explore complex datasets, there are plenty of techniques, and most all of them demand tradeoffs (see MAD vs. SD or other metrics of "interestingness" for clustering). A number of descriptive statistics ("extremality" for example) rely upon limit behavior of specific distributions and are case-by-case.

I don't think you'll find many silver bullets for either descriptive or inferential statistics. You have to choose your tradeoffs based on what you want to accomplish.

1) it is pretty amazing that normit transformations (map the quantiles of a non-normal distribution onto a Gaussian and use that) don't seem to be on this guy's radar. We use distributions with linearly additive and affine invariant properties (normal plus normal is normal, bernoulli plus bernoulli is bitwise bernoulli) because we find linear algebra very useful. Nonparametric tests and procedures erode your power; normit transformations usually increase it. I did part of my dissertation on this; it's partly to do with asymptotics, but also partly due to the robustness of Gaussian error assumptions thanks to the CLT.

I realized recently that a lot of the trouble people have with training neural networks stems from their lack of training in basic model evaluation. If you stack a bunch of shitty penalized regressions (which is what linear/logistic + relU hinge loss represents) you now have one gigantic shitty regression which is harder to debug. If your early steps are thrown out of whack by outliers, your later steps will be too. Dropout is an attempt to remedy this, but you tend to lose power when you shrink your dataset or model, so (per usual) there really is no such thing as a free lunch. But most of the tradeoffs make more sense when you are able to evaluate each layer as a predictor/filter. Scaling this up to deep models is hard, therefore debugging deep models is hard. Not exactly a big leap.

There is a reason people say "an expert is a master of the fundamentals". Building a castle on a swamp gives poor results. If you can't design an experiment to test your model and its assumptions, your model will suck. This is not rocket surgery. A GPU allows you to make more mistakes, faster, if that's what you want. If you have the fundamentals nailed down, and enough data to avoid overfitting, then nonlinear approaches can be incredibly powerful.

Most of the time a simple logistic regression will offer 80-90% of the power of a DNN, a kernel regression will offer 80-90% of the power of a CNN, and an HMM or Kalman filter will offer 80-90% of the power of an RNN. It's when you need that 10-20% "extra" to compete, and have the data to do it, that deeper or trickier architectures help.

If you can transform a bunch of correlated data so that it is 1) decorrelated and 2) close enough to Gaussian for government work, you suddenly get a tremendous amount of power from linear algebra and differential geometry "for free". This is one reason why Bayesian and graphical hierarchical mixed models work well -- you borrow information when you don't have enough to feed the model, and if you have some domain expertise, this allows you to keep the model from making stupid or impossible predictions.

Anyways. I have had fun lately playing with various deep, recurrent, and adversarial architectures. I don't mean to imply they aren't tremendously powerful in the right hands. But so is a Hole Hawg. Don't use a Hole Hawg when a paper punch is all you really need.

2) What (good) statisticians excel at is catching faulty assumptions. (I'll leave it to the reader to decide whether this data-scientist-for-hire has done a good job of that in his piece) So we plot our data, marginally or via projections, all the damned time. If you don't, sooner or later it will bite you in the ass, and then you too can join the ranks of the always-plotting. However, choosing which margins or conditional distributions to plot in a high-dimensional or sparse dataset is important to avoid wasting a lot of time. So whether via transformation or penalization (e.g. graphical lasso) or both, we usually try to prune things down and home on on "the good stuff". Prioritizing what to do first is most easily done if you have a number and can rank the putative significance by that number. Use Spearman, use energy statistics (distance correlation), use marginal score tests -- IDGAF, just use these as guidelines and plot the damned data.

Corollary: if someone shows you fancy plots and never simple ones containing clouds of individual data points, they're probably full of shit. Boxplots should be beeswarms, loess plots should have scatterplots (smoothed or otherwise) behind them. And for god's sake plot your residuals, either implicitly or explicitly.

3) see above. The author is good at fussing, and brings up some classical points. But they're not really his points. Median and MAD are more robust to outliers than mean and standard deviation, but that makes them less sensitive, too. Check your assumptions, plot everything, use the numbers as advisory quantities rather than final results.

summary(df) will do that for every column of a dataframe with less code... which is why few people use fivenum()

The majority of defaults in R exist because statisticians do these things all the time and don't like typing any more than programmers. Time spent typing could be spent thinking about what assumptions are being violated and whether the results offer any useful insight.

Your winnings, sir

Yes, the author is serious that investors worry about losing billions of dollars. If your house or savings became worthless overnight, I suspect you'd be worried too.

It's a statement of fact. If UCARTs come to prominence on the market, Kite & Juno no longer appear to be the only game in town (or even the only CAR-T game in town!). Toss in some grade IV/V events in their clinical trials, and an investor in these companies might have reason to worry.

Individuals and societies can have conflicting priorities. Sometimes the same individual may be conflicted (plenty of oncologist friends invest in various companies, and are distressed if a competing company's drug blows their holdings' product out of the water, even as they consider switching patients to the more effective drug). Put differently, you can own a Tesla and hold Exxon stock ... and that does not make you a bad person.

Infant leukemia is (thankfully) rather rare. Nobody who's ever seen a cold, gray infant in a morgue drawer would ever say that those lives are worth less than some company's profits. But they might regret investing in the company!

That's a function of perverse incentives, rather than some sort of immutable truth. If there were stronger penalties for (e.g.) cartel-like pricing (e.g. TKIs), and more price transparency for the true costs of care (ibid), we might see less of this. Or if, say, the primary market for such drugs adopted a single-payer system where the payer negotiated on behalf of the patients, rather than on behalf of its shareholders.

Gleevec and Sprycel are the canonical examples of "competition makes the price go up" but if a single payer dominated the market and used this as leverage, it would be simple enough to just say "not at that price" and force manufacturers to play ball. The real problem is regulatory and legislative capture which prevents this from happening.

How's it compare to HungryBot from infinome?

Calorie counts are typically adjusted; manufacturers don't typically just toss a bag of, say, Ho-Hos in a bomb calorimeter and use the raw numbers from Igor or whatever recording program happens to be around.

The usual macronutrient values (9kcal/g fat, 5 kcal/g EtOH, 4kcal/g carbs and protein) are surprisingly decently calibrated for metabolic inefficiencies.

They do not, however, take into account hormonal responses, which are probably quite a bit more important than once assumed. So there's that.

Oh god, I missed the passage about the 400 t-tests. Your professor is the guy on the cover of this book:

https://www.statisticsdonewrong.com/

If your comment was a book, I'd lovingly put it on the shelf next to Reinhart's masterpiece ("revised and expanded ... with three times as many statistical errors and examples!"). Unlike Shakespeare, it appears nonfiction is well within the capabilities of the Interwebs (asterisk).

(asterisk) For many years, it was believed that millions of monkeys hitting millions of keyboards would eventually recreate the works of Shakespeare. Now, thanks to the Internet, we know this to be false.

Even in ICUs and NICUs, where it already is being automatically gathered and fed, it turns out that making sensible design and training decisions to get usable performance is harder than anticipated.

News flash, dermatologists don't just look at moles and oncologists don't just do differentials... let's see how these isolated systems deal with cleaning up TKI- and IST-related gastric bleeds, C. diff code browns, and timing chemo around liver resections for sepsis. All in a day at County...

My bet is that clinicians who use AI to amplify their own abilities will come to run the system. Errybody else gets to be a glorified NP, at best, or (worse) administrative ;-)

You present this as binary: doctors OR ai.

I propose that the doctors who use ai to make themselves damned near infallible (by catching the corner cases where the ai is poorly trained) will make out like bandits, just like top lawyers. The ABA is a monopoly, too, and legal cases are perfect fodder for RNNs. Yet good corporate lawyers still make money faster than they can count it.

Meanwhile, shitty lawyers are out of a job.

Lather, rinse, repeat, but with a million times more regulation and inertia.

Nice theory but acute medical emergencies are often stochastic. Let me know if you get to the point where you can predict a pulmonary embolism in members of the general public with enough accuracy to dispatch an ambulance.

Preventable conditions are often a combination of genes plus environment plus luck. Trouble is, the "luck" part dominates more often than we'd like. This is why preventive medicine is often accused of being tremendously arrogant.

We've got a long ways to go, and it would help to focus on the highest marginal benefit items. (We were supposed to have flying cars, and all I got was the sum total of Hunan knowledge on my phone... not a bad trade, on balance)

Sparse x low-rank: if you collected every cell in the matrix, you paid too much for your sensor :-)

Emmanuel Candes' lectures on compressed sensing changed my life.

True, if the marginal cost is tiny, a lot more doors open. But my recent experience with a VR headset we got my daughter for xmas ... did not knock my socks off.

Predictions are difficult when they concern the future. I'm skeptical, but I've certainly been wrong before. We shall see.

Sure, there are verticals that get extra value out of it. But flight sims (for example) are actually easier to "get right" with regards to subsonics, etc by constructing a box to put the person in, and immersing them directly.

Similarly, telemedicine was a thing 20 years ago and it does work in some cases, just like robotic surgery with the DaVinci system. But most people want to have a doctor diagnose them and then perform whatever procedures might be necessary, and especially the wealthy customers hospitals really want tend to prefer doing that with a human. So most of the value add comes from the surgeon rather than the surgery. (Even in oncology, bedside manner is worth a tremendous amount -- to the tune of 8-figure restricted endowments for colleagues of mine)

Speaking as someone who participated in said research, I've seen this cycle before. Maybe it will end differently. But I personally am skeptical. YMMV.

> two completely different technologies

Which nonetheless have the common side effect of nausea in a non-negligible fraction of the population...

VR was workable 20 years ago; I was working on it at that time. Thing is, it only provides a tangible benefit for certain use cases, and most people don't derive enough value from those use cases to pay extra for it.

Argue all you wish, but both 3D and VR are now on their 3rd or 4th hype cycle, and the evidence from previous cycles is that people just don't value them enough to justify the cost.

And take his/her valuable contributions elsewhere? Y combinator will surely collapse like a house of cards /s

Or rent a bigger AWS/EC2 instance to prepare for the eventual demise of old school HPC

Fair enough, SMA is certainly not awesome for anyone whose kid is dying from it. I should be more measured in the words I choose. "It's a shame that a lab run by a guy who may be an asshole slave driver could negatively impact the perception of SMA research, but if he is in fact playing fast and loose with the facts, it will be better in the long term for him to stop working on SMA, or anything else"

Usually that gets filed under theory or philosophy.

Well, I didn't claim to be a trained historian, which is probably the correct training for this type of work. Based on your temporal evidence, I must concede the point.

However, I did not actually pull this straight out of my ass: http://www.thefreedictionary.com/statist

Thanks for the history lesson (seriously). I learned from it.

You think that bogus work doesn't siphon funds away from good quality research? What universe do you live in? With a 10% pay line, even most good research won't get funded. For bogus research to get funded, better work MUST go unfunded. Guaranteed that will lead to more dead kids and a longer wait for effective treatments.

And if Rubin was falsifying data, then so much the better if it wipes him out. Few things are more disgusting than preying on the horrible situation of kids with rare diseases just to crank out research indirects and promotions, mostly on the backs of slave-labor postdocs with no alternative employment in this country.

So, to sum, if Rubin falsified the work, fuck him twice as hard as other frauds for his mendacity.

If he did not, and this is all much ado over nothing, then I sincerely hope he will be vindicated and I wish him the best.

Reminder: the terms "statistics" and "statisticians" come from the common root word "statist".

I am a trained, practicing statistician (clinical and medical, not governmental) and this always gives me pause. "Data science" is a shitty term, because science without data isn't science at all. Moreover, these days it's just long for "IT guy". But I'm not sure what's better as a term for the real thing, where experimental design and inference are as important as predictive accuracy in one specific context.

Anyways. Statistics was always built around statism.