Machine Learning Deserves Better Than This

Machine learning has been overhyped and many people have overpromised what it can do.

It's useful for recognizing objects on images, it improves automatic translation, it can rate chess positions, in general it's doing well in certain types of problems.

The thing is, people were expecting it to revolutionize every discipline and it turns out it's incredibly hard to build something useful with it in a domain badly suited to it.

I've taken a look at a handful of CNN based models for COVID diagnosis via chest scans and I believe they are pure noise.

The convolutional layers often get activated nowhere near the actual chest, and they do not seem to grasp the anatomy in scans. The reason is, they were trained as pure "covid-not covid" detectors. Unless you have billions of real scans I don't think that's a good idea, which is why such models need some kind of an organ feature extractor first. Even trained humans suck at this task, and it's not because they miss details.

Machine Learning, as understood in this context of Image Analysis with DeepLearning, is a pretty young field. I'd place 2012 and Hinton's work then as the starting point and maybe even 2015 when ResNet was introduced. That is in the best case 9 years. (AI and ML exist since Turing in 50s but in many forms)

Fortran was made public in 1956 and only 12 years later use of goto considered harmful was published.

It is still a young field with everyone wanting to get in, many without any clear goals or standards, just to publish anything. Trash certainly and from what I've seen myself by reading papers a lot of very badly hidden plagiarism. A ML analog to use of goto considered harmful is very much needed to raise the bar. Scientific integrity, consideration for the greater whole, 100% replicability and basis on sound principles of mathematical theory, most notably probability and statistics.

Yet that is the price of freedom from corporations and big universities. Science for everyone and by everyone. We can't let bad apples, even if they are the vast majority, force us into willing deposition of our rights. If sites like ArXiV went away then access to knowledge would cease under the boots of Springer and Elsevier.

Some sort of blind rating papers incognito could perhaps help to have the good rise above the rest. No information on authors, no information on raters. Just votes mirroring the quality, or lack of, w.r.t the paper.

Machine learning used to be called 'data mining' back in the 1990s and early 2000s. We should revive that name, since it describes the field quite well in many ways.