Transformers in Medical Computer Vision

The main challenge with medical data is still the access and ownership. ViT transformers looked good, but only when training on huge amounts of data. Another problem is the distribution of the data, having good results with an internal dataset of a hospital does no guarantee getting the same results with another hospital's dataset.

I didn't know about the HATNet mentioned in the article, but it looks an interesting paper to read.

There's a recent article[0] showing that a good training strategy with a ResNet can be better than novel architectures. From my experience, a ResNet-34 / ResNet-50 is usually enough (at least talking about classification), and the main bottleneck is your dataset, class imbalance, and handling out-of-distribution samples.

[0] https://arxiv.org/abs/2103.07579