I've been involved in a federated learning radiology project, the biggest issue is image technique and labelling.
Different centers practice very differently, with different imaging protocols, disease prevalences, and labelling/reporting.
This project was looking at renal masses and the only part that worked well with federated learning was image segmentation and probability that the mass is a cancer, this is a competency I expect out of a first or second year trainee.
It was horrible for predicting subtype of cancer as we couldn't get a good training set (few of these lesions are biopsied, specific MRI sequences that may help are not done the same way in every center) which is what the goal was and more of an experienced generalist/subspecialty radiologist skill.
Practically subtype doesn't make too much of a difference for the patient as if it's "probably a cancer" it'll just get cut out anyway, but highlights a challenge with federated learning.