It's not easy at all.
By your measure, Professor Ferguson at ICL is an expert in predicting diseases. He's got a big team and has been doing it for more than 20 years. Many published papers. Invited repeatedly to advise world government's due to his expert status. Widely asserted to be the best in his field.
Guess what - it appears every prediction he ever made was wildly wrong. His predictions for COVID were also wildly wrong. His code was released and it was full of severe bugs. He told the world millions would die of COVID and then when they didn't, claimed his expert recommendations had saved them, all in a fraudulent paper that assumed as a prior that the virus in Sweden mysteriously behaved totally differently to everywhere else and that only lockdowns can possibly affect the course of an epidemic (despite knowing that it isn't true).
This guy has been a failure and fraud his entire life. So has his entire field. It's not just him. All the expert epidemiologists produced models that failed to track reality in any way at all.
Yet, he's an expert by any proxy metric you care to pick. His colleagues rally around, his university protects him, Nature magazine published apologetics for him. He's an expert, unless you pick the only test that matters - are his papers correct?
I know it's scary. But decades, perhaps centuries of near sheep-like behaviour towards science and academics has led to their institutions being saturated with incompetence and fraud. They have no working institutional mechanisms to detect or clear it out. That's why so many fields have "skeptic" movements that "deny" the science. Usually they aren't actually science-deniers as painted, they're people who expected the so-called science to actually care about the scientific method and do their sums right. When they discover that it isn't happening they blog about the errors they found, and the next thing they know, they're being demonised by the press and the very self-same scientists they're criticising. Not because they are wrong, but because they are dangerously right.