If the dataset mostly contains edge cases, model performance on the dataset is going to be poor, but it's not an issue I think.
But how could the real-world accuracy be computed? Is a separate dataset needed for that purpose?
But how could the real-world accuracy be computed? Is a separate dataset needed for that purpose?
However, in practice we see that it is very hard to collect a good dataset. There is a great twitter thread from Abubakar(CEO Gradio) about this topic: https://twitter.com/abidlabs/status/1423067498862219267