Easy to understand but hard to master. Any slip mixing training data in validation or testing data and your metrics are useless. It can be difficult to impossible to spot and give newbies false confidence in their model.
The same is true pretty much everywhere else (cryptography, sport, electrical engineering, aeronautics, ..). There is no greater teacher than a head injury or an electrical fire, the question is whether OP's fear is warranted or whether it should be encouraged. Perhaps they should not set the bit rate on a video codec until completely understanding the fundamentals of that technology too? I'd suggest like in many similar areas, gatekeeping is a not insubstantial force