I think that a lot of scientific discoveries originate from initial observations made during engineering work or just out of curiosity without rigour.

Not saying ML methods haven't shown important reproducibility challenges, but to just shut them down due to not being "useful science" is inflexible.