In a field where progress is measured in tenths of percent points, that's not true. Think of it this way: the error rate drops from 19% to 15%, or from 1 in 5 to 1 in 6.
The ~5% improvement reported here might just be an artefact of the data collection or random variation, rather than a consistent repeatable change.
> The ~5% improvement reported here might just be an artefact of the data collection or random variation, rather than a consistent repeatable change.
You're questioning method or data representativeness, not significance. 250 samples is just about enough to for a 5% difference in NHST (stddev is around .4, so 1.64 sigma is .4/15.8*1.64=0.04 for single sided testing).