> How is this a failure by software?
The outcome - a guy being charged with a crime using bogus evidence is the story. The software may have been right, but the people who operate the software didn't trust it. Regardless, the story is trying to point out we might not be ready to put our faith in AI for criminal prosecutions at this time.