Just curious: how come the total observations of the confusion matrices are so different? Shouldn't the same data set be used in each trial to fully evaluate the performance differences? Maybe I missed some details in the article...
This was written as an after-the-fact look back after a year long project and the target testing set did change a little along the way. The big change was splitting by manufacturer (and eventually by model family), which changed the number of applicable samples to compare performance on.
I cut some stuff from the article for length, so I totally get how that could have been unclear.