On the other hand missing completely it in 3 out of 30 tests (samples with a number of selected "optimizations") is a "terrible" 10% is
awfully worse (in an alarm system).
>For the experiments, we selected 30 scenes from the previously used videos with the next requirements. Each scene is made up of at least 5 frames, filmed in a fixed scenario, i.e., in the same place, and the pistols are clearly visible to a human viewer. ...
The model successfully detects the pistol in 27 scenes with an average time interval AATpI=0.2 seconds, which is good enough for an alarm system. The detector fails to detect pistols only in three scenes.
I am probably particularly picky but (from the paper):
>This section analyzes the performance of our best detection model on seven public videos, with low qualities, downloaded from youtube. Six of them are pieces of well known films from the 90s, James Bon: The World is Not Enough (video 1), 3 pieces from Pulp Fiction (video 2, 3 and 4), Impossible Mission: Rogue Nation (video 5) and Mister Bin (video 6). The seventh video is a long pistol threatening video (video 7).
... if you cannot spell Bond or Mister Bean, and you classify Mission Impossible: Rogue Nation as a 90's movie (calling it Impossible Mission) you don't look like having been particularly accurate when writing the article, if the same accurateness is in the actual numbers, there is a problem.
Anyway the "final" sample (30 scenes) is so small that it makes very little sense to derive anything from it.