From 128x96 image. Another way to look at it is that on RPi you can handle 5 frames per second with 640x480, which probably means one frame every few seconds with the full RPi camera resolution. Sounds like the same results as python with opencv?
Also, the reason why we scale down the input image is not for cache effectiveness, it's simply for reducing computations needed. Maybe except for naive matrix multiplication and convolution implementations, which is what this repo does. But there is no point to discuss performance if you are using a naive implementation which by design ignores cache/instruction latency/anything Computer Architecture related. Please, at least take QNNPACK as baseline.