My quick read of the stats offered says that the smaller model's accuracy suffers considerably. I could definitely see running similar tests on it though!
All that said, the matching tensorflow offers from my understanding is also not exactly what I'm after. I'm primarily concerned with matching identical-to-humans images, possibly with small modifications such as size changes. Think more "are these two images identical" vs "give me pictures of dogs"