This is mainly due to breeding. Dogs can vary between 8-80 lbs, depending upon breed - some will fit in handbags, others will barely fit into a car. Cats, on the other hand, have significantly less variation (between 8-25 lbs[1]).
Further complicating the problem is that we have bred some dogs specifically for facial features and shapes. An English Bulldog, for example, has a drastically different face than a Labrador.
However, the vast majority of cat breeds retain the same facial features and shape. Those that do vary (e.g, Siamese cats) differ by small amounts in comparison to dog breeds.
This is why building a "dog or cat" detector is reasonably straightforward (if it's not a cat, it must be a dog), but building a "dog, cat, or other" detector is far more complex.
[1]:http://www.petobesityprevention.com/ideal-weight-ranges/