1. Building special-purpose hardware for neural nets is a good idea and potentially very useful.
2. The architecture implement by this IBM chip, spike-and-fire, is not the architecture used by the state-of-the-art convolutional networks, engineered by Alex Krizhevsky and others, that have recently been smashing computer vision benchmarks. Those networks allow for neuron outputs to assume continuous values, not just binary on-or-off.
3. It would be possible, though more expensive, to implement a state-of-the-art convnet in hardware similar to what IBM has done here.
Of course, just because no one has shown state-of-the-art results with spike-and-fire neurons doesn't mean that it's impossible! Real biological neurons are spike-and-fire, though this doesn't mean the behavior of a computational spike-and-fire 'neuron' is a reasonable approximation to that of a biological neuron. And even if spike-and-fire networks are definitely worse, maybe there are applications in which the power/budget/required accuracy tradeoffs favor a hardware spike-and-fire network over a continuous convnet. But it would be nice for IBM to provide benchmarks of their system on standard vision tasks, e.g., ImageNet, to clarify what those tradeoffs are.