Exactly. The problem is that it's hard to find a good balance between having too many hyperparameters, versus basically telling it the answer beforehand. In particular, it was hard to figure out how to let it learn the stride of the convolution, but perhaps we could have done without it.

The other problem is that to get a reasonably-sized convnet by pruning and tying a fully-connected net, you need to start with a huge fully-connected net, which slows things down.