A friend of mine from academia was considering going into industry, so went to some data science meetups. Someone was giving a presentation about convolutional networks, yet did not know what a convolution was. At first I was startled, but in the long run machine learning applications will be decided just as much on user experience and design features as on algorithmic choices.

I'm not a web programmer, but I imagine few developers could remember the mathematics[0] of the sorting algorithms that are fundamental underpinnings of their work (if they ever learned them at all). Yet I'm not sure it matters, even to great developers. The same thing will probably ultimately be true of machine learning. Honestly, you need not know what a convolution is to build a perfectly usable convnet. (And ultimately you may not need to even build your own if you can use a nifty amazon API.)

Whether NIPS should care or not is a separate story. It seems a little sad - I took all this hard pure math as an undergrad, and it doesn't seem to be important if all I'm doing is changing a few layer parameters (even if the change is ingenious).

[0] mathematics as in proof of sorting, proof of bounds on space/time complexity, etc.