An interesting idea. Yes, you can usually use a convnet with any number of channels per pixel.