Newer Android phones allow you to set an accessibility menu shortcut. If you only have one menu item selected, the shortcut just toggles that menu item. I press vol-up & power to toggle greyscale.

I'm not sure what kind of phone you have, but on an iPhone you can set your phone to toggle B&W and normal color by triple clicking the home button.