Open-Sourcing Bit: Exploring Large-Scale Pre-Training for Computer Vision
ai.googleblog.com
ai.googleblog.com
You can take a 15MB mobilenet model, add a layer at the end, fine tune with half a dozen examples of a few different image classes (in a few minutes on a consumer grade laptop), and recognize lots of different examples in real time with a web app reading continuously from a webcam.
The advances made in Computer Vision in the last ten years are mind blowing.
JFT is the exception. I find JFT interesting so I pay close attention to anything using it, and as far as I've noticed, no model has ever been released based on JFT, going back to 2015 at least when it was much smaller. It's always either held back or the released model is based on public datasets (eg BigGAN - released G was on ImageNet though the paper notes that the JFT BigGAN completely avoided divergence problems, which is very interesting). I've wondered if legal/copyright issues block any release: there's always someone who tries to argue that a model is a derived work, and nothing in the JFT-300M papers mentions having licenses covering public redistribution.