Hi everyone! Author here. Let me know if you have any questions, this is one of my favorite subjects in the world to talk about.
My first thought is that it makes sense that averaging together a bunch of local predictions would work well on the ImageNet task, since the different classes tend to have obviously different local textures, and class-relevant information makes up a large part of the image. I would be very curious to see if the technique is as competitive for other tasks.
Or are low-level features the "easiest" part for the network to learn?
Edit1 : I understand of course the academic purity of working from raw data.
Edit2: so simulated means lots of samples, on policy learning, but also very cpu intensive.
I'm not sure what the most interesting problems with that property are. Maybe making specialized classifiers for people based on personal labeling? I've always wanted e.g. a twitter filter that excludes specifically the tweets that I don't want to read from my stream.
So Google/Bing/etc have their own pre-trained models for translations.
How would I access that in order to develop my own refinement w/ the domain specific dataset I put together?
For example, https://github.com/google-research/bert (the multilingual model) might be a pretty good starting point for a translator. It will probably still be a lot of work to get it hooked up to a decoder and trained, though.
There's probably a better pretrained model out there specifically for translation, but I'm not sure where you'd find it.
o the lower the level of representation the more important it is to increase the level of abstraction by learning or defining manually new features
o in the presence of (fine-grained) raster (automated) feature engineering is especially important. Therefore, feature engineering is important in audio analysis (1d raster) and video analysis (2d raster).
According to them, transfer learning can improve a time series model if you pick the right dataset to transfer from, but they don't seem to be getting the same unbelievably strong transfer results that you'd see on images and text.