Google Brain Residency
tinyclouds.org
tinyclouds.org
I disagree with him about needing a "Rails for deep learning," I think it's quite fine now in practice, especially since DL code is typically significantly less than a web app. A file for data pipeline, a file for your model, and a file for training and/or inference. Not sure it really needs to be much more complicated than that.
People forget that the reason Deep Learning came to prominence was engineering that enabled use of GPU's (Thanks to Alex K.) and large dataset from ImageNet competition (Thanks to Fei-Fei Li).
Importance of software engineering in moving the field forward is often under-appreciated. This blogpost beautifully illustrates several instances where a great implementation made it easier for researcher to speed up experimentation and lead to breakthroughs in Computer Vision. [1]
I am building Deep Video Analytics which aims to become the Rail/Django/MySQL/(favorite analogy) of Visual data Analytics [2,3].
[1] http://www.computervisionblog.com/2015/01/from-feature-descr...
There's plenty of horrible code out there, particularly tensorflow code, which matches all the classical metrics for 'spaghetti code'.
The complexity of the implementation might be hidden away by the framework, but there's no excuse for writing massive hundred+ line functions that do multiple things with copy-pasted code blocks.
That's just bad code, in ML or not.
I think they accepted residents from different backgrounds and different levels of experience.
Why would you assume that a temp job at an advertising company would be "more advanced" than work conducted at an actual research institution?
Comcast isn't just a cable company these days, but it is still a cable company.
They also created the most power NN processing silicon that uses far less power than anyone else. A pod is doing 11.5 quadrillion FLOPs is pretty impressive. Just sayin.
It's not even the kind of fundamental research that places like Xerox PARC, IBM, or Bell Labs used to invest in.
Frankly, it's kind of insulting to imply that a one-year Google residency would be "more advanced" than years-long graduate programs at places like MIT, Caltech or CMU.
However the twin prime factorizer idea was interesting and could potentially lead to some factorization speedups (based on heuristics) if done correctly.
Either way though, prime number distribution is quite tricky - we should probably be looking in other number bases as well rather than just base 10.
Deterministic primality tests are rarely used in practice -- they're too slow, and the probabilistic tests have such a high accuracy that it doesn't matter.
This interested me so I went looking for how primes would work in other bases, this answer implies that primes are the same regardless of base:
A prime is a prime no matter which base you use to represent it. On the surface one might think that in Hex you would have 35 = 15 as "usual," but it really turns out that 35 = F.
The example 21 doesn't work too well because it is not prime. The base ten number 37 is better, because it is prime, but its Hex representation is 25, which sort of looks non-prime. Hex 25 is not, however, repeat not, 5 squared.
Okay, enough for examples. The fact of being prime or composite is just a property of the number itself, regardless of the way you write it. 15 and F and Roman numeral XV all mean the number, which is 3 times 5, so it is composite. That is the way it is for all numbers, in the sense that if a base ten number N has factors, you can represent those factors in Hex and their product will be the number N in Hex.
Relating to your question about base 13, the base ten number 13 will be represented as "10" in that system, but "10" will still be a prime, because you cannot find two numbers other than 1 and "10" that will multiply together to make "10".
I hope this helps you think about primes in other bases.
Right. If you have some quantity of apples, then can you lay them out in a regular rectangular-shaped grid (more than 1 apple wide)? If not, the quantity is prime. You can count the apples in any base you like.
Wow, where does it stop, then? Can we upsample modern 4k films? Or deeper dynamic ranges? Stereo projections from mono images?
I suppose that the limiting factor is the resolution/color depth/perspective channels of the training images?
(Actually I think twice as close might be 16k, but you get the point...)
How do these sorts of techniques perform with respect to temporal coherence?
If I have to choose a channel, or even choose whether to turn the TV on or off, there's still room for improvement.
The program invites two dozen people, with varying backgrounds in ML
... "with varying backgrounds in ML" -- does that mean that relative beginners have a chance of being accepted?
(I have a personal stake in this! I'm an academic researcher who is getting into ML and working on legal applications, and I'd totally apply for this if I thought you didn't have to basically be Andrew Ng to get in...)
I haven't met the other residents yet (the program doesn't start until mid-July), but based on a few online interactions and meeting some of them at the interviews it seems that a fair number are undergraduates or grad students with some research experience in ML or computer vision. Others seem to have backgrounds more similar to mine, i.e., they were researchers in physics or neuroscience or something and are transitioning to ML.
An experienced machine learning researcher like Andrew Ng would probably not join the team as a Brain resident. We hire experienced machine learning researchers and engineers all the time (see https://careers.google.com/jobs#t=sq&q=j&li=20&l=false&jlo=e... ) and the residency program is probably not appropriate for people who are already experts. It is a program designed to help people become experts in machine learning.
For residents we look for some programming ability, mathematical ability, and machine learning knowledge. If an applicant knows absolutely nothing about machine learning, it would be strange (why apply?). We accept people who are not machine learning experts, but we want to be sure that people know enough about machine learning to be making an informed choice about trying to become machine learning researchers. Applicants need to have enough exposure to the field to have some idea of what they are getting into and have the necessary self-knowledge to be passionate about machine learning research.
You can see profiles of a few of the first cohort of residents here: https://research.google.com/teams/brain/residency/
See the old job posting which should hopefully explain the qualifications: https://careers.google.com/jobs#!t=jo&jid=/google/google-bra...
Loved the article though. Dashed cyberpunks dreams notwithstanding.
Link to project: http://www.wave.one/icml2017/
This has been done before surely, like compression with varying decompression algos the characteristics of which are sent with the compressed stream to enable more optimal reproduction of the originally encoded data??
AFAIK Spotify uses Vorbis for their streaming. Perhaps if they switched to Opus things would sound better.