Google Brain Residency Program – 7 months in and looking ahead | Hacker News Reader

Google Brain Residency Program – 7 months in and looking ahead

research.googleblog.com

126 pointsstablemap58 comments

One of the things I like about the residency program, and google brain in general, is the openness of the research environment.

For example, it is easy for us to release open source code related our research, and in fact we are even encouraged to do so.

Some of us even continue to write personal blogs about machine learning research work done in the program.

In addition to open source projects such as tensorflow, people on the team also contribute and develop other open source libraries such as pybullet, rather than using proprietary, closed-source physics libraries.

Hey long time lurker and first time commenter here. May I ask if you could ping you for a few tips on my application. Right now I am finishing up a master's in robotics (mix of computer science and electrical engineering) at an Ivy league university (also did my undergrad here). I have previous research experience implementing (from scratch) and training convolutional neural nets. Right now I'm doing researching in natural language processing but not with RNNs.

This program sounds very competitive and I would love to get in, so any help is greatly appreciated!

Summary: Google Brain team hired a bunch of 1-year residents to do research with them. They've published 21 papers already and are loving it. Apply for next year here: g.co/brainresidency

A resident sounds very much like a post doc in the academic world.

This made me wonder how well they pay, but there is no mention of money anywhere, which maybe makes it _more_ academic than a post-doc :o.

application page:

https://www.google.com/about/careers/jobs#!t=jo&jid=/google/...

While a different program, Google's Engineering Residency reportedly pays about $100k, depending on location. I would imagine that the Brain Residency pays something comparable.

Aye. True story.

(in case it's not clear, Cinjon is a current brain resident.) And yes - brain residents are full-time engineers and paid as such.

They came to do a recruitment talk at my university. Per the spokesperson, they pay exactly the same as any other introductory job at google (so a hell of a lot more than academia).

Hi there! I'm Cinjon, a current resident in the Google Brain Residency. Feel free to ask any Qs and I'll answer them.

What's the interview process like?

Off the cuff, and probably a lot updated since, you should think of it like the normal Google interview but geared towards figuring out how well you'd do in the research environment. Prepare for the former but understand the latter.

So, coding algorithms and data structures in a white board?

On the phone (joking I have no idea)

What's your background? Given the rather stringent entry requirements, why do you think you were accepted? I'm guessing this is an extremely competitive placement!

Demonstrated interest. From peers in the program, I have found that to be the most consistent marker.

(MIT --> Startups --> Research --> GB)

Thanks, I'm curious as I have a PhD in computer vision and now work for a small company. It wasn't machine learning based, but all this stuff kicked off in the middle of it - had I started now, I would guess it would almost certainly involve deep learning as that's where all the money is. It's looking more and more like we're going to start leveraging deep learning at work. This sounds something that would be really interesting to apply for in the next batch (assuming it's around in 2018).

What does Google look for in applicants for this program? I have an undergraduate pure math degree and an undergraduate publication in mathematics.

My aim is to complete a PhD in CS. I'm currently taking CS coursework before I apply to programs (making up the difference from my math degree). My goal is to do research in machine learning and this program piques my interest. Wonder if someone like myself would qualify... looks potentially tailored/or preferred for those already in PhD programs.

I think that's a great background for the program and matches a bunch of the current residents. You should definitely apply. I suspect it will only make your later PhD experience better.

You should just apply. The people you're trying to impress are researchers and engineers, and they have a long history evaluating folks who have proven their merit outside of academia.

Ok great, thanks! Gives me hope ;-).

Is there a path for regular SWEs at Google to transition into a Brain-like role? I.e. with opportunities to do research, work on the problems you all work on.

Yes, there's the "ML ninjas" program, which brings non-ML SWEs into brain and other groups for a deep dive into machine learning through a project-based approach. (Often, but not exclusively, inspired by a problem from their originating area.). It's not a transition per se, but a great outcome is that the person becomes a bridge for their team between their specific domain of expertise and machine learning.

A full transition would be done by the usual transfer mechanisms. Two of my former students have switched into Brain from elsewhere in Google, for example.

On the residency program page it says Google will sponsor visas for eligible applicants, but the application page it says applicants must be eligible to work in the US. As someone who will be required a working visa, which one is it? Thanks!

Trust the FAQ. Note, of course, that sponsoring a visa isn't a guarantee of success, IIRC.

Why aren't any of you geniuses working on AGI? Such as applying some of these NN advancements to agents in diverse 3d virtual environments attempting varied/gradually increasingly complex tasks and learning based on low-level (grounded) sensory inputs?

Isn't that the point of DeepMind?

I'm a self taught somewhat prolific game developer and super interested in AI but have a poor mathematics background, think there's any chance?

I think I'll learn it on my own for now and try again next year!

Hey, Thank you for answering questions. I was wondering if any of you have blogs describing your experiences so far.

Also: I was wondering how much you interact with the other current residents in the program. Are you all together coming up with new ideas? Or are there multiple small group with their own interests...

Here's a great blog from one of the residents - http://blog.otoro.net/

It's like any other (research) endeavor with lots of people involved. Some of us share research interests and so we coalesce around efforts towards accomplishing those goals. Out of the resident pool, I am currently working with Ryan and have had some discussions about collaborations with Denny and Justin.

Take a look at the published papers though and you can see some pairs that have flourished by working in the same vicinity.

This is a wonderful part of the program though, and it doesn't stop with the Residents. Brain is such a great place for ideas to meld and for people to come together to collaborate; it's designed to encourage that.

Hi! Thanks for taking your time doing this. Are there projects in the intersection between deep learning and neuroscience? As mentioned there are people with neuroscience background but from the papers mentioned, I dont see something really neuroscience related.

GB (and Google more broadly) are very interested in this intersection. So yes, there are opportunities to do awesome work here, albeit actually working in a wet-lab is more difficult.

Hey Cinjon, I am wondering if I may ping you off line for a few questions about applying for this program? I am currently completing a master's in robotics and doing natural language processing research. And just heads up I also replied to hardmaru's comments with a more complete self introduction.

Given what you wrote to Hardmaru, I think you should just apply. The description you gave is a competitive one. Highlight it and then do well on the interview :).

For the residents that quit jobs in industry to do this program, what's next for them? How many residents are converting to permanent employees?

The current group doesn't end until at least June and so, except for those who just applied to grad schools, we're all continuing to do more awesome research here rather than starting to solidify what comes next.

That being said, this is an unparalleled experience in the field and a phenomenal road to travel regardless of what we decide to do afterward.

But as someone who has industry experience, what are the advantages of applying for the residency program over a normal perm position within Google Brain? The Residency program allows you to focus more on research than engineering I suppose?

It's true there are downsides to risking things like quiting your industry job to take a residency with the most prestigious industrial AI lab in the world.

I guess they'll always have the fallback option of taking a $500K/year position elsewhere.

This really makes me think Google is in some aspects the Xerox PARC labs of our generation... is any other company doing this kind of research investment?

Volkswagen, Samsung, Intel and Microsoft[1] all spend more on R&D than Google does[2] and put that funding into a wide variety of areas just as Google does.

There's also non-profit research think tanks like Battelle[3]

[1] https://www.microsoft.com/en-us/research

[2] http://web.archive.org/web/20160329190202/http://www.strateg...

[3] http://www.battelle.org/

Yeah, but if they have results like google they really manage to keep them secret.

MS research has historically been fantastic in computer vision.

The wide array of topics studied by residents reflects the diversity of the residents themselves — some come to the program as new graduates with degrees ranging from BAs to Ph.Ds in computer science to physics and mathematics to biology and neuroscience, while other residents come with years of industry experience under their belts

"Brain residents will spend the first two weeks of the program going through the Google Orientation sessions interlaced with introductory deep learning classes"

It would be great if, Google can make these two week introductory courses available for everyone.

This might be the closest thing there is to it: https://www.udacity.com/course/deep-learning--ud730

The wonderful Chris Olah teaches this, and in some ways he is by making them available through distill.pub. Check it out!

Why aren't any of these geniuses working on AGI? Such as applying some of these NN advancements to agents in diverse 3d virtual environments attempting varied/gradually increasingly complex tasks and learning based on low-level (grounded) sensory inputs?

Probably because they want to produce something interesting or useful within their lifetimes.

They are. The pace of research is not limited to six months.

I really want to work on this!

Thanks for taking time to answer questions. Can you please elaborate a little about the interviewing process?

Wonder how research this is, what if people are looking for more engineering oriented experience?

It is a very research-oriented experience, but you can definitely frame your work as more engineering-intensive. Quite a few of the residents have done some more really impressive engineering feats in the research domain.

If you are looking for an engineering oriented experience you should apply for a full time Software Engineer or Intern position.

Already a fulltime Engineer myself, I guess what I am looking at can be more phrased as applied machine learning, to solve production related problems. This is somewhat different than a regular engineering role though.

We have a few programs for this. As I mentioned above, the ML ninjas program is like an internal engineering nfocused residency for existing SWEs. There's also a huge amount of internal training courses to get up to speed on machine learning. There are, however, options for existing employees. (step 1: apply for job.)

The role of Software Engineer at Google is boarder than in other companies. Applied Machine Learning and actually even most research is done by Software Engineers.

Reply on news.ycombinator.com