naturalgradient

So as someone working in reinforcement learning who has used PPO a fair bit, I find this quite disappointing from an algorithmic perspective.

The resources used for this are almost absurd and my suspicion is, especially considering [0], that this comes down to an incredibly expensive random search in the policy space. Or rather, I would want to see a fair bit of analysis to be shown otherwise.

Especially given all the work in intrinsic motivation, hierarchical learning, subtask learning, etc, the sort of intermediate summary of most of these papers from 2015-2018 is that so many of these newer heuristics are too brittle/difficult to make work, so we resort to slightly-better-than brute force.

https://arxiv.org/abs/1803.07055

An interesting effect I have noted for me when doing research is that even if it only takes one extra click or changing one parameter to generate a plot, or run some extra analysis, I will only do it selectively and start rationalising it to myself that I know when a certain plot or analysis will help and when it won't.

This becomes especially egregious over month to year long experiments where I run the same experiment every day on end.

There was really no reason not to auto generate every possible plot, every possible analysis every time (and I cannot use ipython notebooks or things like that because it's many distributed things chained together with lots of scheduling).

The productivity gains have been enormous and are hard to overstate. I don't dread any experiment any more because even in a large complicated distributed setup, everything from initialising kerberos tickets to tons of config files, restarting services, running multiple experiments dependent on each other, and generating plots and summaries and committing them to a repo is one command. Anything that's analysed once is evaluated always.

I now almost look forward to setting up new experiments because of the pleasure I get from just chaining together calls from my control utilities.

All I have to do is pull on my laptop and I download a filter with all results pre-generated paper ready. I think a lot of people do this in experiments where everything is on a single machine, but I haven't seen it as excessive from other phd students doing complicated distributed stuff. There is always a lot of manual command line args passing, manually changing some config while instead of just creating dedicated scripts, etc.

Me too, at work (PhD student) I take enormous pleasure in writing end to end fully automated pipelines. Nothing that needs to be done more than once isnt automated pretty much. At home, I would never buy or use an Alexa/Google assistant/use Siri or any 'smart' device, I simply don't want to bother with it.

I came here to post this!

And of course the (sort-of) French equivalent about life in modern times: In search of lost time.

This is sort of what the report of the week ('reviewbrah' [0]) is doing, no? Reviewing fast food in a quirky manner. Million+ subscribers on youtube, really interesting to have followed this for 5+ years when he had a few hundred subscribers.

Doing the exact same thing with very small refinements, over and over. It's not that glamorous, but I reckon at 1M+ subscribers he is doing quite a lot better than many obscure luxurious lifestyle personalities.

[0]: https://en.wikipedia.org/wiki/TheReportOfTheWeek

So given the choice of a comfortable, safe, environmentally responsible mode of transportation, you chose to go for the option where you get to burn lots of fossil fuel and move at a speed in a metal box at which you cannot control it any more if anything happens and which you had no experience in, putting your own and other lives at risk.

As a German, I wish we would impose strict speed limits on the Autobahn to avoid tourists, or anyone for that matter, do such wasteful dangerous joyrides.

Note: have been in a car accident so appreciate I am very very biased against how offhand and casually people go about such behaviour, how lives are destroyed for no reason.

I am not predicting anything, this is how it has been implemented. You are aware this law is already in effect and thousands have been deleted/banned?

But this debate is pointless because you clearly neither speak German, understand German law, nor are interested in revising your opinion that this is anything other than a fight of the righteous against the bad. The law is likely unconstitutional, widely overreaching, and does in fact delete these things:

https://verfassungsblog.de/das-netzdg-und-die-vermutung-fuer...

https://netzpolitik.org/2018/die-meinungsfreiheit-und-das-ne...

https://www.zeit.de/digital/internet/2017-03/heiko-maas-gese...

There are countless examples, e.g. Facebook censoring satire accounts:

http://meedia.de/2018/01/03/netzdg-twitter-sperrt-account-de...

You are celebrating authoritarian censorship because you think it targets the right people. You are calling me out for something you have no firsthand knowledge or expertise on because of tribalism. Stop.

You seem to be completely uninformed regarding the actual implementation of the law. Any criticism of anything can and will be deleted under the scope of the law. Want to criticise any policy?

Hatespeech. Criticise religion? Hatespeech. It would of course never hold up in court, but that doesn't matter, for social media platforms deleting anything that could be construed as hate by anyone who feels offended is the path of least resistance.

You are not made accountable for breaking laws that already existed, you are being censored for something that is not illegal, never was, still isn't, but could offend someone if it's disagreement on a sensitive topic. It's the end of any sort of free speech on social media in Germany.

I don't have a feeling anyone wants him to go away, me and most of my peers (PhD) students hold him in rather high regard. Also feeling he is right about the fact that so much deep learning work is just derivative and marginally improving over original LSTM work.

This is an uncomfortably biased article, I think

https://www.nytimes.com/2016/11/27/technology/artificial-int...

is much more balanced.

I have a 2015 Macbook Pro and Im sort of anxious because if it breaks, I really don't want to get a butterfly keyboard one. Is there any idea when they will release a successor?

From my observation (from knowing a few people there from my lab), DeepMind is still organised relatively similarly to university groups.

Professors moving over are made to lead research teams, and while there may be some overall organisation goals, researchers still care about their individual career advancement and prestige as measured by paper output. So individually, most people working there are incentivised towards doing what is best for their individual prestige in the research community for after leaving deepmind.

If you are making that argument, you should refer to the ICLR deadline, which is well earlier, but still after July 2017.

You can play the 'work towards the next prestigious thing' game as long as you want but you will be happier the earlier you quit it.

At Oxbridge, DeepMind is definitely the preferred employer, alsmost nobody is interested in an academic career, finance is not especially popular but always an option in London, hedgefunds heavily recruit. I would say next popular option after DeepMind or big tech in Silicon Valley is London startup scene for many.

As someone entering the workforce soon, it's very disappointing to see teach leaders who enjoyed the spotlight on their elevator ride up completely disappearing and pretending they have nothing to do with what they brought onto the world.

Even if you pretend you didn't know (and at the top, it's your responsibility to know), you can't pretend not to know going forward. Quit. Do something.

Had a lunch conversation with other CS PhDs today (mostly in machine learning) - Facebook is so out, everyone actively spoke out against working there. Was uplifting to hear.

My apologies, I thought from a throwaway this was some additional information and had not read the article yet.

Do you have a source? I am not doubting this, just curious on the connection.

Honestly I don't think I need to do, I have interviewed many people for doing very code heavy research projects and not once have I felt asking them to do a white-board code exercise would have contributed to my evaluation meaningfully.

Love this, although not living in that area.

Just to chime in: I'm a PhD student in CS/machine learning with a competitive enough CV to interview anywhere, and I have gone through the motions at many companies previously (e.g. for interning), and there is nothing I hate more than coding interviews.

It literally makes me hate companies, it turns me off from interviewing at a place, and it makes me deeply resent them even if I get an offer after a ridiculous series of interviews.

After you have subjected me to a terrible/demeaning interview process, I might still work for you, but I will never be a loyal or feel any inclination to contribute more than I need for personal gain, because you have established that nature of the relationship via the interview. Sorry for the rant.

I hope you succeed, so very much.

No I am absolutely certain, I was offered more than 2x for the same position by another tech company.

I was so surprised by Amazon's offer I conducted a small survey amongst friends and acquaintances to find which of my offers (there were more) was 'normal'.

The thing is, you can, as Amazon shows, have an enormously successful business by treating employees badly and not paying top of the market. Of course you can. Paying more does not always get you the best, and for many many tasks you don't need Jeff Deans, you just need someone to work hard and get it done.

There is tons of talented people willing to work for less than the top to get a chance to prove themselves, to learn something, to get their shot. It's just that they tend to leave Amazon (c.f. retention rates) after that, and Amazon seems to be doing ok with that. It's just a fundamentally different approach to employee management.

"The compensation for more experienced employees is substantially higher"

Citation needed. Higher than compensation for experienced employees at Google/Facebook? I am not even disputing Amazon pays more later, I am saying Amazon never pays top tier AND treats employees badly AND has no perks or particular benefits to make up for it, so nobody who has the option (that I know of) wants to work there.

Well compensated? Amazon pays significantly below other mega cap tech companies, and as a PhD student in machine learning with a lot of friends interning and applying for jobs I can confidently say Amazon is nobody's first choice due to low salaries and working conditions.

I'd rather say I am amazed they get anyone great working for them because it really is clearly and painfully obvious that almost everyone would rather work for Google, Facebook or Microsoft regarding machine learning.

Have first hand knowledge of salaries. Amazon offers 30-50% less than other top tech companies for machine learning. There is literally no silver lining or upside as far as I can tell.

In that context, it does not matter if Amazon pays more than <other job> in <other industry> or treats employees better than <other company> in <other industry>.

If you can go short on either one, you can create any bet via a weighted + leveraged long/short pair bet. I think bitfinex allows this.

Well, that is the thing right?

People historically went to work in any number of unethical organisations, including industrialised mass murder, human experiments, knowingly poisoning or defrauding people..the list is endless, and it was just any other job to them.

The structure of an organisation means nobody feels individual ethical responsibility, they are just doing their job after all.

In fact, I would say most corporate employees feel like that about the large anonymous bureaucracies they work in, but there is still a good number of people for whom this is unacceptable, the type that ends up saying no to hush money, the type that whisteb-lowers, the type that ends up being shot for part of a resistance against a dictatorship, and so on.

People again and again do this even though it often has terrible consequences for their personal fate, and I think this is the exact same kind of person who does not want to hire or work with the other type, the one that just does their job.

I think developers and really anyone dealing with Amazon needs to understand that there are no 'win-win' business relationships with Amazon. If you win, Amazon will try to take it away from you eventually.

It stuck out with me because it made me realise that many students were not at all emotionally well equipped to handle certain situations of adult life.

A problem here especially is how quickly anything done by the university is framed as an attack on the mental health and safety of students.

For example, in a recent survey, students were asked about their mental health issues. Examples of the university not being supportive enough were things such as 'I was not given an extension on an essay'.

These incidents make me realise that a non-trivial fraction of this generation of students really feels intense emotional distress to a point of needing counselling over things my generation would book under normal aspects of adult life. In these cases, there is also little self-reflection on one's own responsibility to handle such situations - everything is framed as an attack.

Reporting from Oxbridge:

This is a real issue but actually most people have reasonable notions of free speech. It is just that the types that end up being welfare/student/women/minority officers are extremely aggressively attacking perceived attacks/hate speech/dissenters/traitors to the cause.

If anyone remembers the Tim Hunt story and the witch-hunt that followed (and let us not forgot the role the Guardian played in peddling that), there was a situation where Tim Hunt was supposed to give a speech at a college.

The women's officer was outraged and riled up freshers who where thoroughly convinced that a man accused of making a sexist joke merely speaking in a room inside their college was a violation of their own safety, a phrase I recall is 'The college is our home and our safe space'.

Most faculty/graduate students/undergrads are actually very reasonable and want to hear dissenting opinions. The problem is just that most well-adjusted people quickly tire of outrage-driven student politics, so it's left to the ultra-aggressive types.

I completely agree, the code of conduct has been used multiple times as a trojan horse in the sense that it only contains things nobody could reasonably disagree with (because who would not want a welcoming environment), only to then be used to shun /outcast people based on views particularly on gender and diversity. This is one of the issues Sam Altman referred to in his blog that simply cannot be talked about in Silicon Valley currently.

It's not my intention to not work with women. I do collaborate with women. I am just saying if I am being put in a position of power in teaching, and I can choose between male and female undergrads, I would rather not be in a position of power over females in this current climate because we are at a hyper-sensitive stage.

There is no specific gain for them being taught by me instead of a female PhD student, and there is only risk for me, so all things being equal, I am suggesting that this is a stance many men might being to take or at least find themselves wondering about.

And you are virtue signalling by taking my comment ad absurdum and describing a situation to which nobody could reasonably respond with 'no'.

Think about this and the divisiveness of this debate. In phrasing like yours, no grey areas exist. It is the bad versus the good. Read the actual article I linked written by a woman (identity politics makes me say this, because as a woman she has more voice than I).

I am describing a chilling effect and a reaction I observe in myself and suspect in others, you respond with 'How dare you' in an outraged attempt to silence a debate. What is this contributing other than making yourself feel good about your righteousness?