mdda

Look at it from an interview perspective. If I ask "are you interested in exploring ML", and you're enthusiastic, my next questions are : What have you done? Have you taken any courses? GitHub? Blog Posts?

If the answer is that you're waiting for a special sign that it's worth doing before making an effort, then that really tells me that your enthusiasm for doing ML is not reality-based. Doing the ML thing is a pretty different mindset from other software jobs.

Hmm. There are weekly similar counterexamples among lottery-ticket purchasers. However, don't invest your life savings in the lottery - the optimism factor is a killer. Better to invest your savings / future in something more rational.

I saw the same thing being done with a theatre in Manhattan in 1998 (to make way for the Disneyfication of Times Square) : https://en.wikipedia.org/wiki/Empire_Theatre_(42nd_Street)

I don't know of any chat logs.

But it was very interesting to see the 'next response' candidates for the two sample chats in Table 1 (p3 of the PDF). In particular : it was alarming to see how much their Deep Learning response selection mechanism had to chose not so much the best response out of a selection of decent responses, but more the most acceptable response out of a selection of mostly horrible ones.

And there are even a couple of photos within the closed bit of the Station : https://www.google.co.uk/maps/@51.5046516,-0.1479124,3a,90y,...

He may have sent an actor along to represent him for the interview / photoshoot.

At a typical 'Ballroom beginners class', there will be no-one there who can already dance (i.e. if they were already great then they'd be in the next class up, etc). But also most people will have two left feet : Happy klutzes all around, without needing the excuse of alcohol to feel liberated. And there's always something to talk about... People are there just to have some fun, but don't feel the need to go 'CRAZY'. It's a nice atmosphere. Perhaps go for an icecream afterwards :-)

Do you have to pay tax on your German earnings (earned in Germany, which you already pay tax to the German authorities on)? Almost all countries only tax you on the money earned (approximately) within their borders. So for most 'expats' it amounts to taxing interest and dividends, and maybe rental income on your old house.

Not so the USA, which has a claim on all money earned by Americans world-wide. There's a ~U$100k earnings exemption (so many people aren't paying much tax back to the USA), but high-earning USA expats are in a different tax-world from their overseas colleagues.

> NN for image classification task

You've created your own straw man here.

> "You realize now how computationally intractable this task is on modern hardware?"

Here are the people that prove it isn't computationally intractable : https://blog.openai.com/evolution-strategies/ - but to say they've discovered a new breakthrough method is over-selling the result.

People have known that training NNs (for any purpose) using evolution works well since the 1990s. The rise of the NN frameworks has made doing differentiation much easier now than it was before (and having gradient hints is intuitively a good idea). But for OpenAI to allow their PR people to declare this as a novel advance is ... surprising.

But the judge wasn't using the newspaper report as part of his consideration. He was just offering it up as something that the defendant should be willing put into the court record (instead of taking the Fifth). The fact that it appeared in a paper is, OTOH, clear-as-day evidence that stuff is not being shown to the court, which he's saying has some value as a datapoint on its own (irrespective of the content).

Home (lead is a teenage girl, other major character is not a love interest, nor actually male).

Up (about a guy escaping being sent to an old people's home .. or something).

I can't speak for llimllib, but I didn't read their comment as being a reward for having a bath. The key bit of the suggestion is that having a bath is implicit in the whole situation.

Similarly, me (+spouse) have never offered any reward for completing any task - and see many other parents having to enter into a what-if kind of negotiation. I like your Mercury example, but would consider taking it further: "I can't tell you about Mercury until you're dressed". It sounds logical enough to a 5-6 year old (even though these things are not logically connected).

I guessing these tactics will have to change at some point - but we've successfully navigated around most of the tantrum stage so far.

Wow - this brings back memories... When I was an undergrad, I took a summer job at Perihelion writing demos for the ATW.

From what I recall, things weren't going great, and most of the staff was grumbling about the Atari side of the machine. Since Atari was where the latest funding had come from, there was a strong desire by management to show how the Atari hardware (display, disk interface, etc) was essential to the whole set-up. Whereas from an engineering point of view the Atari front-end was pretty much dead weight (vs. the Transputers that hung off the back).

Less than a year later, Perihelion went into liquidation, and a friend and I bought the contents of the building (boxes like 'Misc Electronics #14' and 'Papers #5'). We spent the following summer holidays using the customer list (i.e. the Papers) to flog the remaining Transputers (in the Misc Electronics)... Fun (though slightly morbid) times!

Rather that talk to other people, you're literally talking at them.

It's just after 5m45s : https://youtu.be/yq0ecBGg5Q0?t=5m45s

FWIW, my guess is that a lot of the novel stuff that has been released in the last week is because of the impending ICLR deadline (Friday). The review process for that conference allows the papers to be updated until the reviewers' decisions are made. So getting the text 'finalized' isn't an essential step right now.

I'm originally from the UK, but feel that 'healthy skepticism' there can all too often be the label that people give "why bother, it'll probably fail anyway". The boundless optimism of the US may be more beneficial in the long run.

I gave a talk a PyConSG this year[1], which included a demonstration of training a Reinforcement Learning model on a 'Bubble Breaker' game. There's also more detail available[2].

The Jupyter notebook is included in the GitHub repo[3], and includes a 'scaled down version' that takes ~5mins to train on a MacBook's CPU. There's also a downloadable 'full scale' model that was trained in ~7hours on a Titan X. It plays the game (on average) better than me...

[1] http://blog.mdda.net/ai/2016/06/23/workshop-at-pycon-sg-2016 (has slides, and YouTube link) [2] http://redcatlabs.com/2016-07-30_FifthElephant-DeepLearning-... [3] https://github.com/mdda/deep-learning-workshop : have a look at notebooks/7-Reinforcement-Learning.ipynb

Are they different by a factor of 10 or more? If not, then they're within an order of magnitude (base-10).

It depends on whether the simulations are on fixed-length 'books' (the analogy used in the post) or on something more self-structuring (like Genetic Programming as a simple model, or the actual case of DNA coding for protein / cell / body assembly).

You'd also need to robustify the compiler itself to bit-wise errors, and its output.

But also imagine that the algorithms you write had to work (perhaps slightly differently, but essentially the same) when hit with bit-flips in the code. How would one do that? Perhaps have multiple copies, or coding redundancy, or fix-up routines, or fall-back instructions. All the kind of stuff that mother nature already figured out. But that should be unsurprising, since without that robustness, the evolution thing would kill our progeny almost every time. Building the robustness is almost as important as building the functionality.

The author (and everyone) should play with Genetic Programming. This is where programs (i.e. expression trees) get to mix together to produce new expression trees, according to their 'fitness'. It's an extension of the regular fixed-length Genetic Algorithm stuff, but the structure of the representation itself is adaptive.

One surprising thing is that (in addition to fitness improvements over time) the expression trees evolve 'robustness', since there is a subtree survival advantage if a tree's descendants are not disrupted by the crossover/mutation operators. That is to say, the process learns/evolves to evolve better.

So it shouldn't be so surprising that nature evolves a process for self-healing DNA errors, etc - it can be demonstrated in-silico pretty simply (i.e. occurs even if you don't 'engineer' for it).

If you put your discrimination detector on an API, you would enable the original model's creators to eliminate bias by training against it. Resulting in an anti-discriminative model, through a generative / discriminative process.

s/TensorFlow/a deep RNN like this/ would make more sense.

TensorFlow is just a framework (as are Theano, Torch or DL4J) for expressing the network architecture. Framework:Network ~ ProgrammingLanguage:Algorithm

But if you added (say) a spring into the linkages, the rest-position could be tuned mechanically to be more energy efficient (e.g: zero power). The effect of the spring could be backed out in software.

But it's not until people have played with the use-cases, etc, that one really needs to think about efficiency in terms of battery life.

Thanks for posting the Reddit link : I hadn't previously realised that the quality of the commentary in /r/MachineLearning was so high.

I came across your link I had "check the real-ness of SARM" on my TODO list (along with the v1 of the PDF on my reader). Now to try and parse through what was real about the method and what was hubris...

Since there's no link at the of Part 5 to part 6:

Part 6 : https://bothsidesofthetable.com/sorry-guys-it-s-the-size-of-...

Part 6cont : https://bothsidesofthetable.com/pitfalls-in-market-sizing-pa...

Part 7? : https://bothsidesofthetable.com/pitching-a-vc-dealing-with-c...

... and then the numbering / search functionality seems to hit a dead-end. Pointers welcome...

Hmmm - I'm also thinking that this is one of those things that probably has a much better explanation - and the science/maths will (hopefully) backfill why it works so well.

I half-remember from somewhere that as long as the gradient descent direction has the correct sign 'in expectation', then the SGD will ~work. So there's a whole lot of flexibility in there for having a good idea that at least doesn't fail horribly.

For instance, in other DeepMind work, they do lots of asynchronous weight updates - and the accuracy decrease from ignoring any kind of 'locking' is dwarfed by the speed increase of being able to run more stuff in parallel.

Another image I can't shrug off is that of Q-learning in a game, where the updates implicitly pass back from 'the future' (which also works ~better than it should). In this case, the linear model would just be an estimator of where the update values are going to land...