I have wanted to do exactly this for a long time. E.g. Bear is great but I want to make all kinds of modifications to it but I can't.
How do I follow along?
2,464 karma · joined July 25, 2010
https://tali.link
https://twitter.com/taliesinb
How do I follow along?
While the practical notebooks we used in 2018 were already amazing, my colleagues and I worked hard on expanding and improving them for the 2019 Indaba. In case anyone wants to check them out, here's a full list of the practicals (the links go to Google Colab notebooks)
1a: Machine Learning Fundamentals, http://bit.ly/2nfeu4W
1b: Build your own TensorFlow, http://bit.ly/2obD1rK
2a: Deep Feedforward Networks, http://bit.ly/2oQSjSQ
2b: Optimization for Deep Learning, http://bit.ly/2nqkyHB
3a: Convolutional Networks, http://bit.ly/2nqdQRX
3b: Deep Generative Models, http://bit.ly/2nnkisU
4a: Recurrent Neural Networks, http://bit.ly/2ncgYkx
4b: Reinforcement Learning, http://bit.ly/2oO4lMK
They're also available on the official github page: https://github.com/deep-learning-indaba
We'd love any feedback on the practicals to improve them for next year!
If you have some ML expertise and would like to volunteer to be a tutor at next year's Indaba, which will be held in Tunisia in late August 2020, drop me a mail at contact (at) taliesin.ai. I also encourage anyone to apply to attend as a student!
Really? The gradients computed by AD are the exact answer to the following question: if I were to change this input or parameter an infinitesimal amount, how much would it change the output of my function? That is always meaningful (when it is defined), and means what I just said. You can easily make functions where it is not defined, of course, just like you can make a sphere into two spheres with the Banach-Tarski theorem!
But there are vast, vast forests of numerical computation employed in industry, science, finance, engineering, where it is almost always defined.
And even for more “chunky” computations where the non-differentiability is more severe, there are algorithms like REINFORCE that you can use to estimate gradients through these parts.
Also, from the GP’s thought about applications beyond DL, my favorite examples so far are for model-based RL [1] and Neural ODEs [2]
For sensitivity analysis it might be disastrous to conclude that an output is sensitive to an input when it is actually not, merely because an intermediary ReLU hit 0, for example.
A conservative approach could be to define versions of the relevant functions that threw exceptions at such points, or that also calculated the trusted margin of the resulting gradients; non-differentiability would then produce a zero trust margin.
Moreover, a statically typed language like Swift is a much better starting point for this kind of effort than Python. Array shapes and dimensions are already a type system - you might as well go the whole distance and get all the other safety, readability, and efficiency benefits!
PS shout out for named array axes as the future of array-based (and hence differentiable) programming... see http://nlp.seas.harvard.edu/NamedTensor for a good rationale
You can find out more on the mailing list https://groups.google.com/a/tensorflow.org/forum/m/#!forum/s...
Basically, I can summarise the argument as: for complex, real-world projects involving multidimensional arrays, having to remember which dimension means which thing, and track these through various tensor transformations, is the 21st century equivalent of how prior to high level programming languages we had to manually track which registers we we were using to store what quantities, and how this changed in different parts of a program.
It was the real dark ages of programming, and we’re in that same place with a lot of complex deep learning models now.
So my question is: has this been discussed at all in the Julia community? Is this on anyones radar? Since one of the core competencies of Julia is array processing and elegant metaprogramming, I would hope that it can gracefully absorb the idea of named array axes in a way that feels natural and complete, and could start showing up in ML frameworks like Zygote and Flux down the line...
Hasselt argues this is because if you need accurate updates to your Q values, you need to trust the learned model you are using for simulated rollouts on the states on which you are sampling. But if your simulation model is trustworthy on these states, it is because it saw a lot of real transitions from these states from the actual environment. But then you might as well just have stored those transitions in a big enough replay buffer and use ordinary Q-learning with experience replay. And this indeed seems to be the case: when you give Rainbow DQN a nice big replay buffer, it is more sample efficient (both real and imagined samples) than SimPLe. Hasselt leaves some wiggle room for learned models to help with action selection and credit assignment, though.
My counterargument to this (supplied with zero evidence of course!) would be that with the right inductive biases, a learned model can generalize quite accurately and with very few seen transitions, and hence be so sample efficient that it would outperform the replay memory approach. I'd imagine that the kinds of inductive biases that are appropriate for a varied meta-environment like Atari are quite general things like 'visually localized objects typically only interact when they approach or touch each other', and 'the arrow keys likely control one localized object'. There are approaches for how to encode such priors; [2] is a good survey paper, and [3] employs some of these ideas for RL. Moreover, these are the kinds of priors that one imagines are encoded or biased towards by evolution in actual animal brains.
[1] https://arxiv.org/abs/1906.05243
To answer my own question, yes: http://papers.nips.cc/paper/7112-scalable-trust-region-metho...
I’ll add some things that relate to this perspective: there is a follow up paper to the DL via Hessian-free optimization paper by James Martens that develops a variant of AD which calculates a special curvature quantity which is useful for efficient second order optimization. This emphasizes that AD is just one of a family of program transformations, most of which are probably waiting to be discovered. For example, would it be useful to have an AD-like pass that calculated trusted regions for gradient updates? And: why do we not do more iteration during DL in which we interleave forward and backward for a single layer only? (rhetorical question, there are many reasons but asking this question forces you to think deeply about a lot of things)
Also, if you generalize VAEs to be arbitrary differentiable programs you immediately run into versions of message passing. For example, vector quantized VAEs are a precisely a soft version of the EM step in k-means. So how would we develop more advanced variational programs in which this process is better managed? There may be very different forms for the inference and generation phases that are not related by a program transformation, or ARE related but only in a high-level way best captured by functional programming primitives.
Any other people nerding out about this stuff please say hi!
Either way, can you recommend a good survey article on ILP and the last few years of progress on that front?
But the neuro-symbolic concept learning paper I gave already shows the potential translucency of these kinds of hybrid systems: the linguistic interface (its a VQA task) allows one to simply look up the feature vectors and programs associated with particular phrases or English nouns. Similarly, the scene is parsed in an explicitly interpretable way, with bounding boxes for the various objects on which reasoning will commence. This 'bridge' theme between natural language and the underlying task space is really powerful, and it probably makes sense to figure out how we include them for systems that have nothing to do with natural language.
https://arxiv.org/abs/1901.11390 contains another great example of how interpretable such models can be, especially if they are generative. Take a look at those segmentations!
Lastly, https://arxiv.org/abs/1604.00289 lays out this vision in a lot more detail.
Of course we need to keep the gradients, architecture search, hyperparameter tuning, scalable training to massive datasets, etc., but there is a growing sense that the programs we write can encode extremely powerful priors about how the world works, and to not encode those priors leaves our learning algorithms subject to attacks, bugs, poor sample efficiency, bad generalization, weak transfer. Not to mention a host of rickety conclusions that are probably poisoned by hyperparameter hacking.
Conversely, we need to try to avoid the proliferation of black box systems that require heroic efforts of mathematical analysis to understand and debug. Take for example the highly sophisticated activation atlas work by Shan Carter and others, which was needed to reveal that many convnets are vulnerable to an almost childlike kind of juxtapositional reasoning (snorkler + fire engine = scuba diver). Beautiful work, but to me it would be better if that form of analysis wasn't necessary in the first place, because the nets themselves were incapable of reasoning about object identity using distant context.
We need systems that are, by design, amenable to rigorous and lucid scientific analysis, that are debuggable, that admit simple causal models of their behavior, that are provably safe in various contexts, that can be straightforwardly interrogated to explain their poor or good performance, that suggest modification and elaboration and improvement other than adding more neurons. We need to speed the maturation of modern deep learning out of the alchemical phase into something more like aeronautical engineering.
The major innovations in recent years have been along these lines, of course. Attention is a great example, basically supplanting RNNs for a lot of sequence modelling. Convolutions themselves are probably the ur-example, of course. Graph convolutions will be the next major tool to be pushed into wider use. To the interested observer the stream of innovations seems not to end. But the framing that makes this all very natural is precisely that of this being the union of computer programming, where coming up with new algorithms for bespoke tasks is commonplace, with automatic differentiation, which allows those algorithm to learn.
What remains exciting virgin territory is how we best we put these new beasts into the harness of reliable AI engineering. That is in its infancy, because how you write and debug a learning program is completely different to the ordinary sort... there are probably 10x and 100x productivity gains to be realized there from relatively simple ideas.