What would be your recommendation to learn about these topics ? Moocs, books, having some pet project ?
318 karma · joined February 6, 2013
I hope this can be made to work reliably. I generally prefer web pages to pdfs.
Optimisation in program space would be trying to find both the structure (connections and activation functions) and weights of the NN, which is not what we do currently. We tend to hand/engineer (or keep what works best empirically) the architecture (1), then train by finding the best weights.
I am very interested in approaches to efficiently opmitise in program space, and DL/backprop doesn't feel like it
(1) although this is starting to change.
Google Brain/DeepMind are also pushing some of those ideas. They must be, since they aggressively poach all the top researchers from those labs...
Ng approach is different: he wants a world powered by Deep Learning, so his goal is to make applied deep learning thrive. His strategy to do that: give those data-hungry models even more data, which is completely reasonable.
Those two approaches - fundamental research and applied deep learning - are often referred to as AI, causing much confusion.
My understanding: you get nothing tangible from the grant ("The money is a gift"), no contracts or IP... To me it seems like a bet that the networks effects and the gratefulness of the grantees will pay back somehow. I think this is a smart move because investing in smart individuals is itself smart, although the devil is in the details.
Is it surprising, then, that many of the most accomplished individuals still have the imposter syndrome ? It is a cognitive tradeoff: the price you pay for the ability to never stop improving is feeling a little weird about yourself.
That reminds me of this: https://waitbutwhy.com/2015/11/the-cook-and-the-chef-musks-s... Especially this diagram: https://28oa9i1t08037ue3m1l0i861-wpengine.netdna-ssl.com/wp-...
I have a really hard time keeping reading after this passage.
Some of my favourite videos:
- Neural Representations of Language Meaning - Tom Mitchell https://www.youtube.com/watch?v=pRBf8BWAG3k
- Computational cognitive science - Josh Tenenbaum (co-author of probmods) https://www.youtube.com/watch?v=2WQO9e5Mdj4
For example in this review: https://arxiv.org/pdf/1605.01138.pdf, a model-based approach is compared against a deep learning approach. It is very interesting to me that even if deep learning performs better, its failure cases are very different than human failure cases (deep learning is not sensitive to "physics illusions"). Whereas the model-based approach is very much in-line with human performance characteristics. Also, on this task (physical scene understanding), deep learning generalises very poorly.
> Recent progress in artificial intelligence (AI) has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways.
> We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn, and how they learn it. Specifically, we argue that these machines should (a) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; [...]
https://arxiv.org/pdf/1604.00289.pdf
Some others papers related to these ideas:
- Learning a Theory of Causality http://stanford.edu/~ngoodman/papers/LTBC_psychreview_final....
- Theory-Based Causal Induction: http://cocosci.berkeley.edu/tom/papers/tbci.pdf -
Very easy to get going and very hands-on.
This is the branch of IA/ML that turns me on the most. When we get there, it will redefine many jobs (as well as create an existential thread as a side effect).
I think it is sad that the terms AI and Machine learning do not make a clear difference between narrow and strong AI, this is the source of much confusion.
Random idea: what about instead of the "search by algolia" text, you replace the magnifying glass icon inside the search bar with an algolia-themed magnifying glass. For example you take the 'o' of algolia that looks like a clock and add the handlebar in the bottom right. And that's your new magnifying glass icon. And make this new icon the default integration.
This way, people would come to associate the fast-search with that logo, without having a "by algolia" text cluttering the UI.