therajiv

I've tried a few online jeopardy simulators before, but have always just ended up going back to hunching over a laptop together with my friends on j-archive.

This looks much more convincing than ones I've seen before, especially the fact that you can just play prior game boards without any fuss. I'll have to give it a try with my friends, but so far gameplay feels smooth and the jeopardy-like accents are a good touch.

Are applications like batteries, semiconductors, solar panels, etc. bottlenecked by the number of available materials? Also, I wonder if the discovered materials are kind of "interpolating" between materials that are already known, or if they expand the convex hull in some way. (Though perhaps it's difficult to precisely define what the convex hull of materials is.)

Seems like this could have dramatic effects on some professions where 60+ hour workweeks are typical even outside of Korea, e.g. being an early startup employee or adhering to the strenuous pro gamer practice schedule. While it's probably not healthy to work that much, in the short-term, throwing more time at a job depending on the profession can sometimes optimize your reward function.

NYT and Nature News.

I ran into this guy randomly at MIT once, during the offseason (last February). He was such a chill guy. Hearing this news makes me like him even more - he's not letting one of his passions get in the way of another.

TLDR is that models will become more abstract (current pattern recognition will blend with formal reasoning and abstraction), modular (think transfer learning, but taken to its extreme - every trained model's learned representations should be applicable to other tasks), and automated (ML experts will spend less time in the repetitive training/optimization cycle, instead focusing more on how models apply to their specific domain).

You provide great examples, thanks. I guess I was just hoping that the article would spell out those situations as clearly as you did.

Sorry if I came off that way, that's not at all what I meant. My comment was more along the lines of putting a face to statistics; reading about the numbers of people dying is a weaker motivator than hearing the stories behind the lives that are lost to cancer.

It's not clear to me how malicious actors can manipulate this observation to confuse self-driving cars. That said, I don't think this discredits the point of the article; it's important to note how easily deep learning models can be fooled if you understand the math behind them. I just think the example of tricking self-driving cars is difficult to relate with / understand.

I knew Maryam was incredible the first time I heard her give a talk at Stanford. To my fellow cancer researchers out there: this is about as good motivation as we'll ever have to keep plugging along.

This is exactly my point - the danger of "anthropomorphization" lies in taking the brain analogy too far. That is, there shouldn't necessarily be a link between research in neuroscience and advances that make deep learning models more accurate. The tasks are completely different (human learning vs. minimizing a loss function), and it's important for researchers in both fields - neuroscience and AI - to keep that in mind.

As someone primarily interested in interpretation of deep models, I strongly resonate with this warning against anthropomorphization of neural networks. Deep learning isn't special; deep models tend to be more accurate than other methods, but fundamentally they aren't much closer to working like the human brain than e.g. gradient boosting models.

I think a lot of the issue stems from layman explanations of neural networks. Pretty much every time DL is covered by media, there has to be some contrived comparison to human brains; these descriptions frequently extend to DL tutorials as well. It's important for that idea to be dispelled when people actually start applying deep models. The model's intuition doesn't work like a human's, and that can often lead to unsatisfying conclusions (e.g. the panda --> gibbon example that Francois presents).

Unrelatedly, if people were more cautious about anthropomorphization, we'd probably have to deal a lot less with the irresponsible AI fearmongering that seems to dominate public opinion of the field. (I'm not trying to undermine the danger of AI models here, I just take issue with how most of the populace views the field.)

Well, the data wasn't showing that the past few years were anomalous; rather, there were fewer high-magnitude earthquakes than expected. I don't think this has anything to do with being overdue for an earthquake. Most likely this is just because with events of low frequency (e.g. these higher-magnitude earthquakes were predicted to occur once every ~100 years by the linear model), large percent deviations from the expected value are more probable. Basically if you flip a coin 10 times you might imagine that 3 heads and 7 tails is pretty common, whereas 300 heads and 700 tails on 1000 tosses is comparitively extremely unlikely.

Wow, the discussion on the Fukushima civil engineering decision was pretty interesting. However, I find it surprising that the engineers simply overlooked the linearity of the law and used a nonlinear model. I wonder if there were any economic / other incentives at play, and the model shown was just used to justify the decision?

Regardless, that post was a great read.

I'm surprised this works well. If the method is a state-of-the-art convolutional neural network architecture, you generally would need more than 10 images -- even with transfer learning -- for passable accuracy. Algorithms for medical diagnosis, for example, generally require between 100 and 200 images to do well. Though those are generally transfer learned from ImageNet CNNs. So I'm curious as to which dataset this face recognition uses for weight initialization, or it uses another ML method entirely.

Wow - literally just saw an article about RescueTime on the frontpage of NYT (https://www.nytimes.com/2017/07/05/your-money/where-does-the...). And installed it 5 minutes ago. Coincidence, if I've ever seen one... RIP.

I'll agree with you that it's much harder than it should be (thankfully, finding the implementations is the hard part, not using them), but yes, these methods do exist.

DeepLIFT (the method I linked in my original comment: https://github.com/kundajelab/deeplift), takes a Keras model (with Theano or TensorFlow backend) as input and provides feature importance scores for any desired layer of the network (raw data inputs, inputs to dense layers following convolution, etc.). Keras-Vis (https://github.com/raghakot/keras-vis) is another nice package that allows for easy visualization of saliency maps and convolutional filters. Perturbing inputs and looking at the effect on the output of the network is another technique people use pretty often.

I think there's a lot of room for this space to become easier to use, especially for newer deep learning practitioners. To that point, I definitely agree with the author of this blog post.

The author discusses how linear models are generally more interpretable than deep learning methods, but I'd argue that's actually changing pretty quickly. Especially for large image/sequence inputs (which covers most of the applications that are getting hyped up), linear regressions don't perform very well, and often that performance difference prevents them from picking out important features. Given that fast, scalable methods for feature importance are on the rise (e.g. https://arxiv.org/abs/1704.02685, which the author mentions), you often get equally interpretable feature scores from deep models that are more accurate than analogous ones from linear models.

Basically, my point is that model interpretation strongly depends on how accurate your model is, and because deep learning models are so much better than linear models for some tasks, it makes sense to use them - even if your primary goal is interpretability.

That said, I do believe that if you ever care at all about interpretation, you should almost never be using multilayer perceptrons (which have recently become part of the widening umbrella term "deep learning"), because they rarely work better than decision tree models or basic linear models (and MLPs are generally less or equally as interpretable when compared to traditional methods).