How to Code and Understand DeepMind's Neural Stack Machine in Python
iamtrask.github.io
iamtrask.github.io
The question is, how do you decide which papers to promote to further attention? It's always a ranking problem, because there's always other work you could be looking at. You start your reading method at the point at which you've already decided to read the paper. But how do you get to that point?
(I can describe how I decide, but it's not exactly easy to replicate. It's a pretty insidery perspective. Here's how it played out for this paper.
I looked at this paper soon after it was published because I follow the first author on Twitter. When I opened the paper I looked at who was on it. I know Phil, and I know the sort of work Ed has been doing. I glanced over the paper and looked at the experiments, and saw that it was similar to the neural turing machine and algorithm learning line of work, that's still being evaluated on toy problems. This work is interesting but I'm not doing research on these things, so I won't commit to learning it while it's moved <1 mountain empirically. I did read the idea enough to contrast it with Chris Dyer's stack LSTM parser, which is a model that performs very well empirically. I checked the related work for criticism/comment on that work, and the paper said it's inspired by it. Cool. I'll watch for future empirical evaluations.
In total I probably spent about 10-15 minutes looking at this paper. This is enough for me to remember I'd seen the work, and help me understand future related ideas a little bit better.)
Stack machines are really cool. Are they computationally efficient though? As the stacks grow bigger the number of possible stacks to keep track of grows exponentially doesn't it? Or do they only keep track of some of them?
We could do superoptimization with that. For example, a superoptimized sort.