iraphael

I know one of the biggest struggles of training RNNs in TensorFlow, for instance, is the fact that the graph is static. Other frameworks, like PyTorch, have dynamic graphs.

Will Caffe2 support dynamic graphs (perhaps down the line)? What is your reasoning behind whichever decision?

What happens after the O1 expires? It's a non-immigrant visa right? I'm assuming it doesn't mean you get a green card after 3 years, correct?

It seems like a student project, not groundbreaking research.

Have you looked into ROS and Gazebo?

I am an undergraduate leading an open source autonomous vehicle project [0] and we're currently almost done setting up simulation testing. Gazebo and ROS are very poorly documented but they serve their purpose really well when it comes to creating pub sub and physics simulation for robotics systems.

[0] https://github.com/gtagency/buzzmobile

Link to the actual paper: https://arxiv.org/pdf/1703.01253.pdf

If you're interested in a more introductory talk, I gave one a few weeks ago that goes over the basics of Deep Learning, and how TensorFlow works internally.

https://www.youtube.com/watch?v=DYlHnxfrrZY

that's for use of a recording in court, not sure it applies to this situation

Hey, I'm the current team lead for buzzmobile. I'm glad our repo is useful, we've been working hard to make it as clean and maintainable as possible. Definitely feel free to reach out or send feedback if you have any :)

Out of curiosity, is this a result of your employer having flexible working hours and you being able to get the work done in that time, or are you also being paid less? (or are you in freelance, etc?)

Github link: https://github.com/tensorflow/fold

Paper link: https://openreview.net/pdf?id=ryrGawqex

Well for starters, fine-tuning can be done in a variety of different ways. You can pretrain your model with a larger, different dataset, or you can train an autoencoder that learns some useful representation of that larger dataset and use the encoder as a base for fine tuning.

Another approach I've seen that was really cool is Model Distillation [0], which is basically the training of a smaller NN with the inputs and outputs of a larger NN (where the output is slightly modified to increase gradients and make training faster).

[0] https://arxiv.org/pdf/1503.02531v1.pdf

Which is why I'm left wondering, why is it that "Silicon Valley Is So Nervous About H-1B Reform". Unless the headline is simply clickbait, I don't think the question was answered...

It should be noted that transfer learning is an umbrella term for many ideas that revolve around transferring what one model has learnt into another model. The method described here is a type of transfer learning called fine tuning.

that was it! thanks!

same issue on chrome, so not a firefox thing. I think we're missing something

This seems to be the important bit, that describes what makes their learning "fast":

"The objective (...) is to maximize the expected total discounted reward accumulated during a single trial rather than a single episode. Maximizing this objective is equivalent to minimizing the cumulative pseudo-regret (Bubeck & Cesa-Bianchi, 2012). Since the underlying MDP changes across trials, as long as different strategies are required for different MDPs, the agent must act differently according to its belief over which MDP it is currently in. Hence, the agent is forced to integrate all the information it has received, including past actions, rewards, and termination flags, and adapt its strategy continually. Hence, we have set up an end-to-end optimization process, where the agent is encouraged to learn a “fast” reinforcement learning algorithm"

However, one would note that this is still bounded by learning of the RNN, so I don't really see how this approach makes the algorithm much faster than "slow" RL, as loads of trials would still be required for any real learning. Maybe someone more knowledgeable could pitch in.

An interesting bit is that this is coauthored by Kristen Stewart. But it talks about a lot of interesting things on how to tweak and evaluate Neural Style in a production environment where the results need to serve the story (rather than a loss function).

This is the interesting part:

> New dataset is smaller in size and similar in content compared to original dataset: If the data is small, it is not a good idea to fine-tune the DCNN due to overfitting concerns. Since the data is similar to the original data, we expect higher-level features in the DCNN to be relevant to this dataset as well. Hence, the best idea might be to train a linear classifier on the CNN-features.

> New dataset is relatively large in size and similar in content compared to the original dataset: Since we have more data, we can have more confidence that we would not over fit if we were to try to fine-tune through the full network.

> New dataset is smaller in size but very different in content compared to the original dataset: Since the data is small, it is likely best to only train a linear classifier. Since the dataset is very different, it might not be best to train the classifier from the top of the network, which contains more dataset-specific features. Instead, it might work better to train a classifier from activations somewhere earlier in the network.

> New dataset is relatively large in size and very different in content compared to the original dataset: Since the dataset is very large, we may expect that we can afford to train a DCNN from scratch. However, in practice it is very often still beneficial to initialize with weights from a pre-trained model. In this case, we would have enough data and confidence to fine-tune through the entire network.

Besides a cool new application of GANNs, I don't see if this architecture is much different than normal GANNs. Anyone else have thoughts?

> Available first on HTC Vive

I can't find if it'll have support for DayDream View, which I just got for free with my pixel. Does anyone have more info?

For an interesting understanding of the state of copyright today, I recommend reading Free Culture, by Lawrence Lessig.

I bought the kindle version but it's also available online at http://www.free-culture.cc/freeculture.pdf

So is material going flatter? Isn't that partly abandoning the paper-like material metaphor?

Blog post from a while ago, with nice visualizations, but heavy browser computation:

http://www.r2d3.us/visual-intro-to-machine-learning-part-1/

> I encourage our greatest minds to build the tools that will computationally create almost all possible scenarios a driverless vehicle can encounter

That's not as simple as the writer seems to think it is. Autonomous vehicles can only interface with the world via sensors, and sensor data that looks real, but is actually computationally generated, is one of the best ways to make a model overfit. It can learn the exact function and simplifications a developer has made in order to generate the fake data.

Then you can say "alright, record the data and modify it". But you can't actually modify, for instance, point-cloud data that much to account for arbitrary viewpoints. What you can do, and what most companies do afaik, is record the data as a "unit test". The car then responds to the test and, given the result actuation, it is evaluated whether the test passed (e.g.: you want to drive into the sidewalk? test failed). So if the car wants to do a correct maneuver in the test (i.e., one that passes the evaluation), but it is different than the maneuver the test has recorded, then you suddenly don't have the data to assess how well the car did. I know there are ways companies use to solve this but I assume it's not generating data, given the problem presented above.

tldr: generated data isnt as straight forward and can lead to bigger problems later on.

> In theory, both of the accumulators are unbounded, but in practice, we noticed their values remain quite small

This empirical fact is used to their advantage, but it is not proven that this is generally the case (tbh, it might be nearly impossible to prove it). This is okay, but it makes the model seem kinda yucky to me.

> All of the models use 1024 LSTM nodes per encoder and decoder layers.

Ah, as I was reading the paper I kept looking for how they generated a variable-sized attention vector using a simple feed-forward network. It seems that the LSTM is actually capped at 1024 nodes per layer, so the attention vector only can be a fixed size of 1024 and truncated to the number of unrolled steps.

These seem to be the papers for DeepMask [1] and SharpMask [2]

[1] https://arxiv.org/abs/1506.06204

[2] https://arxiv.org/abs/1603.08695

I'm reading up on Cellular Neural Networks and it seems like they are simple convolutions, of which you can specify the kernel. In fact, this library is just calling scipy.signal.convolve2d() with different kernels [2].

The research (from the original 1988 paper [1]) apparently tried tackling the signal processing problem from a different angle (namely, having connected neighboring 'cells'), but the end result is the same as what we know today. I actually wouldn't be surprised if CNNs were a precursor to today's ConvNets in some way.

TLDR: these 'CNN's are convolutions. They aren't ConvNets because they don't have pooling or fully connected layers.

[1] http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7600 [2] https://github.com/ankitaggarwal011/PyCNN/blob/master/cnnimg...

which brings up the other problem of why aren't there backups

And this is one that explains everything OP needs to understand the confusing sentence from their intro:

http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip...

I'm interested in the consequences of, instead of "everyone" carrying out SENPAI, having a more efficient STB Matching System (Spin The Bottle).

Sure, it could backfire: interested parties may fall victims of the universe's Random Number Generator, but it makes for some interesting Game Theory dynamics, which I call the Tinder Card Stack Dilemma.

Any one party could be more willing to express interest in any of the other parties (assigned by STB), if they believe there is a low chance of them being matched with someone else they are more interested in.

On the other hand, they could become more picky, and only express interest in parties they are very interested in, rejecting others because they believe there is a high chance of them being matched with greater interests.

There is a definite trade off between efficiency and overall population happiness when choosing between STB and the solution described in the paper. Because STB is more efficient, it ends up being way more scale-able. But bigger populations greatly intensify the TCSD.