fchollet

Two repos to get started building reinforcement learning agents with Gym and Keras:

https://github.com/sherjilozair/dqn

https://github.com/osh/kerlym

They place an emphasis on Github star count, but that is a terrible usage metric. Leaf got a lot of stars because it got a lot of HN exposure, but at the same time Leaf had no users. Assumedly that's the real reason why they are stopping development.

Here are the metrics that matter.

Over the lifetime of the project:

- 14 contributors

- 40 issues opened

- 161 forks

In the past month:

- 7 issues opened

- 6 PRs opened

In the past week:

- 0 issues opened

- 0 PRs opened

In other words, a ghost town. The lesson here is that a lot of HN exposure does not automatically convert into a lot of users.

I think (or hope) that you forgot the "/s" at the end.

If anyone wants to switch to TensorFlow but misses the Torch interface, you will always have Keras: https://github.com/fchollet/keras

> If SF's city officials can pull it off, with the increased investments in public transportation eventually San Francisco is going to become a global A+ city.

This is the most hilarious comment I've read on HN in a long time.

The functional API removes the need for a dedicated autoencoder layer. But maybe in the future we will have a dedicated autoencoder model, if there's interest in that (with autoencoder-specific methods).

Layers have always had an activation argument, this is not new. And yes, there is still an Activation layer. You can specify an activation function via either option.

Yes, I think we should work on a FAQ to introduce common ML concepts and their implementation in Keras. Any specific concept that you had trouble with?

A few advantages:

- it's much easier to use. Using pure TensorFlow is considered "advanced" and requires familiarity with deep learning, understanding of what a symbolic computation graph is, etc. Keras, meanwhile, is meant to make deep learning more accessible.

- even if you don't care about accessibility, Keras provides higher-level building blocks that speed up your workflow even if you are an expert. It is currently used by dozens of companies and hundreds of researchers, precisely for this reason: it allows quick prototyping.

- with Keras, you can work with both Theano and TensorFlow interchangeably. They complement each nicely in a workflow: TensorFlow has low compilation times, which is great for debugging, and Theano tends to be faster for runtime (especially for RNNs). So you can prototype in TF, train in Theano, then to switch to production you can export the TF model.

I agree that sklearn-compatibility is the strong point of Skflow. If you are familiar with Keras, note that you can do the same with any Keras model, via the sklearn wrapper: https://github.com/fchollet/keras/blob/master/keras/wrappers...

> I don't think the SKFlow author(s) are at Google. Prettytensor's are, but it isn't Google supported.

I believe they are. Also I do believe that PrettyTensor is an internal Google project.

It's really neat stuff. The 3rd example is especially stunning.

For more projects like this, you can also check out the neural style transfer implementation in Keras: https://github.com/fchollet/keras/blob/master/examples/neura...

This is the script that OP's project was adapted from.

So Periscope and Moments would be the saving grace of Twitter? On what planet does the author live?

It's served from Microsoft Bob Server ®

> Is it common practice to use the most frequent words as features ? It looks like they don't carry much information, by definition.

The common practice with a small-ish dataset is to use e.g. the top 10k or 20k most frequent words, but filter out the top 50-100 so most frequent words, as those indeed do not carry much information. A commonly used weighting scheme is TF-IDF (https://en.wikipedia.org/wiki/Tf%E2%80%93idf), which comes included in Keras.

Anyway, this is a cool ML starter project. Keras makes it really easy to do this sort of fast experimentation with a range of different neural networks models.

Confusing, still pretty buggy. Maybe it's a bit early to release?

I'll try to compile some constructive feedback in a bit.

These two things are non-trivial, but not particularly hard in themselves. However, doing them at ultra-low latency becomes quite a challenge. Doing anything at ultra-low latency is already a challenge, but especially so when what you're trying to do is running a deep neural net for entity recognition or gesture recognition.

Excellent news. Hopefully with this release we will be able to lift the remaining limitations of the TensorFlow version of Keras (tensor contraction, float<->bool casting, and RNNs over sequences with arbitrary length). https://github.com/fchollet/keras/wiki/Keras,-now-running-on...

Congrats to the TensorFlow team!

> Maybe they should make a law or something that 1 out of every 100 news must be fake

In the world I live in, the proportion is far higher. Especially if you are talking about the mainstream media. Hell, easily one of 100 peer-reviewed scientific papers is fake.

While it would be much more energy-efficient, such a process is bound to have a very low rate of weeds destroyed / minute.

Here's the MS COCO leaderboard: http://mscoco.org/dataset/#captions-leaderboard

Google's Show and Tell seems considerably superior to competing approaches.

I suspect this would result in rather superficial weed destruction, since the roots underground would be unharmed and would cause the weed to regrow a few days later. Unrooting the weeds definitely sounds more reliable to me.

Let the laws of physics do the recurrent math for you. Analog RNN computers would be very interesting, but they would first setting in stone the basics of the algorithms we use. We are still only beginning to explore the algorithm space, and this requires a flexibility that analog computers (or even ASICs or FPGAs) don't provide.

It's still not clear whether the future of AI will even involve neural networks at all. Intuitively, they seem so inefficient.

YouTube videos?

When people don't have to pick their housing based on the location of low-wage jobs (dense cities), I would expect the rent to go down due to better utilization of the total housing supply.

Here's a relevant link: http://www.researchgate.net/post/Is_deep_learning_with_decis...

Two examples:

- using layers of random forests (trained successively rather than end-to-end). Random forests are commonly used for feature engineering in a stack of learners.

- unsupervised deep learning with modular-hierarchical matrix factorization, over matrices of mutual information of the variables in the previous layers (something I've personally worked on; I'd be happy to share more details if you're interested).

I have, actually. Here's the code for the experiment, with a link to download the data: https://github.com/fchollet/keras/blob/master/examples/skipg...

I also recommend using Gensim for word embeddings.

> the technique is fairly new and used in deep learning, popularized by some of the guys that popularized deep learning, usually mentioned in conversations about deep learning

Logistic regression with regularization is fairly new? 'Pioneered' by the same people as deep convolutional neural networks? Are you certain about this?

How? There is no recurrent data flow in word2vec. Word2vec maps words and their context with 2 embeddings, a dot product and a sigmoid. That's it.

The "deep" in deep learning refers to hierarchical layers of representations (to note: you can do "deep learning" without neural networks).

Word embeddings using skipgram or CBOW are a shallow method (single-layer representation). Remarkably, in order to stay interpretable, word embeddings have to be shallow. If you distributed the predictive task (eg. skip-gram) over several layers, the resulting geometric spaces would be much less interpretable.

So: this is not deep learning, and this not being deep learning is in fact the core feature.

Keras http://keras.io/ is a neural networks library for Python that is rather easy to get started with compared to the alternatives.

I also recommend the tutorials on http://deeplearning.net/tutorial/ (Python / Theano)

And Andrej Karpathy's blog is also a great resource for explanations of deep learning concepts in simple terms: https://karpathy.github.io/