Hacker News Reader

Comment by Findus23 | Hacker News Reader

Hi, I have already expected a question about Neural Networks :)

When I had the idea I also toyed around with word-rnn and similar RNN libraries. The results I got were pretty good, but training was extremely resource consuming. Cuda gave a 8x boost, but still training one of the smaller sites took 20 minutes on my simple graphics card while creating the Markov chain is done in 2 minutes.

I also have absolutely no experience with Machine Learning and just the setup was already quite an experience. So I stuck with what I know and stayed "the traditional way".

View on HN

Interesting. Yes, training takes way more time. In some really big projects like https://blog.openai.com/unsupervised-sentiment-neuron/ "We first trained a multiplicative LSTM with 4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text. Training took one month across four NVIDIA Pascal GPUs, with our model processing 12,500 characters per second."

For smaller ones (and smaller datasets) a few hours of GPU is considered fast (at ~1$/h it is not much!). If you want to use it online, there is https://neptune.ml/ (no setup, $5 free credit for computing; full disclosure: created my colleagues).

In any case, I would be excited to see it on some site with code (like SO) or formulae (math or stats). Especially as I am a big fan of StackExchange and analysis (vide http://p.migdal.pl/tagoverflow/ :)).

Sounds nice, but my plan was to find out how much data I can handle easily on my plain simple desktop PC.

You may have seen, that you can filter by site [1]. Code quite ruined my chains as it didn't appear in blocks, but rather everywhere, so I went the easy way and filtered all code blocks.

I didn't filter math as I couldn't find a proper way to do it, but you can see that it gets quite messy [2]

[1] https://se-simulator.lw1.at/s/math.stackexchange.com

[2] https://se-simulator.lw1.at/q/a-e-3-how-to-span

Well, then taking a subset of math and leaving it overnight will work. (No need to do run it on ALL sites).

Reply on news.ycombinator.com