Neural Networks Demystified | Hacker News Reader

Neural Networks Demystified

lumiverse.io

686 pointsmaxlambert63 comments

I am very interested in this subject but I was unable to finish watching these videos. The background music is incredibly distracting. I have a name for that kind of music, I call it Silicon Valley Music because it is the kind of music used in a lot of startup product videos. The narrator voice style is also pretty much the same that is used in those. I wanted to like these videos but I did not feel that they had any value what so ever. With all of these kinds of videos, I feel like the producers want the audience to feel bliss or learning but that they never actually deliver on that so instead it seems insincere and underhanded.

These are great videos, and the content itself is, in my opinion, pretty good.

It seems to me these have been narrated by a professional. The narrator's voice is pleasant and pronunciation clear, but it seems he doesn't really know what he's talking about.

I would prefer the voice of an actual subject matter expert; they would be able to better convey which parts are actually key concepts, and which parts are just intermediary math. Now it feels like everything is equally important, when it's clearly just some intermediary math to get you to the actually interesting part that follows.

I agree that the continuous music is distracting. The quick fix is to fade the music away for the content. It would be fine to leave in for the beginning, end, and transitions.

I'm confused by your comment. You seem to be saying two different things:

1) The music and choice of narrator is getting in the way of the (presumably valuable) content.

2) The content itself, regardless of the choice of narrator or whether or not there's music behind it, is not useful.

If it's the latter then why complain about the former?

People feel discomfort while listening to these videos because they feel like they are being persuaded by a weak young man to believe things he himself does not believe. This is caused by:

1. High pitch voice. Either it is modulated or not, the voice informs us that a person we are listening to is low social status, therefore subconsciously we assign lesser significance to anything he says.

2. Infomercial-like intonation. Guy may be professional narrator, and his intonation is like you would find in advertisements where someone tries to sell you something really worthless. So people again subconsciously tend to "categorize" this kind of information as unwanted and are used to filtering it out. The effect is strengthened by the choice of music.

3. The narrator clearly has no understanding (or gives that impression) about the subject he is told to talk about. This is subconsciously felt by people through his intonation and the way to put emphasis on random words, and that also translates to two things: again reminds us of infomercials (meaningless talking to occupy time) and malicious persuasion - he wants me to believe in something he does not believe (actually does not understand).

This all adds up to our brain signaling something is wrong, therefore complaints. There is more to education than content.

A lot of Kickstarter campaigns are using that kind of music too.

yes music needs to part ways from otherwise interesting video

Hey, everyone! I'm the founder of lumiverse.io, it's pretty incredible to see our website on the front page of HN!

I want lumiverse to become an awesome community where people can discover and discuss great educational videos.

We've launched only recently, the site is still in active development, I'm improving it every day. If you have any feedback - please let me know =)

(Also feel free to contact me at raymestalez@gmail.com)

The site could benefit from a quick explanation of what it is. I watched the neural networks videos and got curious what the deal was with lumiverse generally, but found no quick answer to that question. You don't have an About Us page (that I could easily find), for example. Edit: Even a tagline next to your logo might do the trick.

Thank you for the feedback! You're right, I definitely should do that, now adding an about page is next on my todo list)

nice work, i am interested in this kind of information, but i am no big fan of the edutainment video format, i prefer text or a university lecture. some feedback about the first video:

- why the music? should this be information or an awfully mixed guitar rap song with the worst flow ever? i can't stand the use of elevator music in these kind of videos and also regard this kind of video production as an act of disrespect towards music in general.

- why not use more descriptive variable names, e.g. hours and score instead of x and y? the mystification of these kind of things comes partially from generalised abstraction and undescriptive variable names. since you already use a real world example, why not reflect it with the variable names?

- less speed, more pauses in general do good for demystification of such a topic, the typical 10-second-attention-span youtube-edutainment-video-style might not really fit here

- the drawings and the general flow are nice and well done

This website curates content, so you would have to go to the content creator with this sort of input (http://www.welchlabs.com/).

I think this is just a matter of learning styles. This speaks to one kind of learner, maybe not others. I personally learn by doing - no lecture, no matter how slow or fast is going to "teach" me - it may lay groundwork, but I learn in the (virtual) trenches.

Personally, I liked the music. Maybe the music could be optional. Might be a cool feature to add, if there's time.

Feedback: 1) the videos go a tiny bit too fast in certain areas, but one can pause and rewind so that's not a problem 2) the background music is too loud/strong which makes it too distracting, I have to focus more than necessary on the spoken words; I'd cut the volume of the background music in half. Otherwise, pretty awesome videos :)

The music has to go. As a human, I can't focus with ambient noise. The rest is spot on. Keep up the great work.

Thanks for the videos, they really clarified a lot of these concepts for me and were easy to follow in spite of having not done much calculus in the past few years.

One suggestion: What if you embedded the ipython notebooks that go along with each video into the page so that people could follow along? Even if it is static html like you see here http://blog.fperez.org/2012/09/blogging-with-ipython-noteboo... and not an actual hosted notebook.

I'm glad they were useful! The author of these videos is Stephen Welch(http://www.welchlabs.com/), he makes really fantastic tutorials.

Embedding notebooks is an interesting idea, I'll experiment with implementing it.

Nice work!

It would be cool if there were pointers to recommended videos/courses from other organizations (Stanford, MIT, etcetera) for further learning.

PS: I would have wished my college classes had such relaxing background music :)

Well, i got to the frontpage of lumiverse and was catched by quite some intresting looking videos. Good job there. But i have to say, the video player could use some improvements. I switched to youtube when i could because the video player resets its volume after each video and it is kind of hard to use.

Oh, thank you for letting me know, I will work on that.

Really happy I came across this, been searching for a website like this for a long time. Thanks!

Every beginner in Neural networks should probablystart with this and follow with Karpathy's http://karpathy.github.io/neuralnets and may be later on http://neuralnetworksanddeeplearning.com/

Also http://colah.github.com/

I didn't see this, thanx!!

These are also available on Stephen Welch's youtube channel[1]. He uploaded the last one in this series Janaury 2015.

[1]: https://www.youtube.com/playlist?list=PLiaHhY2iBX9hdHaRr6b7X...

Per guidelines : https://news.ycombinator.com/newsguidelines.html

  Please submit the original source. If a post reports on 
  something found on another site, submit the latter.

The youtube URL you cited would probably be a better source link.

Yes, it was misleading I thought all these videos were original content of that site; in fact, they are original content produced by Welch Labs: http://www.welchlabs.com/

My initial feedback would be that the music is way too loud and that I really like that you used Python to show how to construct the bits and pieces.

Nice, one suggestion, please ditch the music, its distracting and doesn't play well with speaker's voice.

Why have neural networks become synonymous with backprop networks? Is it because those are the most successful? What happened to bidirectional associatve memory and Kohonen maps? Does anyone take the biological inspiration of neural networks seriously anymore?

people are mostly interested in what is successful and backprop/gradient descent (with certain specific architectures) is what has been dropping error rates on all sorts of tasks in recent years.

This is really great, but like a few other people, I found the music incredibly distracting. At first I thought I had left my Spotify on!

At the risk of not sounding very clever, I have to admit it's way too fast for me

These (excellent) videos made me realize that picking up AI as a hobby (which I intended) would not be a good idea in my case. There are simply too many different domains of knowledge I would have to learn about to be able to finally do something slightly useful.

I neither have the time nor the brain to do this. Fascinating stuff, though.

My thoughts exactly. There's even too much of meta-knowledge here :) Without some solid background, it's not something you can casually pick up as a hobby. But yes, as a programmer (not dealing with anything AI-related - right now it's middle-of-the-road mobile apps) I am very impressed by what deep learning proves to be capable of

That's the thing with these "tutorials". I remember watching one video that was "Calculus in 5 mins". It made a lot of sense, but that was after learning it, but to someone who's never been taught the topic, these might not make any sense

It's not a binary difference between learned and untaught. Many people have taken calculus classes and probably not enjoyed any lasting insight, either because the teaching was bad, or there were too many other subjects swimming around their mind at time. Or never taken a class but developed a surface familiarity with the main ideas. These types of tutorial could be useful for them.

Man the the second video gets steep, quickly.

That was my impression too. There should be a video to demystify that second video :)

Yes! Where can I turn to for this?

Is there a course that teaches me just the maths that I need to understand this?

Plan A: The material is difficult, so present it very slowly and carefully.

Plan B: To avoid boredom, present the material as quickly and densely as possible, with lots of constantly changing detail, with simultaneous visual, audio and even some light background music.

These videos are a bit like "Hitchhiker's Guide to the Galaxy" meets "Khan Academy".

Some people will like them, and some won't. I like them.

no one can ever claim there that is a shortage of tutorials with regard to neural nets (and deep learning)

Tutorials - sure. Actual rigorous analysis - not at all.

Well, this is pretty good http://neuralnetworksanddeeplearning.com/

Also these lecture notes https://github.com/joanbruna/stat212b

No no, I mean the theoretical analysis of neural networks. It's true that some authors attempt to do that, in fact even famous researches attempt to do this. However their arguments and analysis often breakdown in the general case and what their papers boil down to is that neural networks are good at modeling functions that they are good at modeling.

In fact the state of the field of machine learning is essentially the state of mathematics before Cauchy and Weierstrass.

What kind of analysis do you miss? There is a huge amount of research analysing all kind of aspects. See Schmidhubers overview paper oder Goodfellows Deep Learning book, or many other references to get an overview.

Of course, there are still many open questions. But I don't think that there is any aspect where there is no analysis available yet.

To demystify NNs further we need to stop graphically representing spurious interactions. If you can perturb or remove a link win between any two neurons i & j, then that interaction is spurious and shouldn't be represented by in the graphical network representation. Doing this iteratively you can start to better appreciate that neural networks are computational circuits that use thereshold functions instead of logic gates.

To me 'demystifying' really means making simpler to understand. Yours sounds like technical pedantry, that while true (idk?) is meaningless to anyone who actual needs a neural net demystified.

Um, no. Show an electrical engineer the circuit diagram of an 8-bit added and they'll know the function right away. The function of the NN is similarly determined by the topological circuitry--in fact the function can often be preserved when you represent these networks as Boolean network, but all that's concealed when we don't remove spurious connections and it starts to feel like weird voodoo mathemagic. This is not an opinion, I've published on this and have seen scientist get confused because of it.

It's a great short introduction, really worth looking through the code examples that they have on github too - https://github.com/stephencwelch/Neural-Networks-Demystified

There is certainly no shortage of new tutorials bubbling up on neural nets. One of my favorites - https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearni...

People who are finding a difficult time concentrating, an clone the git repo where it has ipython Jupyter notebooks.

Link to Installation of Anaconda(by Continuum): https://www.continuum.io/downloads

Link to the Git Repo: https://github.com/stephencwelch/Neural-Networks-Demystified

In case the author of the videos is here, do you have plans to add some videos about Convolutional NNs and Recurrent NNs?

I know a lot of developers like myself that know about traditional NNs, but are not familiar with those two.

Not the author, but I wrote an article introducing conv nets you might find helpful: http://colah.github.io/posts/2014-07-Conv-Nets-Modular/

I also have an article on RNNs, although it's focused on explaining a special version, called an LSTM: http://colah.github.io/posts/2015-08-Understanding-LSTMs/

If you have experience with functional programming, you might find this a nice way to think about Conv Nets/RNNs/etc: http://colah.github.io/posts/2015-09-NN-Types-FP/

Thanks. Very nice articles.

I liked! Already done the first 3, the music didn't bothered me neither the voice (i'm not a native english at all.). As for the concept explain, it's true that some equation are a little bit out of reach for someone that has less mathematical background. But i understand the concept so far, which are well explained.

I'm not sure i would be able to just write (or decide) the equation for the neural network. On what criteria can you decide this will work better or not? What are your key to make a decision?

get rid of the music and find someone to talk who uses less sibilance

I saw some of the videos and i agree with the suggestions provided here but i really like the simple way of explaining and less mathematical more programmatic approach. It'd be awesome if you could make some neural networks with cuda or any other library like tensorflow or theano. Good luck.

Cool. I have a word of advice, though. The term 'scalar product' when referring to the product of two vectors is a scalar, not a vector. In the back-propagation video you mis-spoke here.

Otherwise, good job.

Great videos!

Does somebody know a source for a nice data analytics/machine learning taxonomy or something (grouped by the class of problems the different methods solve)..?

This looks really interesting. Thanks! I'll save these to start watching later.

Awesome video ! Crisp and to the point

great content, most annoying music ever!

Mathematical more programmatic approach.

Reply on news.ycombinator.com