A Guide to Deep Learning | Hacker News Reader

A Guide to Deep Learning

yerevann.com

417 pointsadamnemecek34 comments

The primary thing missing from all of these guides is that you need to have two things for ML:

1. A purpose for utilizing it

2. A data set to train/act on

Without that, all you get are a bunch of shovels and picks, but no idea of what kind of wood/bricks you need or a plan for the house.

Thanks for the feedback, Andrew!

This guide is designed mostly for those who already know what problems they want to solve, but don't know how to start or where to look for high quality and up-to-date educational resources. Also this is designed for those who want to do research in this area and just want to develop better shovels (that was me 2 years ago).

Regarding datasets, each course we suggest in the guide has its own way of dealing with datasets. Most of them teach how to work with MNIST, which is pretty good for many purposes.

I agree that another guide on datasets could be useful for some people.

Yes, thank you for the resources and my comment is not an indictment on your page as I think it's well organized.

Moreso I'm commenting broadly on the recurrence of similar guides and how they are generally not accessible to people with no exposure to ML.

All the best.

I will mention that you can get surprisingly far with just a little bit of labeled training data now. Transfer learning is a pretty powerful technique.

Undoubtedly. However there are three sets of data you need to a functional system: Training Data and Validation data sets (a lot of sources exist already) and the data set you want to get insights from (Novel data sets usually).

These of course work together in reinforcement scenarios.

So for example if you want to build a tool that tells you what kind of objects are in any given image you can use ImageNet to Train and Validate your CNN. At that point you have a classifier. But for an actual useful application you need a user to submit new images to actually give some kind of valuable output.

It's that second data set that is super hard to get - because it's novel.

You still need to train a model on what you're transferring from. It's not like you can manufacture trained models out of thin air.

I think the idea is that, for example, you start with some pre-trained visual object-identification network, remove the final logistic-regression layer, and replace with a layer which you train your problem on.

Everyone's into deep learning, but what would I actually do with it? With some other field, like computer graphics, one can fairly quickly get a 3D cube spinning on their screen and know it has some relation to the special effects in the Star Wars movie they just saw. No one makes it obvious what the hobbyist can expect to do with deep learning or how it relates to the broader world.

Everyones into Graphics programming, but what would I actually do with it?

The uses are myriad, you can build Star Wars Movies, you can build computer games. You can do CAD and 3D modelling and printing, you can visualize fluid simulations

But each is an entire field.

The same with Deep Learning -

You can do image classification for medical diagnosis. Self Driving Cars. Realtime Translation. OCR. You can model chemical reactions by doing latent space exploration. You can model gene-gene interactions. You can build a image recogniser to tell if your cat is on the couch. You can build an AI that plays GO, or any other system or game where the rewards are time-delayed and sparse. You can make it play Atari games.

You can stitch satellite images together. You can reconstruct parts of photos that are missing. You can colour black and white images. You can de-noise wind sounds from microphones. You can search for comets. You can monitor deforestation. You can count cars in car parks. You can search vast ocean areas for survivors. You can build security drones that fly around at night and look for anomalies. You can turn a webpage of unstructured text into structured query-able forms (see named entity recognition). You can create visual art (google: neural style transfer)

These are just a small drop of some of the cool things that are going on - and while it might be difficult for you to advance state of the art without a GPU cluster (just as it would be difficult to advance state of the art in Graphics programming) the hobbiest can certainly start in any of these fields, just pick one that inspires you.]

EDIT: I run a Deep Learning startup - if you want some pointers or help getting started, or would like advice don't hesitate to email me - it's in my profile :)

One of my partner's tasks as a legal assistant is to go through mountains of OCRed PDFs and classify them and extract pieces of data so that lawyers and paralegals can go through them more easily. Do you imagine deep learning would be an appropriate means of automating that, or is it overkill?

No not overkill at all - I think the first step is to augment the human so the machine helps them, then it can totally replace all of the laborious stuff as time passes.

Imagine something like: A deep network reduces a 20 page document to a summary of 4 or 5 sentences, you can click on these sentences to "expand" them out, eventually getting to the original text. Saving them from reading the whole document

A separate classifier automatically classifies the document into one of say, 20 categories (or whatever is appropriate).

A Deep Learning named entity recogniser extracts the Human names, Dates and times, Email addresses, Company Names, email addresses, Money amounts, and numbers from each document, then off to elasticsearch for indexing and easy searching.

Then we can start to play with higher level legal concepts that (for example) set precedent, or search for certain logical fallacies. (the next step past machine learning is machine reasoning - and it's starting to be possible now)

Not overkill at all, but youy should know that there are a lot of competitors in the e-discovery software category, and they do a lot besides classification, starting from the ingestion of email archives, through extraction of attachments (recursively, because Outlook PSTs get mailed as attachments too), identifying and attributing quoted text, deduplication of messages and documents, SNA, clustering, normalization, canonicalization, NER, synonym identification (aside from correctly identifying people referred to by nicknames, advanced implementations can also figure out when people are using a code-word to evade filters), etc.

I have built deep learning systems for similar problem domains. I am happy to chat to you about your example. Email me if you get a chance.

You can ride your bike with no handlebars. ;)

deep learning is where databases where 20 years ago.

you can easily imagine putting data in a db like server, and querying that data asking things about that model in respect to other things at your disposal in different times.

One area I find interesting is classification. E.g. I'm building a search engine for lectures, so I built a utility that classifies search terms based on whether they are for a speaker or not, based on the text of the term on Wikipedia.

The next stage will be to take articles posted to the various programming language subreddits, and build a classifier that classifies articles by language or topic. I did this previously for talks based on heuristics, but this will allow for a lot more categories: https://www.findlectures.com/?p=1&class1=Technology&category...

I don't know. I'm not even into deep learning but I still see things all the time I wish I could test deep learning on.

If links are broken for you, then turn of your adblocker, because he is measuring clicks with Google analytics and he's JS is broken thanks to the missing ga function.

#issue reported

thanks for reporting. It should be fixed now

Hrant!! maloch, front page :)

I'll bite.

We see these being posted every week. Why?

No one has yet won the "These are the best resources for learning ML" aggregator and a lot of people are interested in and building them. So people keep posting them.

Many people (especially students from universities) keep asking me and my coworkers at YerevaNN about good educational resources. They have different levels of math background, some want to study theory and watch visualizations, others want to play with the code before reading formulas..

At some point we understood it's better to spend some time and build a guide that will cover most of these questions (and, as always, we spent a lot more time on this than we expected)

I think OP is asking, why spend time writing another guide when there's plenty that suffice, including full books and papers on the topic?

I was looking for something that would answer common questions for some time, but couldn't find anything. Pointing students to research papers unfortunately doesn't work in most of the cases. Deep learning book is too hard for some and they want to see a comparison of different sources.

This looks really good, and an interesting wag to describe the landscape. Is it just my phone or are the links completely broken on phones?

Confirmed that links are inactive on my phone as well, probably because they don't seem to be prefaced with URI schemes (i.e. their http:// and https:// prefixes).

They don't seem to work on my Mac (Safari) either.

this is by far the most visually attractive list of deep learning resources i've seen. and they hit a lot of the main points.

I didn't see k-means nearest neighbor in the list. Also, wouldn't a mathematical statistics be a prerequisite?

As far as I can tell, Deep learning in practice is pretty far ahead of the theory, so people are largely following their intuition around rather than being guided by theory.

I saw stars indicate difficulty; do colors also have some meaning?

They contain resources from the same source. e.g. all the "green" content comes from http://www.deeplearningbook.org/.

Good consolidation @adamnemecek...

Reply on news.ycombinator.com