AI Playbook | Hacker News Reader

AI Playbook

aiplaybook.a16z.com

806 pointsjbredeche113 comments

I found this really frustrating to read; on 'training your own models':

> Deep learning is a class of machine learning that has gotten a lot of well-deserved attention in recent years because it's working to solve a wide variety of AI problems in vision, natural language processing, and many others.

What others? Be specific.

The problem is not that people don't know how to build neural networks.

The tensorflow tutorial is extremely comprehensive and there are lots of resources for people who know what problem they're solving to try to build a solution.

The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.

Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.

That's. All. It. Does.

It takes an input array and outputs an output array; a magic f(x) function.

The thing people don't understand well is not how do I implement f(x); the problem is how do I convert my domain problem (I have 2000000 shelf products in my database and a tonnes of sales information) how to convert that into an input array of numbers? What is the output array of numbers I want out of the function?

The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.

That's why its good for images; because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem.

...but if your problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.

tldr; The problem people are having is figuring out input and output for domains other than image processing; not how to build a NN, or how to choose which NN to build.

Guide misses the point entirely.

(You might argue this is somewhat uncharitable to what is actually a well written guide to ML; but you can read the about section for yourself and see what you think; I feel like the guide needs to focus sharply on practical business outcomes if it wants to do what it says its doing)

Technical but important quibble. You've described supervised learning: given pairs (X, Y), learn the function F such that Y = F(X).

Unsupervised learning takes samples and generates more of the same kind of data. We can see unsupervised learning as being given samples (X) and training a function X = F(noise) that transforms a random vector into an output whose distribution resembles that of the original generating function.

Reinforcement learning sees a reward R(X) of policies X and learns to find X such that R(X) is high.

All of these have been known to operate on trees or other representations instead of arrays.

This matters because there are business problems that are not supervised learning. Reinforcement learning problems especially. Sometimes I can say how well an output is doing and I want an output that does better.

In practice a lot of problems arise because people do not understand the difference in the types of problem that are appropriate for supervised or unsupervised learning, or in other ways trying to wriggle out of having to generate real labels.

Thanks for writing this. It sounds like what I've experienced in this kind of work isn't just accidental. I am working on leveling up math skills to be able to understand the process for TensorFlow etc, but I get the feeling that once it's finally tractable I'll still have the same problems of finding how to transform the inputs into whatever works best for the model, what to output, and how to use the output. This seems to be the part that can't be solved by off-the-shelf packages. And the part where many or most exciting discoveries will be made in the future.

You might find useful to study algebra. It helps finding mathematical representations. You are in luck if you can find a representation that can be mapped to a vector space.

Definitely, Linear Algebra is next up, and it is seriously needed.

Would say Algebra (and Linear Algebra as mentioned below) are the most useful to learn for ML/AI?

I would suggest following Siraj on youtube: https://www.youtube.com/watch?v=N4gDikiec8E really insightful!

I can't take it. He's blasting memes out at 1hz. That's 60 times the acceptable safe limit.

>>The problem is that people don't understand what they can do with this technology, or even, in a meaningful sense, what it is.

Yes, exactly.

As a "Pull" programmer [1], I can't tell how I would use this technology to solve problems I have or my company has. This makes it difficult for me to get motivated to learn it, despite all the buzz and excitement.

I went through the tensorflow tutorial. While it is interesting, I didn't find it useful, the way learning a new programming language or framework or library would be.

[1]http://v25media.s3.amazonaws.com/edw519_mod.html#chapter_71

I could not agree more. There should be more articles like this one (https://hackernoon.com/how-to-better-classify-coachella-with...) clearly outlining the steps what needs to be done to solve a real world problem. The algorithms by themselves are very powerful.

>...but if you problem is, given an input of say [price, date, origin] and an output of [value to business], you can solve that problem quite trivially without needing any neural network.

That's one general statement over another. Consider https://arxiv.org/abs/1508.00021 as a counterexample.

Yes it is; and you are correct, it's not necessarily trivial; but the point I'm making isn't about reducing very high dimension inputs into lower dimension outputs or picking the best function to map between data sets.

...how you implement those functions, how you optimise for the best results and performance is an implementation detail.

If you are picking your solution before you know what your problem is, you are falling into premature optimisation.

Understanding how to reduce your problem domain into a series of functions that take numerical data and return numeric data is the key practical problem that most people face, and it's poorly described in the literature.

(That's why most people just pull out that classic line '...its great for image recognition and ...many other things...')

It is an implementation detail if the current implementation is viable, but if it is not, then designing a viable one might be critical

Totally agree. As someone who does scientific modeling, fitting models is easy. Finding the model that correctly describes my problem and can be trained with existing or easily acquired input data is hard.

In many cases this is true but it is important to note that for deep learning, the training was the hard part that was missing for a long time. Better weight initialization and momentum methods were what really made deep networks work (not just GPUs, as some people tend to believe).

GPUs were necessary but not sufficient, true. But the real change (at least for supervised learning) was not in the polishing on training methods (an incremental improvement), it was the availability of big enough data sets. No amount of modeling can make up for insufficient data.

> The reason this works well for images is because you can say, ok, this output number represents a tag (dog, cat, etc) for each discrete value, and the input numbers are just an array of the raw pixel data.

> because a traditional function implementation of f(x) that takes a I of dimension say, 65535, and produces an output of dimension 1 is a hard problem

Nitpick maybe, but you do not represent each class by a number. This would create a bias to consider subsequent categories as similar (why would aardvark=0 be more similar to antilope=1 than cat=2?). Instead you have one dimension per possible-value (aka one-hot encoding) and the value in that dimension represents how likely the photo belongs to that class (with values summing to 1, softmax, if it's a one-class problem, or no such guarantee if it's a multiclass problem).

> Given a array of input numbers [i1, i2, i3...] it provides a black box that can map that input to a set of output numbers [o0, o2, ...] where the dimension of the input and output are different.

> That's. All. It. Does.

No. That's. WHAT. It. Does.

Your emotional overreaction to this site has induced bias leading to a false negative -- mistaking hype for signal. Fact is, using deep learning, newbie grad students have routinely and resoundingly outperformed nearly all the state-of-the-art techniques in signal processing in less than FIVE years of trying. Image analysis and speech recognition have been revolutionized; 40+ years of R&D expertise has been largely supplanted in the blink of an eye. Dismiss this tsunami at your own peril.

Does DL scale to harder problems? Why yes, it does. No computer implementation of the game Go had outperformed the best humans until deep learning was used, then it beat the best humans almost immediately (unlike chess which took decades to reach that level using traditional AI methods). I have little doubt that significant achievements await DL's application to more complex tasks like language analysis too. To believe otherwise would be severely short-sighted.

Does deep learning also solve trivial pattern rec problems like regression or curve fitting as efficiently as some linear ML models? No, but who cares? By that reasoning, we should dismiss Einstein because many 12 year old children can multiply numbers faster and for less money.

Please don't let my comments that 'The problem people are having is figuring out input and output for domains other than image processing' suggest to you that I'm saying that no one is doing interesting things with machine learning, or that I think that the solvable problems with it are somehow limited in scope to just that. There are plenty of things you can do with it and its quite remarkable how effective it can be, you're quite right.

I feel https://www.tensorflow.org/tutorials/word2vec is really good example of what I'm actually talking about.

Given a corpus, how do you represent it in a form that is suitable for processing?

Actually, it turns out that's a really hard problem which a lot of work has gone into; but many tutorials and introductions to ML skip over it as though it were a trivial implementation detail on the back of which type of NN you decide to use, or how many layers you want.

I wrote the parent comment because on reading http://aiplaybook.a16z.com/docs/intro/survey-goals, I got excited that maybe someone had actually broached this topic in a meaningful way.

...but then as I read through yet another guide to installing tensorflow followed by assert!(example 101 works), I lost all enthusiasm for it. I don't know; I'm happy to admit I'm perhaps overly critical, but this wasn't a false negative for me, it was just a frustrating unfulfilled promise. Just read the tensorflow tutorials, they're better.

The parent didn't react emotionally and didn't even attack deep learning as a subject. He said deep learning gurus are failing to make it understandable.

Here's someone else who cherry-picked a quote and ignored the central argument: https://news.ycombinator.com/item?id=14347068

Almost every one of your sentences are talking past the parent. If you want to be argumentative instead of helping the parent as he asked, then it might be best to find actual quotes to disagree with.

Yeah I find that a lot of experts have a very difficult time trying to explain things in simple terms. But maybe they haven't been asked to.

The ultimate goal of deep learning is to remove the need to transform input.

The traditional computer vision pipeline works like this: you preprocess the image, featurize it with features like SIFT, and then run an SVM classifier. However, this pipeline is totally beaten by end-to-end neural networks.

Same will hold true for other domains. As long as we've discovered the right architecture, aka. the family for f(x), learning still boils down to training your models.

Some examples from the AI Grant project (click Spring 2017 finalists):

https://aigrant.org/

So true. What's the best reference, guide, or write up you've come across that helps clarify?

This is an important insight, however I think more of a research topic. The guide was careful to provide guidance on problems that have a very firm foundation. It didn't want to discuss something that would just drag its audience into the weeds.

The dimension of the input and output arrays needn't be different, right? There's no reason you can't input a image into a convolutional neural network and output an altered image of the same size.

Couldn't say it better. Totally agree.

> tldr; The problem people are having is figuring out input and output for domains other than image processing.

This and some of your other comments in this thread lead me to believe you are unaware of how DL is being used in NLP. I would argue real industry use of DL for NLP dwarfs that of DL for images. Personal assistants, search, customer service, sales, chat are the big general cases, let alone company-specific tasks (.e.g. financial data, reviews) or other NLP domains that are not particularly industry-adopted (.e.g. translation, summarization). There's a lot of demand for NLP DL experts, and it's no surprise this year that Socher's NLP DL class had 700 Stanford students enrolled.

Okay, but you cherry-picked a quote and ignored the central argument, which is that this website doesn't address how to map your problem to the domain of deep learning. Nobody cares how many students the DL class at Stanford had enrolled. Your comment is kind of trying to defend deep learning (when it needs no defense) while putting down the parent without actually providing any meaningful assistance.

I think that's exactly what he's pointing at: People in general are unaware as to how to map such datasets into NNs. You could help by giving an example of some common application strategies from NLP, instead of just saying that it's done.

Artificial Intelligence is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.

Please don't write this off as a reddit-esque comment. I really think the analogy holds -- just take a look at Product Hunt front page, or HN for that matter.

Some people are really doing it, but they're not really doing it properly. Some of them will get better at it, and some will realize they started doing it way too early.

In a few years though, most people will be doing it, and it'll be great (they'll exagerate how much they do it though).

Yes, the analogy holds.

It was pretty incredible to open up https://vimeo.com/215926017 and click at a random timestamp (29:30) and hear "we're trying to use AI to diagnose cancer by examining the DNA that's free-floating in your bloodstream."

This is clearly the future, if it can work. That's probably one of the biggest startup ideas that you could possibly have. I hope that whoever's behind it is successful.

Freenome is the company, they're doing interesting work but it's still early days. This isn't actually a very new idea, it's been floating around for years in academia (I've personally been hearing it about it for at least 5 years). The hard part isn't really the AI per se, it's getting enough DNA from the circulating copies (i.e. sample prep) and being lucky enough that the cancer you're trying to diagnose actually puts out enough DNA for you to measure at an early enough stage to be clinically relevant. In other words, if you can only get DNA when it's at stage 4 and metastatic, your DNA test isn't actually very useful. If you can get stage 1 cancers, now you're talking, but in those cases tumors tend to shed a lot less DNA (among other reasons because they're smaller).

A very good friend of mine is their CSO, I think they've got a really cool idea and could indeed go far, but it's a fiendishly hard problem.

As someone who has zero experience but is extremely interested, could you ELI5 why it's difficult to detect whether some DNA is cancerous?

In other words,

The hard part isn't really the AI per se, it's getting enough DNA from the circulating copies (i.e. sample prep)

It sounds like there's a way to detect "This DNA is definitely cancerous," but the problem is that the test only works when you give it enough DNA. My question is, where does this "enough" criteria come from? Why doesn't it work with a single cell? (I know there are obvious reasons, but I don't know what they are.)

These are all good questions, and full disclosure I know only the basics in this area. DNA can be identified as cancerous for a variety of reasons, though typically what you're looking for is some mutation you know to exist in cancer. This could be a single point mutation (insertion/deletion/change of a given DNA base pair), a loss/gain/modification of several adjacent base pairs, or whole scale genome rearrangements (deletion, insertion, inversion, replication) that will result in of two pieces of DNA being adjacent although you know they shouldn't be next to one another (see https://en.wikipedia.org/wiki/Philadelphia_chromosome for a quite well known example). You could also look for copy number variation (gene X has way more or way fewer copies than expected). I'm probably missing some as well, but those are the biggies.

To detect these, you're going to use either something like microarrays, QPCR, or DNA sequencing. Today people talk almost exclusively about high-throughput sequencing (HTS) because it has gotten quite cheap, gives you large amounts of data, can be made massively parallel and isn't as hindered by needing to know very precisely what you're looking for (microarrays and QPCR require somewhat more targeting; HTS also needs you to know what you're looking for, but not in quite the same ways).

In all of those cases you need a sample of DNA (duh). These can all be made to work if you have a single cell in hand, it's tricky but sample prep does exist for single cells at a time. Trouble is, you need that cell in hand. There are about 5L of blood in the human body. A human cell will range from 1-10 pico-liters. You will take a blood sample of 2 milli-liters. Chances are pretty slim your one cell will be in that 2mL. Even if it is, there are billions of other cells in that 2mL adding additional noise. Usually though, we're looking at circulating DNA fragments with these technologies, and depending on the stage of the cancer and the type of cancer, there may be enough to pick up on in the blood. How to do that best is an area of active R&D.

Just dug up review if you want to learn more:

http://journals.plos.org/plosmedicine/article?id=10.1371/jou...

Thank you very much for the breakdown and the reference!

For some reason I assumed that this detector would be embedded into your circulatory system, so that it eventually scans all 5L of blood in your body rather than getting lucky with a 2mL draw. Now that I say it, it sounds pretty unrealistic, but I wonder if it's even remotely possible.

So if we want to get a little sci-fi but still within the realm of not-physically impossible...

If you wanted to monitor for a very specific piece of DNA or a small number thereof, you could have an inline piece of hardware that samples the blood continuously, trying to enrich for those. You could then use a hybridization-based (kinda like a microarray) test to check for the presence of that DNA. Right now, the relevant technology just isn't there. You'd need new materials (of the top of my head they'd need to be highly resistant to fouling, among many other properties), probably a completely new filtering/enrichment system (again fouling and clogging and specificity and sensitivity are huge problems here), and a decent, cheap, high sensitivity and reliability hybridization based detector. All of those things are quite hard, but I don't think they're completely impossible. I do think it's highly unlikely we'll be seeing this in the next 5-10 years though. But I could be wrong! :)

A similar device was proposed a while back-- a nanopore DNA sequencing array implanted into the heart that checks all circulating DNA. In context, the authors had proposed embedding some enzymes in every neuron to record action potentials, neurons would excrete the DNA into the bloodstream, and then you would recover the entire connectome and all the action potential history by DNA sequencing any circulating DNA that makes it to the heart.

> Even if it is, there are billions of other cells in that 2mL adding additional noise.

Billions? I think you meant millions.

Oops! sorry, right you are, small multiplication error, but what's a few orders of magnitude between friends ;)

No problem. It's just that I recalled some cell counts from bloodwork and figured that had to be off.

So, the real key breakthrough here wouldn't be in the analysis phase, but rather in the collection phase? Would a silver-bullet-to-cancer tech look something like "nanites that passively diffuse throughout your tissues; then, when "activated", grab the nearest cell, squeeze the liquid out to compact it, and high-tail it over to your bladder to deliver those sample cells into your urine"?

I would claim that a more important company in the space is Grail (Illumina spinoff with about $1B in funding).

>it's getting enough DNA from the circulating copies (i.e. sample prep)

If only Theranos had actually worked...

You don't need AI tor that.

Yeah it seems like it's just lots of in vivo prototype tests plus statistics?

"AI" is what we're calling statistics these days..

You need it raise money. Cancer + AI = $$$

In Los Angeles area, how does one get involved in these projects? I would like to volunteer at a local research lab. I am coming from a computer scientist background and would like to learn more about the medical space.

Look at the list of faculty for UCLA/USC in the Biochemistry or Bioengineering areas of focus. Find a lab or two that have a blurb that interest you, read a paper or two from those labs, then email the prof.

Much like in CS/Software, it'll help if you've read a bit of their work and/or can show some way you could contribute directly. i.e. "I specialize in CS and did a side-project analyzing <gene_dataset_x> where I found <something_cool>. I read your <paper> and found <this_thing_you_did> was very cool for <insightful_reason>. I'd love to get more involved in the field and possibly find a volunteer position where I could contribute, would you have time to chat on the phone or over coffee sometime?"

One place to look for genomic data:

http://www.internationalgenome.org/data/

There are others, google around and if you could also look at https://www.plos.org journals to keep an eye out for new freely available articles in this and other areas of biology. Good luck! :)

NantWorks is basically doing this. You could get a job there. Otherwise academic labs and Cedar Sinai maybe.

I find a lot of discussion of AI skips over all the hard parts of productizing. Recently wrote a quick article on that topic for anyone who is interested: https://medium.com/towards-data-science/hard-earned-advice-f...

"AI products are hard. They require solutions to problems that are impossible for the human mind to comprehend... "

The start of your second sentence seems to me pretty inaccurate.

How about "They require solutions to problems that the average human mind shies away from attempting to comprehend..." (I assume the rest of the argument is "thus making it hard for anyone with the problem your product solves to realize they even have that problem.")

That's still remarkably grandiose.

How so? No one really has any idea how, say, NNs trained on machine vision are working. Hand designed programs to do machine vision don't even get a tenth of the accuracy of NNs. That's just one domain. And one that has been very well studied by humans for decades with little progress.

AI solves problems that are vastly too hard for humans to figure out.

> No one really has any idea how, say, NNs trained on machine vision are working.

False. Plenty of work has been done to show what exactly is going on inside NNs including some very impressive visualizations.

> Hand designed programs to do machine vision don't even get a tenth of the accuracy of NNs.

And that's false too. Hand designed programs to do machine vision can get significant fractions of the accuracy of NNs and sometimes even perform better and/or faster depending on the domain.

It is a lot more work to write that code though and a NN might clue into features that you weren't aware even existed.

Sure there are some pretty visualizations of what images turn on a neuron or whatever. But no one could hand code an algorithm that does the same job that's even half as good.

>And that's false too. Hand designed programs to do machine vision can get significant fractions of the accuracy of NNs and sometimes even perform better and/or faster depending on the domain.

Source? A few years ago people thought it was amazing an algorithm could recognize cats 12% of the time. And even the best non-neural approaches use some form of machine learning.

>a NN might clue into features that you weren't aware even existed.

That's the whole point...

Your statements are nonsense. Yes, we do have a good idea of what goes on inside a NN trained on images and Yes hand designed programs exist and work and sometimes work very well.

It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not.

Recognizing cats is not the sum total of machine vision problems and those 'pretty visualizations' go a lot further than what you give them credit for.

I've been doing nothing but reading up on machine learning and working through piles of example code and models for the last couple of months and I think based on what I've seen so far that we are definitely headed in the direction that you suggest. At the same time we aren't there yet and such hype does not help.

Keep in mind that the domain of machine vision is extremely broad and much wider than Kaggle competitions and that saying 'we have no idea' when clearly we do tries to sketch NNs as magic and they're not.

If anything the present generation is best characterized as an optimization of what you could have done for many years, the only reason they are the way they are is because we'd like to have our results in a reasonable time. That makes applying NNs to large datasets possible and that in turn is what helps to solve some of these problems.

>Yes, we do have a good idea of what goes on inside a NN trained on images

I'd love a source for this. The last time I checked I didn't find much. Sure we know the first few layers do stuff like edge detection. But the higher layers are much harder. And even if you know images are activating them, writing code that can accomplish the same task is much harder.

>It's just that NNs are rapidly moving ahead and maybe some time in the future both your statements will be true but today they are not.

The best NNs have already surpassed human accuracy at imagenet. That's insane! No one expected that result a few years ago. Yes object recognition isn't the only task in machine vision, but it's pretty representative.

> I'd love a source for this.

I will do some digging for you tomorrow, it's 3:30 am and I really should get some sleep.

> Sure we know the first few layers do stuff like edge detection.

Yes, basic primitives.

> But the higher layers are much harder.

Well, those higher layers are built up out of the primitives the lower layers contain. So you get things that are conceptually easier to understand rather than harder.

> And even if you know images are activating them, writing code that can accomplish the same task is much harder.

That I completely agree with, I've basically gone that route in the last couple of months.

> The best NNs have already surpassed human accuracy at imagenet.

Yep. It's extremely impressive.

It's a watershed moment in the history of computing and half of us are still asleep or in denial. Which is fine by me. But please don't hype. It isn't magic and it isn't perfect - yet.

edit: ok, found two of the things I've looked at:

https://stats.stackexchange.com/questions/114385/what-is-the...

https://stats.stackexchange.com/questions/146413/why-convolu...

And this is interesting code:

https://github.com/yosinski/deep-visualization-toolbox

If anything I think that NNs as applied to vision problems are somewhat easier to make sense of than doing the same for NLP or audio.

Though it definitely feels like cheating compared to actually programming a solution.

An alternative name for 'deep learning' would be 'automatic multi-level feature engineering', but that wouldn't be nearly as sexy.

It's a bit more than 10%. Refer to the submissions to ILSVRC .

>> Hand designed programs to do machine vision don't even get a tenth of the accuracy of NNs.

"A tenth of the accuracy" of a 100% accurate model is 10%. So you're basically claiming that nobody had achieved anything with machine vision before NNs (you probably mean AlexNet in particular), which is patently absurd and completely unhistorical.

Why is it that this subject attracts the attention of so many people who know so little of what they're talking about?

In real world datasets a single class may only be in 1% of the data to begin with. The vast majority of images on the internet do not contain cats. So making a cat recognizer that is even 10% accurate is a significant challenge, simply because of the low base rate of cats. Imagenet in particular has 1000 equally balanced classes, so each class makes up 0.1% of the data. And 10% accuracy is way better than chance.

Additionally very few machine vision systems are completely hand designed. They all use some form of machine learning and optimization. Just not necessarily NNs.

Classification is only one problem in machine vision. As in, not even 10% of the whole.

Edit: Btw I just had a look to confirm and what you're saying about "1000 equally balanced classes" is inexact, to put it midly. ImageNet has 100,000 SynSets (as in WordNet) and aims to have about 1000 images illustrating each. Individual images may illustrate several synsets- you don't get one image per class, say.

Also: AlexNet was not a cat recogniser and having unbalanced classes is a problem for machine learning, but not a dealbreaker, per se.

>Classification is only one problem in machine vision. As in, not even 10% of the whole.

Classification is extremely representative of the challenge of machine vision. Good classifiers can often be adapted to do localization and other tasks, e.g. by scanning the image to see what parts contain an object.

The Imagenet Large Scale Visual Recognition Challenge has 1000 classes which you can browse here: http://image-net.org/challenges/LSVRC/2015/browse-synsets I forgot that Imagenet itself has more classes than that. But it's best known for this specific competition of 1000 classes, and everyone uses that as a benchmark.

>freaking AlexNet was not a fraking cat recogniser.

I never said it was... It's actually a dog recognizer mostly : )

No it's not, it's just the one that neural networks and amateurs on Kaggle are good at.

There's more to computer vision than image classification, multicamera geometry is still necessary for any serious work in computer vision.

Impossible for my mind perhaps :)

I'm curious, did you not hear the feedback? I've been looking at how writing is miscommunicated and misheard and I would be interested in understanding what you heard. I couldn't find an email address in your profile here or on medium; Hopefully you'll see this and respond (in email is fine, mine is in my hn profile, I won't share it with anyone else).

We've developed abstractions to help reason around complexity, high dimensionality, and the problems that can be solved with each. My intention with that particular sentence was more around the actual navigation of that space. However, I didn't put much effort into that paragraph and intend to revise it when I have a bit more time.

Just added my email to my profile - feel free to reach out if I'm missing something or you'd like to discuss more.

Is this a subtle ad for clarifai?

For the record, I've worked on building a classifier for nudity detection and not much get's better than starting with a pre-trained model and fine-tuning to your specific rules. I wouldn't be surprised if this is the future of deep learning: Start with the closest pre-trained model that has trained for months using custom clustered hardware and fine-tune further on commodity hardware.

Have you looked at the Yahoo Open NSFW model? I ported it to Algorithmia if you want to try it out to compare:

https://algorithmia.com/algorithms/spullara/YahooOpenNSFW

Ya I have, it is the 1 that does best so far besides a custom fine-tuned model. It beats google, msft and clairifai.

That's not the future, it's what is standard in ML application engineering. For example, a good portion of classification systems use models based on ImageNet, VGG, Ngram or AlexNet and then fine tune the last (or last few) layers.

Transfer learning is an old and well known technique and is used ubiquitously. [1]

[1]http://cs231n.github.io/transfer-learning/

That's a very good business model as well to sell these pre-trained models. Like a market where you choose what you need.

That's what Algorithmia is trying to do.

[1]https://algorithmia.com/tags/machine-learning

Or, better, you submit a pre-tagged sample set and it tells you which models match and how well.

I think this is true not just of images but also for text with things like word2vec etc

While this site has some great info for non-technical folk who are serious about learning AI, I think the site will fly over the heads of most "line-of-business leaders" at whom it's aimed.

In my experience, those folks think of AI like...

My company is collecting tons of data in [insert data warehouses here]. How can I bolt on some "AI" and find out what's "interesting"?

> My company is collecting tons of data in [insert data warehouses here]. How can I bolt on some "AI" and find out what's "interesting"?

For quite a few business types, this can be further simplified to just:

How can I bolt on some "AI"?

It's a nice bonus if you have some data and can find interesting things from it. But the goal for a surprising number of managers is just to be able to say they have some advanced AI technology doing, well, who knows what, just so long as there is definitely some advanced AI technology doing it.

This is a great point, and I think I fall into that bucket you describe. I have just enough information to be dangerous.

There does seem to be a lack of available tools even for someone with a well-defined problem with available labelled data to validate implementing machine learning to a problem. I think that there's a need out there for a software with a GUI that can accept data and let you try some ML approaches with it and take the coding out of it - and it doesn't seem like it would be insanely difficult to build.

Check out outlier.ai. Also I think there was a company in the most recent YC batch that tackled this problem.

This is spot-on. There's an assumption that if you have large/big data, then AI will be your next big step. That's absolutely not a given. I've been told that AI is just "Statistics at scale", but you wouldn't call simple decision making like A/B testing "AI".

There's a lot of room for sophistication in an A/B test. You might want to do something like Thompson sampling to minimize the number of samples taken from the 'bad' alternatives, or a contextual bandit to take some demographic information into account.

Agreed. It wasn't very insightful in terms of novel methods.

Fantastic resource - thanks, a16z!

Do you accept pull requests? I'd like to make a suggestion for the software and services page: http://aiplaybook.a16z.com/docs/reference/software

Also, I think there's an emerging category outside of MLaaS that focuses on ML/DL deployment (ops).

Here's a quick intro:

Seldon's mission is to help people predict and shape the future with machine learning. Our products increase efficiency and speed up the time to value for data scientists:

Seldon Core - a machine learning platform that helps your data science team deploy models into production. It provides an open-source data science stack that runs within a Kubernetes Cluster. You can use Seldon to deploy machine learning and deep learning models into production on-premise or in the cloud (e.g. GCP, AWS, Azure). Seldon is framework agnostic, it supports models built with TensorFlow, Keras, Vowpal Wabbit, XGBoost, Gensim and any other model-building tool — it even supports models built with commercial tools and services where the model is exportable. Fork Seldon on Github: https://github.com/SeldonIO/seldon-server/

Seldon Deploy - in development - a machine learning deployment platform for data science teams. Features include workflows with approvals and audit trails, advanced experiments, continuous integration and model explanations.

Seldon was part of the Barclays Techstars '16 and we’re working with customers in banking, technology and media sectors.

Disclosure: I’m CEO at Seldon.

Does anyone have a list of VCs that are specializing in investing into AI startups?

I'm looking for a seed and it'd be great to have a list like that (I'm experienced in the field, first started with Deep Learning in 2010, it's on my GitHub...)

There are tons: Data Collective, Array Ventures, Susa Ventures (my fund), Switch Ventures, Amplify Partners, Zetta Venture Partners, AME Cloud Ventures, Ten One Ten, Bloomberg Beta, etc.

That's not an exhaustive list by any means, just some of the first ones that come to mind for me. To be honest, I think the majority of seed funds are interested in AI/ML startups these days. Which one is the best fit for you will depend on the specific flavor of company you're building, your location, your level of traction, et cetera.

If you have any questions, you can email me at: leo at susaventures.com.

Rob May states that he is an active investor in the A.I. space in his excellent AI roundup newsletter: Inside AI (formerly Technically Sentient).

Thank you!

I can't find an use to AI in any of my projects. Am I stupid?

No, of course not. You're just doing work that computers do well.

Crude but subtle in very subtle and insightful way!

I like the twist

Why, thank you.

Judging by contents, this should really be titled "ML playbook", but I guess this doesn't draw that many clicks... :)

Term 'Artificial Intelligence' is vague.

"Deep learning is a name for machine learning techniques using many-layered artificial neural networks. Occasionally people use the term artificial intelligence, but unless you want to sound sci-fi, it is reserved for problems that are currently considered “too hard for machines” - a frontier that keeps moving rapidly." from http://p.migdal.pl/2017/04/30/teaching-deep-learning.html

In particular, two decades ago, letter recognition was considered a holy grail of AI, see:

"Indeed, I once even proposed that the toughest challenge facing AI workers is to answer the question: “What are the letters ‘A’ and ‘I’?" - Douglas R. Hofstadter (1995)

Not it as simple assignment to anyone learning ConvNets (BTW: I really recommend notMNIST as a starting point).

I've seen that Hofstadter quote used in this context before, and always assumed it was being quoted out of context. Do you know where it comes from? I suspect the idea is related to his "Letter Spirit" project - if so approaches like notMNIST don't really speak to it.

Hofstadter often wrote about the difficulty in semantic understanding, and how AI approaches often approached things in such a superficial way that they miss the deeper meaning by design. Many of his projects were aimed at taking on these much harder problems head on. I suspect this quote, while also being a play on words, is aimed at this deeper understanding.

I give some context in the blog post linked above.

This quotation comes from https://web.stanford.edu/group/SHR/4-2/text/hofstadter.html.

And yes, he expected machine to have some 'deep understanding' of what is a letter. My approach (partly in the lines of Dennett, partly my experience with deep learning) is that we overestimate human cognition, which may be more 'a bunch of heuristics' than something 'really deep'. Especially, as some problems can be solved by... pigeons: http://www.sciencemag.org/news/2015/11/pigeons-spot-cancer-w....

In that context, it makes no sense to claim that convolutional NN have solved his problem at all, let alone claim it is a simple assignment.

It is interesting to ask if he was asking the right question or not, but misleading to trivialize it.

Well, it is exactly what I meant this "artificial intelligence" is line that changes. First we consider something some problem too hard for machines. Then, once we solve it - it's "well, but it is just a singular case".

In any case, what do you mean by "the right question"?

For a business guy this is a very useful resource, nice and clear. Can understand some of the criticism but perhaps people are expecting too much.

I specifically liked the Image Feature Analysis tool and the fact this is exposed for others to use on github. Thinking about linking it to my home security camera.

Collect underpants, er portfolio of AI companies

???? (release a self-anointed playbook)

Profit!

I found this quite frustrating. It gives the impression that AI is "plug and play".. well it's not. :)

Deep Learning != AI

Deep Learning is more about pattern matching, not about "understanding" patterns.

Ok, now it's a bubble.

100% agree

Oh man. I wish back the old days where AI would be short for Advanced Industries... ;-)

For people who work with animals, it's Artificial Insemination. I worked with robots and animals and had an intense code-switching problem.

Site is incomplete. Misses out on whole categories of software classified as AI (like Cognitive Computing with approaches like associative memory). Sad.

Reply on news.ycombinator.com