rasbt

Good question! I am using a mix of the latest Python open source frameworks, PyTorch taking the center stage. There will also units on Lightning to make certain things more convenient, like multi-GPU training and hyperparameter sweeps.

love how fast it is

Looking forward to all the new PyTorch-related improvements and features that this new focus will bring to Lightning!

I agree with you, it could surely expand this answer (I am the person who wrote this little answer). Since this was originally just an answer to a question I answered via email (if I remember correctly), I didn't go into too much depth regarding ConvNets, because I just wanted to answer this question "briefly" in this mail. But if there's a demand for that (if it's useful to others) I may end up writing a tutorial on ConvNets one day :). However, there's already an excellent one out there, I highly recommend Dumoulin & Visin's "A guide to convolution arithmetic for deep learning" at https://arxiv.org/abs/1603.07285

you mean material specific to deep learning that is more general and less math heavy? Hm, that's a good question, the resources I'd reference are all a bit math heavy. However, don't be afraid of diving into TensorFlow, it's really a nice library that takes care of all the tedious, mathematical details. E.g., in contrast/addition to NumPy (leaving out the comp. efficiency part out of the discussion for now), it already implements several optimzation algorithms, so you wouldn't have to worry about implementing backpropagation from scratch or so. Sure, it still requires a bit of linear algebra, but it's really more straight-forward than it seems at first glance :). Maybe, you'd be interested in Keras (http://keras.io); it's a wrapper around Theano and TensorFlow which provides a really intuitive interface for building neural nets! Haha, btw. I really enjoyed Joel Gruus, post ;)

Hi @vonklaus, thanks for the feedback. This was actually not intended to be an "article" but more like an answer to a targeted question (I am the author of this little write-up). Basically, someone asked me this specific question some time ago (I think via email), and I answered it with this person's background in mind. Then, I generalized it a bit more and added it to the FAQ section in the GitHub repo in hope that it is also helpful to others. It's really more like a quick overview, idea, explanation in contrast to a fully fleshed-out blog article :)

Technically, you can say that even a single perceptron unit is a artificial neural network -- a single-layer ANN. Multi-layer neural networks (such as a multi-layer perceptron) have at least one input layer, one hidden layer, and one output layer.

Yes, I think you are right. Out of curiosity, when I browsed over Coursera's course catalog, most data science related material seems to be taught in Matlab or R (however, there are also others, e.g., Klein's Linear Algebra class in Python). Personally, I think that instructors shouldn't enforce a language requirement. I believe for big platforms such as coursera it shouldn't be to hard to run an respective interpreter to check the code/answer uploaded by a student.

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer (https://github.com/dib-lab/khmer)

Thanks for the insights! Also here, this comment would make an interesting addition to a "Feedback" section at the end of the article to give people a broader view on this topic. May I have your permission to post your comment below the article?

Just want to leave a comment about the "Matrix" object. I remember the first day I got started with NumPy. Coming from MATLAB, I really felt tempted to use it. However, I also saw an recommendation (can't remember if I read this in the NumPy documentation or in an online forum) that ndarrays are more widely supported. I never really bothered about `Matrix` then, and honestly, I have never ever seen anyone using it. Maybe you will find some examples on GitHub if you really search for it, but it is practically non-existend.

Nice collection, let me add one more item to this list:

Seaborn: statistical data visualization: http://stanford.edu/~mwaskom/software/seaborn/

Interesting! It's hard for a single person to tell, because you can only start one way digging into machine learning. However, from a teacher's perspective, this is a useful observation. I also started with Matlab since it was the language that was used in my classes. However, I think this is also a little bit context dependent: For someone who has never programmed before, Matlab may be more intuitive. I think in an ideal world, you should let your students choose what language they want to use to solve the problem :).

I went through the very same process :). I really like your comment, you highlight something that I forgot to mention in this clarity "It makes it easier for me to collaborate in many different ways: share tools with other teams, transfer ownership of my code, get help when I need". Would you mind if I add it as to a "other people's experiences" section at the bottom of the article?

Thanks for the comment (I am the author of this article).

> I have a feeling the author will find their way to Julia-land eventually, in a couple of years or so.

I have a strong feeling that this will eventually happen :). In an ideal, less busy, world, I would love to use Julia alongside to explore and battle-test it further. Or even develop useful packages, libraries, and functions for it. The truth is, I am currently lacking the time to do that :(. I mean, Python works for me, and I am currently more into the scientific problem solving so that I don't have the time :(. When I say that Python works for me I mean that I am currently happy since it can do everything for me I need, however, this doesn't mean that Julia couldn't do certain things better ;).

Anyways, I really like your comment. I am wondering if you would be okay with it if I include it in a "Other people's experiences and opinions" section at the bottom. I think this would be extremely helpful for people who are new to the "data science field" -- my article is strongly biased towards Python as you noticed :P

I wholeheartedly recommend [scikit-learn](https://github.com/scikit-learn/scikit-learn) - the best organized and cleanest code I've seen so far. It is really organized and well thought-through.

I posted this article some while ago, and I thought it may not be a bad idea to archive it on arXiv. What do you think? The (Latex) PDF version would look like this: http://sebastianraschka.com/PDFs/articles/naive_bayes_1.pdf I just signed up on arXiv and see that the catch is that I'd need 2 recommendations for the categories Computer Science -> Learning or Statistics -> Machine Learning. Would anyone here who would want to give me a recommendation? I would really appreciate it!

In practice, I would say that it is a very challenging task to determine whether variables are totally independent or not: It is more about the strength of independence. Every time something is "estimated", you'd introduce an additional source of error, and in cases of weak independence the violation may be negligible. Modeling the conditional dependences is basically the idea of Bayesian Belief Networks, and you might find the paper "Comparing Bayesian Network Classifiers" interesting in this context (includes comparisons to Naive Bayes) http://link.springer.com/article/10.1023/A:1007465528199

Thanks for the comment, and I agree! I rephrased it a little bit to

"Empirical comparisons provide evidence that the multinomial model tends to outperform the multi-variate Bernoulli model if the vocabulary size is relatively large [13]. However, the performance of machine learning algorithms is highly dependent on the appropriate choice of features. In the case of naive Bayes classifiers and text classification, large differences in performance can be attributed to the choices of stop word removal, stemming, and token-length [14]. In practice, it is recommended that the choice between a multi-variate Bernoulli or multinomial model for text classification should precede comparative studies including different combinations of feature extraction and selection steps."

Sounds cool. I think the most crucial part is really the data (or feature) set. The choice of classifiers makes a difference too, but having good data (salient, invariant, discriminatory) makes the largest difference.

One think about naive Bayes and kNN: Although the performance may be similar, I would consider Naive beneficial if the data set is large. kNN requires to re-evaluate the data. Naive Bayes makes the classification computationally cheaper and with "on-line" learning, you can still improve or adjust it over time

I like this simple and intuitive explanation. Hm, maybe a short mentioning of the "a priori" knowledge would be also worthwhile!

Thanks a lot, I always called it "smoothening" for whatever reason, but it should be fixed now :)

Yes, I see Linear Discriminant Analysis more as useful tool for pre-processing and dimensionality reduction rather than using it as classifier

It's an interesting article with some interesting thoughts. However, unit tests are - for me - a crucial framework and also a main pillar in my project planning process: If unit tests are written before the actual code, it is immensely useful to define goals in terms of what you want the function - that you are going to write - should actually do and not do.

And especially throwing away passing unit tests is a waste, not the unit test itself. Eventually you, or someone else, wants to make edits to existing code. Unit tests are extremely useful to check if everything "still works"

I recently started to use pandas a lot for my data analyses. However, I didn't get to write about it yet. But it will surely follow one day :)

I went through all the resources listed in the previous submission here: http://electronicsforu.com/electronicsforu/circuitarchives/v.... However, people had legit complaints about popups and broken links. So, I went through all resources manually, double checked links, and added more information about the resources.

Right. Yes, I am aware of the threading module. It depends on the type of application what to prefer: memory-bound vs CPU-bound tasks