Hacker News Reader

Comment by inlineint | Hacker News Reader

On the other hand SVM doesn't scale as well as neural networks do because it has computational complexity between O(n^2) and O(n^3) [1] where n is the number of samples in the training set. So if you plan to add more data later you may eventually encounter scaling problems with SVM.

[1] http://scikit-learn.org/stable/modules/svm.html#complexity

View on HN

We found great success with the LIBLINEAR SVM implementation [1] though: Extremely good performance, to the point that it affects scalability too, with predictive performance acceptably close to libSVM with the RBF kernel, for a large cheminformatics dataset:

Paper (open acccess): http://dx.doi.org/10.1186/s13321-016-0151-5

As can be seen in fig 5 [2] in the paper, a dataset size that took ~1 week with libSVM (actually, the parallel piSVM implementation) on 64 cores, took less than a minute with LIBLINEAR, which runs on just one core.

[1] https://www.csie.ntu.edu.tw/~cjlin/liblinear

[2] http://jcheminf.springeropen.com/articles/10.1186/s13321-016...

With an RBF Kernel, you are stuck solving the dual problem which has O(n^2 * m) complexity. With the linear kernel, you can solve the primal scales with O(n*m) complexity.

Good to know, I did not know that! I kind of wish scikit had some sort of CUDA capabilities to speed things up.

Scikit uses numpy which uses BLAS which can be implemented with nvBLAS[1]. I don't know what it takes to get this setup and how much performance boost it gives for SVMs though.

[1] https://developer.nvidia.com/cublas

Reply on news.ycombinator.com