Hacker News Reader

Comment by web007 | Hacker News Reader

> We can see that the scheme that uses sequential batching actually performs worse than the CPU alone, whereas the new approach using DuHL achieves a 10× speed-up over the CPU.

I had to get down to the graph to realize they're talking about SVM, not deep learning.

This could be pretty cool. Training a SVM has usually been "load ALL the data and go", and sequential implementations are almost non-existent. Even if this was 1x or 0.5x speed and didn't require the entire dataset at once it's a big win.

View on HN

>I had to get down to the graph to realize they're talking about SVM, not deep learning.

there's still a ton of usage for classical learning algorithms. I'd be a very happy camper if we could speed SVMs up by a magnitude

> for classical learning algorithms

Indeed, for relatively "simple" models, SVM can get very, very close to deep learning accuracy for classification, with only a fraction of the computing time needed.

Not to mention 'tweaking' required.

I know two projects (fairly simple implementation) of sequential SVM. I believe vowpal wabbit can also do max-margin optimization.

http://leon.bottou.org/projects/lasvm

http://leon.bottou.org/projects/sgd

Yes, i felt cheated when i read it was about training 1/10th of ImageNet on a SVM. I guess IBM are desparate not to be left behind in the race for distributed deep learning platforms.

To be honest, I'd readily cheer any groups working on traditional machine learning advancements despite all the current hype for neural methods.

I'll second that. For all the attention DL/ANNs get... there's still a lot of legwork going on out there using linear models, basic trees, etc. IIRC this years kaggle survey ranked Logistic Regression as the #1 most used model by a long shot.

neural networks are stacked logistic regressions. a lot of the deep learning research benefits logistic regression

you say that as if it's empty hype. but it's not: deep learning works, and works much better by any reasonable metric than SVMs in most problems that require high to very high model capacity.

Yeah they get you really hyped up first and drop the bomb at the end. Still quite impressive speedup, would be better though to show it on a benchmark where SVM's are used in practice

Looking at the normal stuff coming out of IBM they're not associated with good software in my mind. So the more outrageous their claim, the less I believe them. They need to earn a reputation first.

That's like judging all of Google based on the quality of one product. With 10x as many employees as Google and a very loose organization, expecting any kind of reputation is folly.

A product from an IBM consultant is about as related to a product from IBM Watson as is a product from Microsoft being related to a product from Apple.

IBM has 400k employees and god knows how many subsidiaries and divisions, do you really think you can paint them all with one brush because of some negative experience you had with one of their products ?

Not one, we have multiple IBM software products in my company, and they're all consistently the most terrible software you can imagine making.

Sure they might have some divisions that do better, but I have yet to see them.

Reply on news.ycombinator.com