Comment by jaypaulynice | Hacker News Reader

Comment by jaypaulynice | Hacker News Reader

Parent Full thread

I worked at a startup where we used machine learning to review legal documents.

When you have 10M+ documents, it takes a long time for a team of lawyers to review them. So a company with AI/ML is at a huge advantage in identifying the documents a lawyer needs to win a case.

To get accurate results, you need a lot of data, but to crunch that data you need a lot of power, not just speed. A lot of the processing is done now with GPUs (thousands of cores)...traditional computers are CPU intensive and have limited number of cores. Think for example a multi-threaded application running on GPUs vs CPUs...the GPU can divide the problem in more tasks and the processing can be done more accurately not just faster.

http://www.nvidia.com/object/what-is-gpu-computing.html

PS: I think people overuse "Artificial intelligence" we're no where close...but it excites people compared to "machine learning" or "deep learning" or "convolutional network" or just statistics...It's just trying to find an equation that fits the data like linear regression.

I agree with your sentiment: to analyze these amounts of documents you need computers. And I understand the importance of parallelism, GPU computing can be way faster than CPUs for certain tasks.

In a sense, and correct me if I'm wrong, I get the feeling that the whole ML is just starting now because we have faster computers. To analyze such amounts of text and make some sense of it, you'd need a massive "database" or "fitted model" to decipher what the legalese means. Only recently has this been possible to create (thanks to Moore's law). It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.

> It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.

That's absolutely correct. But to pooh pooh it on those grounds is like pooh-poohing the fourier transform around the 70s (which theory had been around for 200 years) and only had then "become practical" with the advent of transistor-based computer. Now the FT and relatives are used in everything from audio processing and compression, image filters, statistical analysis, pretty much every form of scientific spectroscopy, MRIs, etc. If you had generally made investments in "FT-related technologies" you'd be doing well.

Only if you consider techniques like Word2Vec[1] as a non-breakthough.

No one knew you could do it, but yes, it build upon previous work. I was working in the field before and after and if anyone had asked me if one could represent all human languages in only 300 dimensions, and have vector composition be meaningful I'd have laughed at them.

Take using back-propagation to train deep neural networks. People had shown it worked in 1 or 2 layer networks, but despite years of work no one had been able to train anything deep enough to be useful. Then Krizhevsky, Sutskever and Hinton won ImageNet[2], proved it was possible and kicked off this whole ML craziness.

Neither of them is exactly because of more powerful computers, nor magical math breakthroughs. It was more lots of hard work by researchers trying many combinations of techniques until something worked.

These techniques, combined with huge volumes of data and - yes - more powerful computers are what have made the difference.

[1] https://papers.nips.cc/paper/5021-distributed-representation...

[2] https://papers.nips.cc/paper/4824-imagenet-classification-wi...

Reply on news.ycombinator.com