Comment by eksemplar | Hacker News Reader

Comment by eksemplar | Hacker News Reader

Parent Full thread

I’m less skeptical about this than I was about the blockchain bubble, but I’ve yet to see AI or ML actually work on a smaller scale.

It’s a big hype in the public sector in these years, but it’s really all talk. We did a project where we used ML and 1000 server instances in Azure to go through millions of employee cases, to flag cases that didn’t have a certain document. Because the the is new, and this wasn’t something that could be delayed, we also had 10 employs do the same task to make sure it got completed.

The ML project took 2 really expensive employs and 3 months to train the algorithm, then 5 hours to go through all the documents + we had to spend 1 week of 5 employs to clean up the stuff our algorithm had marked as “unreadable” due to terrible scans. The human employs did it in the same amount of time, but made a few more errors.

Over all the ML was more expensive and our politicians won’t favor it again.

On other things where ML might work, our datasets are turning out to be too small, unless we work together with other municipalities and then we’re facing GDPR violations that may be hard to pass through our legal team. Legal is a big issue on a lot of things, it’s not currently legal to automate any sort of process that require any form of validation (as long as it has to do with case working).

That being said, I’m sure stuff like facial recognition will change the world.

As a single batch job that won't be repeated this doesn't sound like a good candidate for ML. ML is more suited to on-going processes.

Why would you use server instances in Azure to do ML? Something like Google CloudML (I'm sure that the other major cloud providers do managed Tensorflow as well I've just never tried it on their platforms) would be a better fit to a project with only two technical staff. Your two staff probably spent a combined total of one-person-month working on infrastructure.

Your issue with small data is very real. People need to stop trying to do ML on small datasets. The results will be sub-optimal.

When you say ML, you must mean deep learning applied to unstructured data (vision, audio).

ML in general can absolutely be used with small datasets. ML is all about finding the right model complexity to fit to the data to maximize out-of-sample performance. If your dataset is small, all that means is that your model will have to be more crude. A simple cross-validated regularized linear regression or a shallow decision tree are ML models too, and you can usefully apply them to a dataset of just 100 samples.

Or instead of hiring an ML employee, building a model for 100 samples I might as well apply my own business intuition.

It depends on the application. You might want to use the data to verify your intuition, which may not be consistent with the data. Or your intuition may have been based on the very same dataset but overfit to it.

I do think that people that think that ML = big data are mistaken. ML is about making the most of your data, however much of it you have.

Then you're mistaking a trendy term "ML" for "hiring an analyst".

ML is ill-defined. But take most textbooks on ML, and you will find regularized linear regression, decision trees, cross-validation and bootstrapping all in there.

In my view, the main difference between ML and plain statistics is that with the latter, you come up with the appropriate model apriori, and then make sure the data satisfies the assumptions so that you can draw conclusions from the in-sample fit of the model. You control for overfitting by choosing the simplest model that is reasonable - often univariate linear regression.

Whereas with ML, you let the data dictate how complex a model you should use. You choose the appropriate model complexity using techniques like cross-validation, and verify the effectiveness of your model empirically.

ML is often used interchangeably with ANNs which I think is a mistake. Take structured data problems on Kaggle and you would very rarely see ANNs as a major predictor in the winning models.

If the number of dimensions is high, building a model by hand could be extremely difficult, and using ML makes sense.

If you have 100 examples and a high number of dimensions you will end up with a very over-fitted model.

Not necessarily. Cross-validation can give you a valid estimate of out-of-sample performance even if you have more dimensions than samples, and even if some of the features you have are (sporadically) perfectly correlated with the target variable. See https://stats.stackexchange.com/questions/295626/does-cross-...

The terms you're using - they come from Machine Learning, which comes from the same division of mathematics as ordinary statistics. And that's how ML is educated at the universities anyway.

So if you've got an analyst sitting on that issue - then your problem is solved anyway.

So again, why hire somebody with a trendy specialty who is <probably> full of BS, when you can hire someone with some respect for classics?

I never said anything about who to hire.

If your model doesn't beat a reasonable benchmark created using some business intuition then this is a bad idea.

Any details as to what ML was used for that you can disclose? OCRing the scanned documents I assume?

>blockchain bubble

There is bitcoin, then there are shitcoins.

Every year someone releases a new altcoin with a youtube video that promises the end of war, starvation, etc... Then suckers buy it. 3 years ago, it was developing 'the ultimate privacy coin', because the libertarians ate that shit up.

Using blockchain outside of currency seems expensive and unnecessary.

I dont see bitcoin bubble popping unless they have a security issue. The rest of the coins might be as good as useless when people realize how expensive it is to run 7 computers to do the job of 1.

Reply on news.ycombinator.com