Hacker News Reader

Comment by oh-kumudo | Hacker News Reader

I mean de-noising techniques, nowadays can be definitely claimed as Machine Learning, even though they have long existed prior to even ML as a concept even being invented...

View on HN

I think the defining characteristic if machine learning system is that they get better as they experience more (see more examples). The question then the ask is two fold (a) do de-nosing techniques improve when they see more samples? (b) do the application of ML techniques applied in this context mathematically equivalent?

That is only the case for online learning systems. Offline learning is definitely machine learning.

You can "de-noise" all you want and you're still gonna get crap compared to what it would be without the noise. It's like the 3d RF equivalent of looking for ripples made by water striders in the middle of a jet ski race.

Seems like it depends on the noise. If it didn't work for removing atmospheric distortion, they wouldn't be building bigger telescopes.

Many nowadays ML (and even "IA")-dubbed techniques are in fact borrowed from good ole operational research.

Indeed, core memory was invented by Jay Forester for is operational research in city planning!

For example?

PCA for example. It is essentially SVD. And it is quite funny though, I believe that there is no learning in PCA(maybe depends on implementation, but the popular one I believe used heuristics), it is still categoried as ML techniques all the time.

PCA can "learn" in the sense that we can apply it to more and more data, and that can affect the directions of the components. For example, let's say we take measurements one at a time and apply PCA to the whole dataset after each measurement; if the first 100 measurements follow some distibution A, then PCA will give the components of A; if the next 1000 measurements follow some different distribution B, then PCA will 'learn' to give the components of B a higher priority than those of A; i.e. it adapts to the data.

In that hypothetical setup, we end up storing every raw datapoint. If we think of these as parameters, then PCA is a non-parametric model; although it's maximally inefficient in terms of information/compression.

The learning aspect could be how many principal components to use ?

That is a hyper parameter to the model. It is not learned.

Reply on news.ycombinator.com