Hacker News Reader

Comment by medo-bear | Hacker News Reader

the question was about "objective that works well in practice for any task". thats pretty global in my books

If you take "any task" to mean literally any conceivable task, then sure.

If you take "any task" to mean "any practical task" or "any task a human would conceivably want to have done", then no free lunch doesn't apply.

Another way of looking at karpierz's comment is through the incompressibility of pure noise at scale.

As soon as some infinitely generated sort of noise is from some subset of possible noise, there is indeed (AFAIK) some kind of an ideal estimator that appropriately compresses that noise source with no bias and less entropy that the full space of possible noise.

I hope this shines an additional alternative light on the topic.

I deeply suspect that this has more than surface similarities with the observation that you cannot extract work from a uniform heat bath, but you can extract work from a temperature gradient. There's probably not much an intelligence could learn to predict about its future environment living inside the core of the sun, even if it could withstand those temperatures -- just randomness everywhere in every direction. But on Earth, with the sun shining in and the coolness of space behind us, there are a lot of causal relationships that can be compressed to make useful predictions.

The flow of entropy is a marvelous, strange, bizarre, and beautiful thing.

The "in practice" thing is pretty important.

In the no-free-lunch version of the world memory is infinite, all mathematical operations are the same performance wise and things the size of your floating point numbers are ignored.

But "in practice" these things matter a lot.

For practical purposes the no-free-lunch theorem is a distraction, or at best something to encourage people to try alternate approaches.

A bumpy landscape indeedy.

Reply on news.ycombinator.com