may, anyone explain what is this
Some thing is "pareto optimal" when there isn't another thing that's AT LEAST AS GOOD in ALL measures, and BETTER in at least one way. For example, if we say there are no ties (for simplicity), then the cheapest language model is pareto optimal; the fastest model is pareto optimal; those which score highest on each benchmark are pareto optimal; and so on.
Tradeoffs can also be pareto optimal: for example, if the cheapest model is also slow, then there will be more pareto optimal models which are "cheapest for their speed"; and so on for other tradeoffs (e.g. fastest that achieves a certain benchmark score; cheapest model with open weights; etc.).
If you're making a decision about which thing to choose, you only need to care about those in the pareto front (since, by definition, anything that's not pareto optimal is objectively worse on at least one measure).
Pareto optimality does not compare one measure against another: something that's 10000x slower can still be pareto optimal, if it's 1% cheaper than the alternatives. To pick a "best" thing, you could give a weight/importance to each measure, and combine them into an overall score: but that's subjective, and might vary between people and tasks. In contrast, focusing on the pareto front is a way to ignore those things that will never be the best, regardless of weighting.
Mapping the cost of something (like an algorithm), and the time it takes (so lower is better for both). 1, 3 and 5 are all optimal in their own sense. No one is strictly better than the other, just different tradeoffs you have to choose yourself. However, you would never choose 2, because for a lower cost you could get the same result choosing 3. Same with 4, 6 and 7, they all have something that's both faster and at the same time just as cheap you could choose.
A pareto front is a bit like the classical "fast, cheap, good, choose 2". There are always tradeoffs, but if something is both slow, expensive and not better than something that's faster and cheaper, it's a bad choice, and thus not on the "pareto front".
Say a race vehicle has acceleration, top speed as defining parameters. Some are slow but accelerate hard, others need a long time to reach very high top speeds. Others are in between, or just flat out bad at both.
The pareto frontier is the set of vehicles that are best: pick one from the frontier and you can be sure that for it's given top speed, none accelerate faster. And vice versa, pick one with a given acceletation and you are sure none have a better top speed
It’s really that simple.
Eschew obfuscation.
The Pareto points are where you sacrifice the least of anything to get the most of everything.
There's the saying about buying computers. Good, Cheap, Fast, pick any two. That's where you would prioritise.
If someone makes something that better, cheaper, and faster, or even pretty close to the best on two of those and clearly better on the other. It's a Pareto point.
Over time computers are getting better, cheaper and faster (software notwithstanding). The leading edge of that advance of all of the things is the Pareto front.
> a Pareto front represents the set of solutions where no solution outperforms any other solution in the set at every objective
I do not believe you are correct when you say
> something that better, cheaper, and faster, or even pretty close to the best on two of those and clearly better on the other. It's a Pareto point.
Since that would outperform on every objective
GP's point that it's prioritisation does not seem incorrect to me. Prioritisation involves considering trade-offs of various approaches and deciding which aspects & attributes to optimise for, at the expense of others.
We choose our items/workflows/technologies/whatever, so we get the best/most efficient/most effective/whatever, across the widest possible set.
Sounds like prioritizing, to me, but I’m just a dumb hick, so I suppose I can be wrong.
Going for the Pareto is when you elect not to prioritise. It is explicitly deciding to not choose one property over another ant to keep everything as much as you can.
Getting to that point can be calculated (in some cases), but I suspect most folks get there by trial and error. Finding out what is effective, and what is not, and choosing what is effective, over what is not, until there's no longer a choice. That often becomes tribal knowledge, and is handed down. There's always someone trying to improve it, and when they figure it out, that gets added to the tribal knowledge. Basically, that's how nature does it, so there's some serious prior art. Natural Selection is brutal prioritization.
In Morocco, they used to announce the end of the Ramadan fast, by holding up a black thread and a white thread, and waiting until they could not tell the difference.
Then, they'd fire a cannon, and everybody would dig into some awesome soup. Sort of the same thing.