Evidence of exponential speed-up in the solution of hard optimization problems? | Hacker News Reader

Evidence of exponential speed-up in the solution of hard optimization problems?

arxiv.org

141 pointsgodelmachine38 comments

Quoting from the abstract:

  > ... a non-combinatorial approach
  > to hard optimization problems that
  > ... finds better approximations than
  > the current state-of-the-art.

So it appears that they're finding approximate solutions, so already it's rather less significant than the title suggests.

Then:

  > We show empirical evidence that
  > our solver scales linearly with
  > the size of the problem, ...

We know that for most NP-Complete problems, most instances are easy. The question then is whether they are testing their algorithms on instances that are known to be hard. There's a chance they're doing something like finding factors of random integers, which we know is easy in almost every instance.

I'm deeply pessimistic.

While I agree with you, you should note that even approximation within any degree of error is NP-complete for a large class of NP-complete problems (e.g. TSP, and the problem MAX-EkSAT used in the paper).

That is, a polynomial algorithm for the approximate problem would be just as significant as one for the exact version.

But that's being NP-Complete, and so can be converted into exact solutions for other NP-Complete problems. Otherwise, by definition, it's not NP-Complete.

So it's not clear what they're actually doing, but if they can solve NPC problems they would say that. So I expect that they are getting approximate solutions that are not then NPC.

That's correct. Given a polynomial time approximate algorithm for Ek-SAT (note, by approximate algorithm I mean along the lines of the formal definition of approximate algorithm, that the solution given by the algorithm falls in some fraction of the real answer for all instances, see https://en.wikipedia.org/wiki/Hardness_of_approximation), you would show P=NP.

My first concerns, namely the usage of analog methods to solve NP-complete problems, lies along the same lines as this post: https://www.scottaaronson.com/blog/?p=2212

Moreover, from what I can see, and as you mention as your original concern, the extent of proof for the 'exponential-speedup' claim exists in the form of some benchmarks, hardly a proof that their method actually works on all instances, which would be needed to show that it is a approximation algorithm as commonly defined.

The purpose of my post was to highlight that for some problems (for example, TSP), even a really crappy approximation algorithm would imply P=NP. As highlighted by the authors of this paper, MAX-EkSAT is in a similar situation, though judging by https://www.cse.buffalo.edu/~hungngo/classes/2006/594/notes/..., unlike TSP, approximation to SOME ratio is possible, though there exists aratio >1 which cannot be beat unless P=NP.

I was simply trying to address the statement: "So it appears that they're finding approximate solutions, so already it's rather less significant than the title suggests. ", since an actual approximation algorithm for some ratio really WOULD be significant (showing P=NP).

I don't think your first statement is true, there exist polynomial time approximation schemes and approximation algorithms for np complete problems, notably a PTAS for knapsack. In other words, an approximation of one np complete problems doesn't imply an approximation of all of them.

I can't fully articulate the reasons for this though.

The reason for this apparent discrepancy is found in the difference between strong and weak NP-completeness.

The fully polynomial-time approximation scheme (FPTAS) for the knapsack problem only runs in so-called pseudo-polynomial time:

https://en.wikipedia.org/wiki/Pseudo-polynomial_time

This means that the runtime is polynomial in the numeric value of the knapsack. Since the encoding of that numeric value only takes logarithmic space (unless you are using unary encoding), the runtime is in fact again exponential in the size of the input.

For this reason, the knapsack problem is called weakly NP-complete:

https://en.wikipedia.org/wiki/Weak_NP-completeness

One can show that, unless P=NP, a so-called strongly NP-hard optimization problem with polynomially bounded objective function cannot have a fully polynomial-time approximation scheme:

https://en.wikipedia.org/wiki/Polynomial-time_approximation_...

SAT, Hamiltonian circuit etc. are strongly NP-complete:

https://en.wikipedia.org/wiki/Strong_NP-completeness

Thus, an FPTAS for these problems would indeed imply P=NP.

While what you're saying is technically true, you have to be careful with the exact definitions. For many NPC problems, approximations are indeed hard as well, but that a worst case statement. The existence of an algorithm that finds sufficiently good solutions for all instances implies P=NP, but that doesn't mean that there aren't algorithms that find very good approximations for all instances that occur in practice, or for almost all instances that you sample uniformly at random. As long as there is an arbitrarily obscure class of instances left where the algorithm fails to provide a good approximation P!=NP remains a possibility. Experimental evaluations of approximation ratios are not sufficient to claim P=NP.

This is a consequence of the PCP theorem, right?

https://en.wikipedia.org/wiki/MAX-3SAT#Theorem_1_(inapproxim...

No. Many approximation problems are known to be NP-hard without relying on the PCP theorem. The PCP theorem is "only" one of the strongest (if not the strongest) and widely applicable result about approximation hardness. (I may be underselling it with this short summary because of the unique games conjecture; in any case, the main point is that there were approximation problems known to be hard long before the PCP theorem was found.)

>TSP

Travelling salesman problem? Can't you get to within a factor of 2 of optimal by constructing a minimum spanning tree:

Once you have a MST (which can be built efficiently), the total weight of the tree is a lower bound for the total distance of a TSP solution. However, you can construct a TSP path that traverses each edge of the MST twice; which means that we know that this path (which is easy to find) is at most twice the total cost of the optimal path.

Your solution works only in metric spaces. In non-metric spaces (distances are arbitrary and you are forced to return a cycle, not a tour that repeats vertices) no constant-factor approximation is possible.

> So it appears that they're finding approximate solutions, so already it's rather less significant than the title suggests

The title doesn't suggest anything about exact solutions as far as I can tell.

> We know that for most NP-Complete problems, most instances are easy. The question then is whether they are testing their algorithms on instances that are known to be hard. There's a chance they're doing something like finding factors of random integers, which we know is easy in almost every instance.

I don't know how they choose the problems they choose to solve but I assume this information is in the paper? If you are skeptical then why not read it and report your findings? Simply expressing your skepticism doesn't seem valuable.

> The title doesn't suggest anything about exact solutions as far as I can tell.

In the jargon of mathematical optimization, there's a difference between "solution" and "approximation". The paper's title says "solution". I would expect the title to say something like "2-approximation" or "(1+ϵ)-approximation" instead.

I was only skimming the article, but it seems they were comparing against benchmarks form this contest:

http://maxsat.ia.udl.cat/introduction/

they did experiments on the 2016 Max-SAT challenge, which seems to me like a legitimate class of problem instances.

Yes, and this does indeed mean that their results may be of practical importance.

At the same time, it's pretty well known that the kind of problem instances arising in practice and in those challenges are not hard instances. In other words, this also means that their result is extremely unlikely to be relevant for the P=?NP question.

This keeps popping up every few months for several years, and seems like BS. Here's an example:

https://news.ycombinator.com/item?id=8652475

And here's Scott Aaronson's debunking:

https://www.scottaaronson.com/blog/?p=2212

If these guys could really solve NP complete problems, they should have some amazing concrete results to show at this point, which they don't.

You link to research from 3 years ago. Does the latest paper (Oct 2017) not represent the concrete results you claim they don't have?

They claim to solve np complete problems, which this paper is not demonstrating.

The authors have built a start-up based on these ideas:

http://memcpu.com/

They provide their SAT solver as a service that you can try.

A related paper I recommend in this context is NP-complete Problems and Physical Reality by Scott Aaronson:

https://www.scottaaronson.com/papers/npcomplete.pdf

It's worth noting that Scott Aaronson has commented on previous work by the same authors: https://www.scottaaronson.com/blog/?p=2212 . At the time, he was convinced that their approach wouldn't scale. I'm not sure about their current approach, but at a glance it seems similar to the previous one.

Previous work: https://arxiv.org/pdf/1411.4798.pdf

Do I have to register for the SAT solver?

Their web page says: "We are currently in Alpha Test. Users can submit problems via our SaaS portal in Conjunctive Normal Form (CNF), which ultimately represents the normal form of Boolean propositions for the problem’s variables and constraints"

I have no problem providing real, hard SAT instances in CNF. It should be really easy to verify whether they have something good, unless they have a size restriction.

Is this similar to what Berkeley MPC Lab is doing with quadratic programming using electricity? I.e. that nature has means to solve optimization problems in an instant and our approximations of this process in the form of differential equations allow us to do something similar, albeit not 100% accurately?

Would you kindly provide link to the Berkeley MPC Lab work?

Here you go:

http://www.mpc.berkeley.edu/research/analog-optimization

Thanks :)

Can someone provide a technical tldr. I'm afraid this is over my head, but I'm extremely curious as I understand it's importance.

My initial instinct is not very important, but we will wait and see.

There have been many, many examples of people achieving significant improvements on many classes of NP-complete problems, and more come out every year, of course the more general your improvement the better. Modern SAT solvers with learning are such an improvement.

The claim of "exponential" seems very dodgy to me, they are running over a fixed set of benchmarks, it's hard to measure an exponential improvement over such a set.

I will wait until I see this peer reviewed, after a brief skim read I am a little worried about how big the circuits they are making are.

EDIT: they mostly compare against 3 specially crafted classes of random problems. Noone really cares about random problems, and it looks to me like they made random problems their system would be particularly good at dealing with. That sets off alarm bells for me.

Authors seem not know about this work: https://pdfs.semanticscholar.org/ff7b/3a7b1dad73797ff7c79ac2...

One of the claims in the fine paper that started the discussion is that there's a need to perform several flips at once to find better solution. The paper I cite does something very similar - it walks along chained variables postponing flips until energy lowers for sure.

They also claim their solver has O(vars) time complexity for many problems, including ones with high density (clause/variable ratio). But nothing revolutionary.

This is really important if not bogus. Any comments from people in the field?

It's like back-propagation for digital logic.

"Huge if true"

Did you look at the paper or just throwing the analogy out there? It seems the described problems are integer programming which wouldn’t have gradients as in back propagation.

The magic seems to lie in the so called self organizing logic gates detailed

https://arxiv.org/abs/1512.05064

Right. See the PDF of the paper at [1]. They're mapping Boolean logic into a kind of analog system.

"SOLGs can use any terminal simultaneously as input or output, i.e., signals can go in and out at the same time at any terminal resulting in a superposition of input and output signals ... The gate changes dynamically the outgoing components of the signals depending on the incoming components according to some rules aimed at satisfying the logic relations of the gate. ... A SOLG ... can have either stable configurations ... or unstable ... configurations. In the former case, the configuration of the signals at the terminals satisfies the logic relation required ... and the signals would then remain constant in time. Conversely, if the signals at a given time do not satisfy the logic relation, we have the unstable configuration: the SOLG drives the outgoing components of the signal to finally obtain a stable configuration."

This is sort of like supervised training of a neural net. I think. Test cases with both inputs and desired outputs are needed, and applying them to the network pushes the parameters towards values that yield the desired outputs. It's kind of like deliberately over-training a neural net to encode some explicit function.

The paper is vague about how you train this thing. It seems like it has to be driven by test cases, but they don't say much about that. It's not clear that this scales. It ought to work for small numbers of gates, but for large numbers, does this process converge?

[1] https://arxiv.org/pdf/1512.05064.pdf

Looking at the authors papers it doesn't look worth reading.

What do you mean? How does that help you to decide if a paper is worth reading or not?

IF the authors have a history of poor work, or no history at all, or a bunch of stuff that has been discredited, then the likelihood that they have solved a big problem declines.

People who do massively important work typically telegraph quality by doing earlier good work. That isn't always true, but it is commonly true.

You could say that estimating a paper's worth by the quality of the authors' previous papers is an approximate solution to the NP-hard "paper quality assessment" problem, which offers an exponential speed-up. :P

This looks too good to be true ... am I missing something?

Reply on news.ycombinator.com