Hacker News Reader

Comment by brap | Hacker News Reader

I feel like “giving up understanding” is inevitable.

There’s some hubris in thinking we can understand everything. For truly difficult problems, it’s entirely possible that humans are simply incapable of comprehending why a solution is true. But ultimately the practical value of applying that solution to the real world is going to eclipse our need to understand it.

Math is just the beginning. I see it happening in other fields too, like physics and biology. Many of us software devs have already given up on understanding parts of our own systems for the exact same reason.

Seems like a losing battle.

View on HN

> For truly difficult problems, it’s entirely possible that humans are simply incapable of comprehending why a solution is true. But ultimately the practical value of applying that solution to the real world is going to eclipse our need to understand it.

This is hardly new or novel, you’ve just described huge chunks of existing engineering disciplines. There is still no complete model that explains why airplane wings work, there are lots of very useful models, but all of them have fundamental flaws where their behaviour completely breaks down in certain very possible scenarios. Notably scenarios where airplane wings don’t spontaneously either stop working or explode.

We also built the entire commercial airliner system decades before we even had computers capable of doing the aerodynamic analysis of airplanes, but none of that prevented us from creating huge engineering disciplines around the empirical data we did have, and slowly chip away at the underlying theories as maths and computing improved over time.

So we’ve always lived in a world where we extract value from systems we fundamentally don’t understand. But that’s never stopped us from working to understand them anyway, and deriving even further value from that improved understanding.

You and the other commenters are all absolutely right, but that’s not really what I was pointing at.

So far our solutions were more or less understood thru some models of reality that we’ve constructed (on our own), which may or may not reflect reality perfectly, and even if most of us never bothered thinking about these models, some people did and they understood them on a very deep level.

But we may be getting to a point where the problems we need to tackle become too difficult for humans to model, or even to notice their existence, like asking an ant how a Boeing 747 works.

Maybe this was always the case but now it seems like we might have a shot at making these solutions useful even if we have no idea what they’re even solving.

That's learned helplessness as a lifestyle.

You observe others behaving poorly and immediately succumb to defeatism, assuming there was no better way.

When presuming, all that matters was the short term economic profit, you simply use the wrong gauge.

Here, long term human survival is at stake, regressing to zoo animals isn't a sensible option.

I think that is the central issue. Is long-term survival actually at stake?

Do you think every Mathematician has a deep understanding of building architecture, engineering, physics, economy, etc., everything necessary to gauge whether something like a 1 terrawat nuclear plant is completely safe in all aspects?

Of course not, so then you have to select appropriate ones. And then you need a process to ingress and egress reviews. Oh what about change? Does that come back to the Math council too?

Oh, we also need global cooperation to pull this off. And then do you trust the people, the selection process for the council (all avenues for corruption are there).

I don't have a solution of my own, so it's not fair for me to criticize the author in this way. I share his concerns.

And it's depressing to think of all the negative scenarios, and be so concerned all the time.

These issues are important and worth thinking about, regardless. Who should teachers teach?

You deal with this every time you wake up, or visit a specialist of any kind. You likely didn't make your own clothes, you didn't grow your own food, dam up any rivers for your water supply, build any power plants to power up your phone/computer, nor did you spend any of the deep amount of focus required to build up the knowledge and technology required to provide us these 'things' in life. The only difference is, you didn't pay much attention to these facts because they became mundane. Take any of them away for any decent amount of time, and suddenly you become acutely aware of what it is you have.

The author mentioned what the issue is: a lot of people are experiencing what it's like to 'catch up' to the more intelligent ideas. Everyone can 'get' the intelligent ideas given enough time. But how much time do you have? And he's concerned someone will rush and create some kind of world-ending solution because they're too trusting of a technology they don't understand.

His only solution is some kind of throttle on the advancement of technology, and arguably knowledge (whose?).

In the future I feel one parallel to this is my research area, SAT solving.

People used to solve logic problems by hand, verify logic. Now you pile it into a computer. The problem has been around for a while, no-one fully ‘understands’ the proof of the four colour theorem as a big chunk is computer proved.

We are now just changing (admittedly greatly) what we can put in a box marked ‘checked by computer.

> I feel like “giving up understanding” is inevitable

People didn't stop understanding how to do addition or subtraction when the calculator came out. That's a simple example, but if the AI is superhuman, I don't see any reason why it cannot break things down into simple concepts. I don't think anything is truly beyond comprehension; it just needs to be explained properly (by someone, or someTHING that really understands it) and, for complex ideas, time taken to understand them.

> given up on understanding parts of our own systems for the exact same reason.

Maybe this will be a sign of when ASI is achieved. AI researchers keep saying they don't fully understand how LLMs work - maybe when ASI arrives, it can explain that fully.

> That's a simple example, but if the AI is superhuman, I don't see any reason why it cannot break things down into simple concepts. I don't think anything is truly beyond comprehension; it just needs to be explained properly

You should try being a teaching assistant or a tutor. "Everyone can be taught everything, it just depends on the teacher" can only be said by someone who never tried this in practice. Just as not everyone can learn advanced math, the best mathematicians also have their own limits and there are things they won't be able to understand or comfortably navigate. Not just because it's a lot of material, but because it's complex inherently.

Indeed. How many scientists understand how a compiler works? Or know about branch prediction on the CPU? If they do, do they lose sleep because it's non-deterministic? I don't. Seems fine.

Reply on news.ycombinator.com