Hacker News Reader

Comment by zahlman | Hacker News Reader

If we can't understand the designs, how rigorously can we really test them for correctness?

Our human designs strive to work in many environmental conditions. Many early AI designs, if iterated in the real world, would incorporate local physical conditions into their circuits. For example, that fluorescent lamp or fan I'm picking up(from the AI/evolutionary design algorithm's perspective) has great EM waves that could serve as a reliable clock source, eliminating the need for my own. Thus if you move things it would break.

I am sure there are analogous problems in the digital simulation domain. Without thorough oversight and testing through multiple power cycles, it's difficult to predict how well the circuit will function, and how incorporating feedback into the program will affect its direction, if not careful, causing the aforementioned strange problems.

Although the article mentions corrections to the designs, what may be truly needed is more constraints. The better we define these constraints, the more likely correctness will emerge on its own.

> Our human designs strive to work in many environmental conditions. Many early AI designs, if iterated in the real world, would incorporate local physical conditions into their circuits. For example, that fluorescent lamp or fan I'm picking up(from the AI/evolutionary design algorithm's perspective) has great EM waves that could serve as a reliable clock source, eliminating the need for my own. Thus if you move things it would break.

This problem may have a relatively simple fix: have two FPGAs – from different manufacturing lots, maybe even different models or brands – each in a different physical location, maybe even on different continents. If the AI or evolutionary algorithm has to evolve something that works on both FPGAs, it will naturally avoid purely local stuff which works on one and not the other, and produce a much more general solution.

This is similar to why increasing the batch size during LLM training results in better performance: you force the optimizer to generalize to a larger set.

And then you change temperatre/elevation/move it next to a router and it falls apart, because after all there is going to be something correlated.

Great, so use ten. Use a hundred. Spread them around. Put one on the ISS.

The problems just have to be uncorrelated.

> Put one on the ISS.

I can see it already: cloud provider offers orbital FPGAs for testing your AI hardware designs

Ask the same "AI" to create a machine readable proof of correctness. Or even better - start from an inefficient but known to be working system, and only let the "AI" apply correctness-preserving transformations.

I don’t think it’s that easy. I’m sure Intel, AMD and Apple have a very sophisticated suite of “known working systems” that they use to test their new chips, and they still build in bugs that security researchers find 5 years later. It’s impossible to test and verify such complex designs fully.

Think about it in the context of LLMs, their internals and the way that we test them; we do our best at the time.

Evolution seems to work at producing "designs" and there's no understanding there at all.

Especially true if the computer design creates a highly coupled device that could be process sensitive.

Results?

Can you always test the entire input space? Only for a few applications.

I am really curious about how you test software...

Literally no piece of software is bug-free. Not one. What are you talking about? Of course it’s impossible to test all inputs, because there’s going to be inputs that you can’t even convince of at the time of designing. What if your application suddenly runs at 1000000x the intended speed because hardware improves so much? How do you test for that?

Hardware doesn’t change over time…

Yes it does. It ages. But even if it doesn't, my point still stands. Or are you insinuating that the engineers over at Intel, AMD and Apple don't know what they're doing, because clearly their CPUs aren't flawless and still have bugs, like Spectre/Meltdown.

It deteriorates, it doesn't change. The functionality is still there and no modern hardware deteriorates to a failing state before it gets obsolete. Yes, I am insinuating that the engineers at intel, AMD, apple and nvidia are incentivized to prioritize expedient solutions over developing more robust architectures, as evidenced by vulnerabilities like Spectre and Meltdown.

print("No Bugs!")

Depending on the language, this simple code actually has a bug:

https://blog.sunfishcode.online/bugs-in-hello-world/

It's a little different in software. If I'm writing a varint decoder and find that it works for the smallest and largest 65k inputs, it's exceedingly unlikely that I'll have written a bug that somehow affects only some middling number of loop iterations yet somehow handles those already tested transitions between loop iteration counts just fine.

For a system you completely don't understand, especially when the prior work on such systems suggests a propensity for extremely hairy bugs, spot-checking the edge cases doesn't suffice.

And, IMO, bugs are usually much worse the lower down in the stack they appear. A bug in the UI layer of some webapp has an impact and time to fix in proportion to that bug and only that bug. Issues in your database driver are insidious, resulting in an unstable system that's hard to understand and potentially resulting in countless hours fixing or working around that bug (if you ever find it). Bugs in the raw silicon that, e.g., only affect 1 pair of 32-bit inputs (in, say, addition) are even worse. They'll be hit in the real world eventually, and they're not going to be easy to handle, but it's simultaneously not usually practical to sweep a 64-bit input space (certainly not for every chip, if the bug is from analog mistakes in the chip's EM properties).

following classic TDD, use novel "AI" to write millions of test cases.

(forgive me, my fellow HNers...)

Were we ever doing that though?

Reply on news.ycombinator.com