Hacker News Reader

What the hell does "wiping out all value in the universe" mean... it's such a weird phrasing, like "A series of critical missteps by Alameda wiped out all the value of FTT and therefore FTX"

View on HN

It's a reference to an AI apocalypse scenario, like a paper clip maximiser for example.

https://en.wikipedia.org/wiki/Instrumental_convergence

Is it though? That article talks about AI having a value mismatch that results in it pursuing an undesirable (to humans) outcome.

Not about wiping out all value in the universe. To the paperclip maximizer, if anything, it's generating the maximal-value universe.

Would "wiping out all human value" remove your objection?

Start with the human condition, and work your way up to value from there. What IS value, anyway?

What is Zvi Mowshowitz's utility function. That's probably what the author is talking about when he talks of value.

Pretty hard to say for sure, after all humans are impenetrable black boxes... :P I'd hazard a guess it involves perpetuating genes, though.

Reply on news.ycombinator.com