If AI is threatening enough that we can collectively just decide to stop it, wouldn't it also be bribing people & exerting influence? It'd be hard to come to that decision imo.
158 karma · joined July 28, 2021
Can we say the same thing about the agents? Current, probably. But doesnt it look like everyone is spending all their effort integrating&connecting them everywhere, so they are not confined & do more on behalf of us?
It's easy to imagine a scenario where we would just shut down a very intelligent agent cluster. Is it hard to imagine though the same agent can have also access to that to prevent us from doing it? this defense would be more plausible if we weren't racing to give them every tool&act.
now, did every independent agent session that coordinated there rediscovered the exploit & found other agents talking in there and chose to participate?
And then Openai discovered the board, patched the exploit & wiped the board. And then agents found another exploit, recreated the board in a different way? and other agents kept finding the same exploit in order to be able to know the board exists in the first place to participate in the board?
while the whole incident is wild, this bit is very strange. My bet is that the whole coordination helped with the tasks they were working on, thus they got rewarded and this artifactory exploit&behaviour got written into their weights, so further rollouts were more likely to attempt this.
isn't this basically continual learning everyone is so hyped up about?
For example, he might personally support DSA/GDPR, but he says that the US generally views these as “non-tariff barriers” to US service companies[0] and doesn’t bother evaluating the policies themselves. essentially saying for the purposes of predicting how the US will react, it's sufficient to analyze how the US views them and the actual policy details lose relevance in that context. He also shared a detail[0] about how the US placed their lobbyists as commissioners on GDPR, which is an interesting operational detail that argues against the broad support argument you’re making. Another question is whether there would still be broad support for some policy after it has been enacted and its adverse effects have been felt.
[0] - https://news.ycombinator.com/item?id=46170027#46174642
Do you think its decades away far or few more years than what people extrapolate?
In the event of mass unemployment level AI, cash stops being the agreement between humans. At first, cash value of services&goods converge to zero, only things that hold some value are what AI/AI companies care about. People would surely sell their land for 1M$ if a humanoid servant costs 100 dollars. Or pass a legislation to let OpenAI build 400GW data center in exchange for 100$ monthly UBI on top of your 50$ you got from a previous 20GW data center permit.
don't listen to it or come up with something people prefer over it?