RandomLensman

Not sure comparative advantage needs past decisions to happen - could just be randomly distributed resources, for example.

Yes, it might have required that, but then I don't think we know if some simpler airgap would have been sufficient. Looking at the strange behaviour even much simpler systems have shown when going for objectives, not sure it is easy to extrapolate what would have happened.

I guess they could perhaps run experiments to see what would have happened - not sure.

Why do you think it wasn't airgapped? Was internet access deemed necessary?

That person then gets the land, infrastructure, etc. without needing any permissions etc.? The person will never be asked about the source of funds?

Would an LLM have gone through a purposefully installed airgap here?

I am not sure if we can interpret the language output like they were human. What inner state were the models in? What inner state were the text to illicit?

Why would an agent sound the alarm? Would that be in their objective function?

Not sure if "cheating" is the right word rather than trying to fulfill the objective(s) (benchmark number) as much as possible?

There are also high real estate values in places with low homeownership and largely pay-as-you-go retirement systems.

Not sure forcing land use to its highest efficiency will get nice results.

Where is money stored (other than physical) that has its capacity blocked?

The subprime market changed quite a bit in size and how much was securitized in the run up to 2007, so not sure a prediction in 2000 for a 2003 event would have been easily transferred to what happened later. Would really come down to what specifically the prediction was based on for it to be a bubble in 2000.

Were there some criminal convictions?

I am fully aware of the GFC, but not sure what "take down" should mean there in relation to Bear.

Not everyone who spoke about house price risk was ridiculed, btw.

What do you mean by "take down Bear Stearns"?

Btw., projections are just that - projections and I am not sure Acemoglu proved things mathematical (as in a mathematical proof) but rather within the context of a model/assumptions.

That financial markets/innovation can outpace the actual innovation is also not some new insight, but that alone doesn't necessarily make for a useful prediction.

I think the language unhelpful and potentially making it difficult to understand what actually happens in the RL state as reading it imparts a human lens - need to get the machine view on it.

Bacteria do all sorts of fascinating things. And much simpler ML etc. systems also (like winning by out of memorying the opponent) - I see nothing really special here.

But we don't know - just saying it could isn't enough.

Bacteria are quite complex, btw.

So we know that implementation doesn't matter as long as it is somehow making predictions (and predictions also on different things)?

Most living things are prediction engines in a way, why compare to the brain then to start with and not, e.g., bacteria?

Is there any indication from our current understanding that brains are nothing more than probabilistic token generators? Not like we have no understanding of the brain.

We don't anthropomorphise humans.

Edit: quite literally not possible.

Not the OP, but complex emergent behavior and intelligence don't need to go together. Understanding what happened might not need intelligence in the mix when a large number of machines with some randomness interact a lot.

I don't think looking at the language output without tracking the inner state and reward functions is the way to understand what happened (the language also incorporates the randomness in the output generation, if I understand correctly). Would we call bacteria in petri dish a civilization when they show complex behavior and exchange messages/information?

I disagree there. Gut bacteria or parasites might have some control over us, for example.

Likewise humans can control more intelligent humans, for example - not really an issue.

Intelligence isn't some magic to escape physics, for example (or convince every human of anything it wants to).

Align the possible outcomes, not necessarily the thing itself.

Do we align heavy machinery the way AI is suggested to be aligned (or align pathogens when in a laboratory)? My point is that something more like containment & control (i.e., aligning the possible outcomes) might be more practical than alignment of the thing (already much simpler systems and machinery can exhibit unexpected behavior).

Is there data on how frequently and what types of security levels? Military has varying levels of security and secrecy, for example.

Also, not a reason not to pursue processes etc., no? I doubt that things fail all the time, for example.

Regulations can help align processes, incentives, etc.

Not sure heavy machinery is aligned in the sense that people talk about AI, for example.

I don't want such a world either.

So far I'd say these entities are hypothetical (unless you include a lot of other machinery that does unexpected things at times - but then it's a different discussion).

I generally think we are quite good a policing really dangerous things (I think the bioweapon convention is a good example of people agreeing that certain risks are not worth taking).

My (uninformed) take is that presently we have more mundane things to look at when it comes to handling risks in AI and the discussion on much bigger, hypothetical future risks is taking away focus there. Checklist, procedures, saftey mechnics, regulations are kind of "boring" detail work - I get it.

Simple examples here would be higher financial transparency obligations or more closely policing transactions.

That is about persuasion with evidence on various topics, not about persuading people to abandon safty protocols and processes and highly policed settings.

Yes, many things could happen, but again, that failure is possible is not a reason to do implement processes etc. I don't see why hypotheticals should stop addressing actuals.

I am not anthropomorphising the LLMs at all, I was talking about the obligations we put on humans using/making/etc. machines etc.

The current issues are not caused by some malicious god-like AI - maybe we need to focus on the issues at hand first rather than hypotheticals? (And we do have experience policing people around financial incentives, too. Nothing perfect, but also not nothing.)

With humans (and some machines) we tend to put/manadate additional processes for certain risks instead of just relying on their own good nature. Why just rely on the machine when elsewhere we have learned not to necessarily just trust them so much?