Blindsight by Peter Watts.

Not exactly answering questions on what AIs would do but: to spoil as little as possible: there is an alien ship that communicates with humanity in a way that is eerily parallel and similar to today's LLMs.

It's a pretty tough read though, I had hard time understanding what is happening exactly. Might be one of the books where it might be a good idea to spoil a bit how the narrative works. Maybe just reading to top tagline on TvTropes page doesn't spoil too much: https://tvtropes.org/pmwiki/pmwiki.php/Literature/Blindsight

Also you can read it online easily: https://www.rifters.com/real/Blindsight.htm

Edit: this novel apparently has been on HN frontpage once https://news.ycombinator.com/item?id=18378221