Not slowly insane, but one that turns the AI into a cruel god, and is more about the humans it torments, check out “I Have No Mouth and I Must Scream”.
After reading the short story, I recommend checking out the Adventure game. It's written by the novelist directly, and is a deeper examination for the characters and the AI through the device of virtual worlds.
This is the best bet if the OP is looking for an existential thriller.