I would argue that "abundance and growing the pie for everyone" is even more unfathomable given how things are structured currently. The wealth gap will continue to widen until something gives.

Well then given that one side is "the situation remains neutral or very slightly improves" and the other side is "unthinkable atrocities", I think it's only rational to focus on the "unthinkable atrocities" part. Ideally, we should be focusing all our energy into making sure that doesn't happen.

Why do you think that the fact that the alternative is unthinkable is a reason it won't happen?

Are you also sure that it is unthinkable to those running these companies? I wouldn't be surprised if these models end up being used for internal security-- that people would try to keep an extremely unequal society stable by surveillance and massive analysis capabilities. I think it's apparent that some use of this sort already occurs and that these companies are already participating.

Closing your eyes doesn't make the danger go away.

What? This makes no sense. You don't like the alternative so you choose to believe in the best outcome?