This is my big fear about pervasive use of AI. I'm afraid that companies, policy makers, regulators, etc. all start letting AI make important decisions without any human understanding of the reasoning behind the decision and the human puppets hiding behind it with no accountability.

I imagine scenarios where AI could be given complete authority to decide who is hired/fired, who gets medical care, who gets food, who gets utilities (water/electricity/natural gas) to their homes, who gets disaster relief, etc. Quite frightening when you think about it. If AI decided to cancel you (and it had this level of authority) your very existence would be in danger.