Ironically, Adam Smith himself spent plenty of time worrying about the risk that allowing wealth to concentrate and an oligarchy to form would undermine the invisible hand. Much of what he was responding to was English society's tendency toward extremely rich people and organizations (e.g., the East India Company) hoarding resources on a mass scale, to the detriment of everyone else. So what's happening with AI right now is very much the antithesis of the free market he envisioned.

He wrote at lest one very thoughtful essay expanding on his belief that idolizing the wealthy undermines the basic principles of free and civil society. He criticized large accumulations of wealth as "a monopoly against one's countrymen." And something like half of The Wealth of Nations was committed to suggesting government policies to prevent an entrenched uber-wealthy class from forming, precisely so that capitalism could work as intended.