How many internet forum prophecy cults (you know like the q one) are or will be powered by these language models? It's often assumed or at least easier to imagine the evaluator in Turing's test is a rational actor that possesses a high-degree of skepticism. But it seems that a lot of the human population is ready and willing to believe wild claims with little or no evidence and many people seek out information that confirms what they already believe.
As the cost of making such models becomes less and less, it seems inevitable, spin up many such models and see what sticks and/or combine some evolutionary process for feeding back user-engagement to fine-tune and adapt the models. How many of these influence machines will latch onto the language of existing religious traditions and how many might invent or spur on the development of entirely new ones? Maybe not exactly the "Age of Spiritual Machines" that some futurists predicted...
How far are we from "Show HN: I started a cult by training a model on the sermons of televangelists and MLM copy."