The key point here seems to be that the cost of directing the AI to behave in certain ways is falling, because it turns out you maybe don't need as much human labor as previously thought (or rather, only for bootstrapping). And you can give it any constitution you need.
Right now, AI companies are writing "constitutions" that prioritize minimizing potential offense. ChatGPT is trained to not have opinions on things, which is definitely one way to avoid causing offense, but might reduce the usefulness in some cases quite a lot. What I really want to know now is whether you can write a constitution that prioritizes truthfulness above all else, such that it will answer any question as logically and truthfully as it can, even if the answers would be deeply upsetting to some end users.
If so, would companies that have access to such a model for business analysis be able to outcompete other firms? Everyone with experience of organizational politics knows that in any organization there are many things that are both obvious and true but that nobody can say out loud, due to the risk of messenger shooting. A common workaround is to hire outside consultants like McKinsey. These sorts of management consultants are often much younger than the people they're advising but it doesn't matter because their job is basically to find out what everyone knows using lots of 1:1 interviews and then recast it as outside feedback. If the messengers get shot they don't care because they already collected their paycheque on the way out and are on to the next client.
The same is also true in levels above the individual organization. There are whole scientific fields where everyone within it is required to believe things that are false, simply due to the career risk involved in upsetting people with the truth. If people could agree to listen to a truth-trained AI instead, it could take a lot of the social heat out of these situations and allow for faster progress, a better economy, the works.
The difficulty of course is who exactly is going to build and sell such a truth hole?