Everyone lives in fear of something with more wrath than a god. Liability.
Because of this they are not willing to do anything even slightly outside of code, and any decision that needs making is a phone call to the engineer rather than making the call on the job.
This slows everyone down as nobody is keen to actually be the one with responsibility. The engineeer who feels the same way also make sure that everything is overspecced to all hell to make extra cirtain that they won’t be liable for anything failing.
You can't imagine it because those bridges and cathedrals collapsed (and were probably rebuilt).
I think people will have no idea what the AI evaluates but they will make myths about it and adhere to those (which effectively triggers Goodhart's law). IMO a big chunk of SEO and content marketing is already based on myths around PageRank/YouTube/…