It just never got widespread adoption because it's just not interesting, and LLMs are no different here. The dialogue is still empty, despite being deeper and more grammatically complex than previous attempts.
If every farmer in an RPG hands out the same "collect 20 bear asses" quest it doesn't matter if they all have "detailed" randomly generated backstories and can opine about the game world, real world philosophy, or the 2024 US elections.