Long before any use of LLMs, OsmAnd would direct you, if you were driving past Palo Alto, to take a congested offramp to the onramp that faced it across the street. There is no earthly reason to do that; just staying on the freeway is faster and safer.
So it's not obvious to me that patently crazy directions must come from watching people's behavior. Something else is going on.