US Visas bounce around between security services (Homeland Security, FBI, DoD/NSA), immigration, and State/NVC. These other departments can flag any visa for additional scrutiny.
When excessive delays like this occur it is often a security flag, because then it has to go back to the origin agency and even the department within that originally created the flag to decide if it is a valid match to this individual (and or the facts are correct).
Keep in mind that the other departments handling this, it isn't their primary responsibility. They're dealing daily with e.g. terrorist threats, drug trafficking, or similar threats/violations. Nobody is interested in prioritizing State's visa processing (of which they receive many).
Plus the people or department who created the original match could no longer exist/work there, and now the request is bouncing between different departments with nobody wanting to take responsibility for it.
PS - Just to be clear I'm not defending this. It is a giant mess and hugely dysfunctional. But to understand why something can get stuck in "administrative processing" for years, you have to realize how deep this rabbit hole goes.