The first is simply that it's cheaper to have an open office. Modern office buildings tend to have large spaces -- nearly whole floors -- that are open, but can be subdivided. But subdivision costs money. Paying carpenters to put up walls isn't cheap, and even cube systems can be surprisingly expensive. Plus managers are accountable for costs. So they choose the cheap option.
The second is that open office plans are good for communication and collaboration, though less good for individual silent work. Managers are essentially all about communication and collaboration, which is why they love these open offices. Unfortunately while programmers need to both communicate and work alone, they need to communicate and collaborate much less than managers do, which is why office plans that are selected by managers make things difficult for the programmers.