I actually think a big part of it is something like a fixed perspective on what the process of computer programming has to be like that comes from an era before it was possible to do such a thing well.
One of the skills a programmer always implicitly learns is to imagine the data their code is operating on—and only to occasionally peek into it with e.g. printlns and debuggers. There's something about the seeming necessity of that that keeps people away from tackling the problem directly. It may be a fish not being able to see the water kinda thing. (At least that's my read on it :)