So, you are right that visual programming doesn't represent how a computer software works.. But it's exactly how computer hardware works.
There are physical and spatial constraints in efficient implementation on FPGA's. Although I don't use them, I saw plenty of examples in my research of people looking at them visually and changing representations to make them more compact or something. Modifying the physical layout outperformed whatever the initial synthesis was.
Yes, visual tools for manipulating physical aspects of the design are used, but there is little to no abstraction there like a language provides.
In fact, my sentiment extends to hardware design, which is why I've been toying (and building real stuff for work) with SKiDL lately, a Python extension that allows for text based schematic capture for printed circuit board design.
Of course, the actual routing of the traces still must be done by hand. However, during schematic capture, instead of drawing the same Zener diode input protection 100 times with a mouse, I just call the code that builds it in a loop.