Of course the port layout is going to be a fiasco. It's just not physically possible any other way, unless you start shoving everything into a single USB-C port and force the user into using adapters and docks, at which point you'd just be removing actual ports like laptop makers have been doing for the past years.
Obviously port layout is harder than just "just move it" but aside from the physical board foot print, mounting hole locations, and GPIO, there doesn't seem to be some holy thou-shalt-not directive that ports can be moved around, considering the USB and Ethernet ports have flip flopped a few times, the ports change shape, get removed, etc. So, move the most commonly used ports next to the required ones, and put them all on one side of the board so this can actually be neatly slotted underneath a monitor or on a shelf or rack mounted without having to buy a bunch of right angle adapters, PoE hats and switches, or a $50 Argon One that moves the power and display ports to a more convenient location on the back of the Pi by means of a PCB.