Chips themselves aren't actually very regular in terms of their layout -- not all tracks are connected the same way, parts are partitioned across different clocking areas, some parts of the chip contain things like memory resources while others do not, etc. You don't actually have a perfectly symmetrical die. FPGAs for example are quite "heterogenous" in the device resources they contain (clocks, DSPs, memory, registers) and they are scattered in many places over the chip. (Note that FPGA resources are fixed while ASIC layouts are not, but both of them use simulated annealing/analytic placement algorithms in the design phase for automatically laying out digital logic among the chip. Both of these problems, while complex, do ultimately aim to solve the optimization problem of minimizing wire-length -- among many other timing/clocking constraints.)

> Does this merely optimize for minimal length of the datalines?

That's how it used to be 25 years ago. It's probably still a factor in the cost function, but the biggest part is timing. Of course, timing and length are closely related.

> It seems counter-intuitive that a random placement would be superior to a more regular one.

It's not necessarily superior if you want to have optimal timing for all paths between all cells. But you don't need that: only a few percent of all paths are actually timing critical. Those determine the maximum clock speed. The others have enough slack such that it doesn't matter that they are placed tens of microns too far.

Random placement (at an ever lower level of detail) is also better to avoid crosstalk. If you'd place a bunch of driver cells in a nicely aligned stack and make those wires go a nicely aligned stack of receiving flip-flops with parallel wires in between, you'd get the mother of all crosstalk problems.