It's that it's horribly ugly and full of warts.
The biggest issue I have with x86 is not even trying to make things orthogonal (until x86_64, which does a tolerable job cleaning up a lot of the mess).
Some of it could probably be improved simply by ditching the syntax, and renaming stuff and pretending a lot of the rest simply doesn't exist, but you'd be left with something that looks very different from a typical x86 assembly listing..
But some stuff you don't avoid unless you go 64 bit: The registers alone gives me gives me hives, starting with the naming convention, where the M68k for example is straight forward: D0 through D7 = data registers. A0- A7 = address registers. For a lot of things they can even be used entirely interchangeably.
No addressing parts of the registers with special names, or mumbo jumbo restrictions on what can be done with different registers (that was excusable when the 6502 was a viable competitor, but even that followed clear and simple rules (Arithmetic on the accumulator (A), indexing with the X and Y registers) and didn't layer complexity on top of complexity. Bank switching / segments? Nope, we don't clutter the CPU with junk like that. Either you use a memory address or IO lines (or, as it happens, on the 6510 in the C64, IO lines mapped to a memory address).
Maybe I'm just still sore the M68k lost the performance race and I "had to" get a PC.