Of course, the GOT referred to by the PLT is itself is a bit of a security vulnerability, because it's usually a writable chunk of memory with a ton of frequently-called function pointers; corrupting it gives you relatively easy control over program flow. The new mitigation is "full RelRO", which resolves all GOT entries during binary load, then marks the GOT segment read-only. This gains in security, but trades off binary load time, and it is still going to be slower at runtime than the JIT. Full RelRO is usually off by default because of this added overhead.
Modifying the code segment to include direct function references (either during binary load or later, during runtime) is also not a great approach because it means being unable to share the code segment between processes, which would increase memory pressure. We could do what Windows does, which is to have static randomization (randomization of binary addresses is performed once per boot, not once per process), enabling code to be "statically relocated" lazily per boot, but this carries its own complexities, and opens you up to different security vulnerabilities.
There's no magic bullet here, unfortunately. There are just tradeoffs everywhere in many directions - between runtime speed, startup time, security and memory usage.