But maybe the landscape of airline booking/pricing has changed since I got this impression.
This problem is largely due to the closed and archaic nature of the GDS systems that are the source of truth. These systems have been running for 40+ years and no one wants to touch them to make substantive changes. As a result, modernisation is attempted via layers on top of layers that just add complexity confusion.
There is a difficult routing and fare calculation problem here, coupled with a distributed consensus problem. While better solutions to this exist or may exist, the biggest hurdles are the organisational ones.
Which will most likely show non-bookable outdated prices more often... there's always a trade-off. There might also be some browser/geo profiling going on.
Why do you assert that?
However, I'm not sure they really changed the engine underneath, and it was always designed as something that then was input to booking sites, often with filters on fares, because QPX (the engine) would show you fares that an agent with full view of GDSes would see - and sometimes the airlines don't want to sell certain fares (it already filtered the special unsellable ones of course, but it couldn't apply arbitrary policy decisions from airlines).