Yes, compiling PHP shouldn't take that much (even though typical WordPress site consists of thousands of PHP files). But as you can see from some of the charts from my work, it can potentially double the amount of concurrent requests the server is able to respond to.
Whole page caching (I call it page cache) is the biggest improvement one can make. However, instead of using WP plugins, in my work I configured Nginx to cache the output of PHP interpreter (FastCGI). The advantages are that you can use Nginx's location directives to bypass the cache early (e.g. logged in users, shopping cart, etc) and that Nginx stores the pages in RAM in a tree-like structure with pretty fast access times. In a heavy WP site (plenty of large plugins, etc), you can go from 200 concurrent requests at 8 seconds to 1000 concurrent requests at 2 seconds, which is a pretty huge improvement.
MySQL query caching should definitely help, though in my testing it did not. Maybe I need a real benchmark with a real 100 people requesting different subpages. Then the DB (MariaDB in my case) might become a bottleneck.
Yeah, SSDs are superb-useful, in any cases. I made all the tests on a non-SSD server. At least the WordPress DB structure is quite flexible, though, as you mentioned, it now suffers from the gigantic joins.
Bad plugins and themes are the root of all the performance problems. But sometimes it is easier to implement caching and replacing Apache with Nginx on the server side than replacing plugins that you really need. That's the main point of the thesis: Time of an experienced developer fixing plugins and themes is more expensive than optimizing the server. But yeah, if you are a developer, learning to write clean and fast plugins is crucial.