We probably will never know what exactly went wrong, but I think the current best guess is this: Intel set minimum metal pitch target for its 10 nm process at 36 nm, while TSMC set minimum metal pitch target for its 7 nm process (equivalent to Intel 10 nm process) at 40 nm. These processes were the last before EUV, pushing DUV to its limit. It turned out that you can do 40 nm but not 36 nm. That's it.
It was an extremely technical issue, with justifications on both sides, and Intel's decision wasn't obviously mistaken, even in retrospect. 36 nm was usual scaling to Moore's law. It was 40 nm that was unusual, TSMC underscaled metal relative to fin and gate. Why 40 nm? Because it was the limit of DUV double patterning. But Intel also knew that! That's why Intel went with quadruple patterning. It was a conservative choice to underscale metal to the limit of double patterning, but quadruple patterning wasn't crazy either. Everyone was using quadruple patterning for fin at this point, so it wasn't thought that risky.
Should Intel have reconsidered this once they encountered yield issues? Yes, but that's hindsight. All processes experience yield issues. Intel probably thought they can be solved, faster than going back to drawing board and doing everything again. Unfortunately they couldn't.
(Above analysis was informed by lots of reading on this topic I did over the years, but https://semiwiki.com/semiconductor-manufacturers/intel/7433-... in particular.)
Intel made a calculated bet, lost, and furthermore made a handful of other bad business decisions. We focus a lot on the latter part when in reality it's unlikely the "business" decisions were the primary reasons for failure... it very well may have been technical problems led by engineers in the first place.
I don't think Intel acted irrationally.
Last, this is the journey of almost all industries. The titans become too complacent, set for disruption by other smaller companies / outsiders. This is just the natural course of business due to inertia. Intel is hardly alone here.
EDIT: All that being said, the right man for the job is in the CEO chair. I personally believe he always should've had the role. We'll see how Gelsinger turns it around. I am optimistic.
Engineers should run engineering companies? Yes, absolutely. The CEO is not the only position that "runs" the company - there should at the very least be a CTO that has equal political clout in terms of long term decision making.
Honestly, it's probably not so much that engineers should run companies, but that MBA/finance-types shouldn't (mainly because it seems their conventional wisdom has collected a lot of toxic ideas).