As a general principle, the Space Shuttle program was massively capital constrained, so many trade-offs were made that compromised safety and operating expenses. I suppose the latter helped in limiting the number of flights, but then again half the fleet and their crews were lost.
Perhaps start with what killed two shuttles and their crews? Besides no provisions for escape unlike all other "capsule" based craft, solid rockets are _dangerous_ because you can't turn them off, whereas liquid fuel rockets will effectively turn themselves off in many disaster scenarios.
So for a get me out of here tower escape system a solid rocket makes a lot of sense, see also their use in ejection seats. But mounting some number of them (more than one unless used alone because you need symmetrical thrust) on the sides of your system means they must work perfectly every time.
The reentry heat shield protections which along with the wings etc. came from a requirement the Air Force says they never really wanted or needed, a single polar orbit mission returning to the launch location which thus required a lot of maneuvering in the atmosphere because the Earth turned while it was up there resulted in the infamously fragile tiles and some more refractory stuff for hotter locations (and eventually a bunch of tiles in a cooler location were replaced by something else).
Rear end heat shields are protected during the launch of a capsule, it's all hanging out there with the Shuttle. Change the foam and larger parts of it may break off, hit, and fatally compromise a part of the insulation system.
Now going from personal worry points and speculation:
The Space Shuttle Main Engines (SSMEs) are for institutional reasons extremely high performance and required a rebuild after every mission, so the SLS "throw away half a billion worth of engines every flight" would need to subtract that labor and parts.
That said, they were fired up and the SRBs not lit until it was clear the SSMEs were running fine, that resulted in three legit aborts and two from faulty sensors. Two failures after launch, split between both causes, noting catastrophic or mission endangering. But, still, you have all their complexity and SRBs.
Landing ... well, that's another thing that's more complex or at least different although landing in the sea also has its issues: https://en.wikipedia.org/wiki/Mercury-Redstone_4
It's a hell of a lot more complicated aerodynamic body/system than a cylindrical rocket and capsule system so that work was harder and more error prone. IBM Federal Systems modified a lot of code between the first and second launches, and caught an introduced error that would have released just one SRB. There's a 1984 CACM issue devoted to all this https://dl.acm.org/toc/cacm/1984/27/9 and is now free to read online or download, I highly recommend it. A lot of programmers outside of that world were concerned the code wouldn't be up to snuff, what you'll read in those articles explains how they pulled it off.
And, again, no escape systems. Fly enough times and things go wrong, the Shuttle compounded this by massively increasing complexity for very little gain.