This decision was entirely due to doctrine. STS was designed as a mostly reusable product and failed costs-wise, so the government basically forced NASA to go with something expendable instead. It is genuinely impressive that SpaceX proved that reusable launch vehicles can be made cost-effective, but it's nowhere near the kind of massive quantum leap that SpaceX's legion of hardcore followers seems to be convinced it is. Everything on a Falcon or a Dragon is public domain 60s technology besides the code and guidance hardware (based on COTS components). The same is likely be true of Starship, which explains a lot of the significant teething issues it has suffered over the past several flight tests.