The OASI fund was at ~$ 2.4 trillion in around 2025. That's only about 1.5 years of payments given no contributions. The net interest from the fund is only around $ 60 billion. If you add in the DI fund, it only changes by about 10%. The $ 1.7 T cost of Social Security in 2026 will not be meaningfully affected by these funds. Mandatory spending payouts will thus have to come from contributions (which are fundamentally just another tax that could be redirected to other priorities) and from the general fund. The contributions brought in around $ 1.3 T in 2026, leaving somewhere around $300 B to be paid out of the funds.
Thus, I think "Social Security is the biggest driver of spending" is a perfectly cogent argument. The mandatory social security contribution is just another kind of income tax.
Obviously we can't just turn off Social Security, for many political, moral, and practical reasons, but any solution to the deficit or debt that ignores social security will be fighting with one hand tied behind its back.