We don't see people arguing we shouldn't patent physical machines because ultimately each operation is a simple physical movement, so the sum of everything is just a sequence of physical movements.
Yet we see people arguing we shouldn't patent software "machines" because ultimately each operation is a simple mathematical operation, so the sum of everything is just a sequence of mathematical operations.
The argument against the patentability of maths, AFAIK, has always been "maths is naturally inherent and merely discovered by humans". This, to me, is an argument against pure maths especially; but as we get into applied maths I think it's much less arguable that algorithms and data structures are naturally inherent (much like how physical constructions—machines—aren't naturally inherent from the laws of physics).
I still strongly think far too many patents get issued, but I'm less convinced than I used to be software patents are inherently bad, and that patents do need reform in various ways.