RSA is more clearly maths (which should be unpatentable) than a UI patent like the scroll bounce which is defined by an imperative if this then do that process and directly affects the world state (by causing light to be projected into it).

I think for software patents do more harm than good but it is incredibly hard and dangerous to try to carve out special rules to keep the few good ones (even if we could agree on which those are).

There shouldn't be any patents for algorithms - algorithms are math, and math is not patentable.