"Mathematical proofs are only applicable where their assumptions are valid. So why would we believe that the where memory isn't infinite (everywhere in practice) that Turing's halting analysis is applicable?"
Because adding the assumption in that memory is finite doesn't make the analysis easier, it makes it harder. There's a reason the abstract mathematical model assumes infinite memory; it isn't because the abstract mathematical model can assume into existence, so, you know, why not, it's to simplify it. Now in addition to all the other things the mathematical proof has to worry about, it has to be constructed in such a way that if at any point memory is exceeded, it also "halts". That is way easier said than done.
As a homework exercise, try to take the halting problem proof and update it to include the possibility that either the halting-detection machine or the machine it is simulating (within the boundaries of the halting-detection machine's resources!) runs out of memory. Pro-tip: If you think you cracked it in a couple of minutes, you did not. For instance, now rather than just waving at the fact the halting-detection machine is running a Universal Turing machine that simply has some "linear" factor of expansion in memory usage and/or runtime and not worrying since you have infinite amounts of both, now your proof is going to be intimately concerned with the details of exactly how the universal turing machine is encoded. What you end up with at the end won't be some neat claim about how the halting problem is impossible, you're going to end up with some rather complicated contingent claim with terms that will include specific details about the size of the encoding and the amount of memory given to the detection machine.
If you do this work honestly and thoroughly, you'll quickly learn that adding finiteness of resource doesn't make the problem more tractable. Rather than a cute little statement about the halting problem, you'll end up with an incredibly complicated statement with dozens of terms in it about what the detection machine can and can not do, and the resulting mathematical statement will be much harder to use.
"This completes in O(2^N) time where N is the size of the memory."
In other words, our computers are finite state machines, which is basically true, so attack them with finite state machine tools. The problem is that even if your "finite state machine" you're modeling is, say, a Commodore 64, and even if you ignore external input and just simulate the machine itself, you have an intractable problem on your hands. The problem grows exponentially as you add each individual bit. Modern computers have a lot more bits than a Commodore 64. This turns out not to be a useful approach.
If you're going to go this route, what you're looking for are sub-Turing programming languages [1] that allow you to write programs in a reduced-power environment that allows more useful assertions to be made. If you are legitimately interested in this, there's a lot of research occurring on the fun fringes of programming language research. But... I can tell you that they are certainly far from a magic wand either. At a minimum, it requires a huge mindset change, and it is very difficult to fit even some common problems/algorithms into the requisite constraints. And "completes in a finite period of time", even if you can prove it, can be a rather vague promise to make.
[1]: Example from a quick google: http://ainfosec.github.io/crema/