We've got a bunch of computers that mostly don't make mistakes at the hardware layer. On top of that, we can write any programs we want. Even though the halting problem exists, and is true for arbitrary programs, we know how to prove all sorts of useful security properties over restricted sets of of programs.
Any software security pitch that starts with "when the software starts acting outside of its spec, we have the system ..." is nonsense. In practice, "acting outside its spec" is functionally equivalent to "suffers a security breach".
Ideally, you'd use an operating system that has frequent updates that expand functionality, that is regularly audited for security problems, and that only rarely needs to ship a security patch. OpenBSD comes to mind.
If software has frequent security updates over a long period of time, that implies that the authors of the system will continue to repeat the mistakes that led to the vulnerabilities in the first place.