I think the Turing tape reflects Some Great Mathematical Truth®, but not any more than any other equivalent paradigm like lambda calculus. In truth, lambda calculus is much better and more elegant, but it would be much harder to compile down to target architectures directly unlike this Turing-tape-based solution. Most hardware implements at least one register that can index some large array of cells (RAM) and an accumulator; this is very easy to target using my architecture. I tried to come up with a model that could represent how we do computation on hardware well.
The "turing machine" is an abstract mathematical concept, perhaps better stated as "a means of generating sequences of whole numbers". It isnt a device, nor does it have devices "connected".
Insofar as CS studies turing machines, it's simply doing discrete mathematics. The job of programmers is engineering: building devices whose behaviour is useful to their users.
No, not at all.
> or is it a cultural preference based on Turing's key historical role?
A little bit, but the main reason is just that Turing machines are a convenient language for introducing complexity theory, which is usually the first and often the only area of CS theory encountered by undergrads. Other areas are fit for other tools. Computability theory typically works with mu-recursive functions, programming language theory with lambda calculi, computer architecture with various sorts of register machines - Turing machines have a central place in CS but not the central place, by any means.