If you write small programs that interact with other programs, then rearranging them on the floor/table becomes a form of programming (similar to how combining UNIX-style programs in shell scripts is a high-level form of programming).
So even when playing just with the papers, you can be programming.