A low cost pencil, paper, and a book is all you really need.
You do know they have the ability to keep notes, right?
It's not one or the other, even in the classroom. Computers didn't kill handwriting or print, and latter don't hinder the use of computers. They all complement each other.
Useless tangent: In my experience (= K-12 + 10 years at universities) the best way to lock things in ones head is to take quick handwritten notes and then reread, correct, and enter them as notes on one's computer.
It's still doable to only have a laptop and use that for everything, if not practical or quick enough in many cases. Pen and paper is nice to have, but it's not 100% required.
But you can certainly teach programming with pencil and paper. Some of the most influential programmers in history of computer science did not do their work on physical computers or used (now) esoteric methods like punch-cards as input. Several of my university CS classes were all pencil and paper, especially midterms and exams.
Even if you get a physical keyboard, are there any mature apps for writing and executing code on the device?
It seems that we need to build mature apps for writing on device.
Yes, yes, I know, Swift Playgrounds. Not convinced.